[Bug]: route.continue() works but route.fetch() with route.fulfill() times out

[dpacmittal]

Version

1.44.1

Steps to reproduce

1. Clone my repo at https://github.com/dpacmittal/playwright-timeout-bug
2. Run npm install
3. Run ts-node index.ts

Expected behavior

route.fetch() then route.fulfill() should behave exactly the same as route.continue() and should not time out.

Actual behavior

route.fetch() times out

Additional context

This happens on chromium (and chrome, chrome-beta). Firefox works fine.

If I add Connection: keep-alive header, it starts working normally.

const response = await route.fetch({
   headers: {
     Connection: "keep-alive",
   },
});

Environment

System:
    OS: macOS 14.2
    CPU: (12) arm64 Apple M2 Max
    Memory: 69.70 MB / 32.00 GB
  Binaries:
    Node: 20.9.0 - ~/.nvm/versions/node/v20.9.0/bin/node
    Yarn: 1.22.21 - ~/.nvm/versions/node/v20.9.0/bin/yarn
    npm: 10.1.0 - ~/.nvm/versions/node/v20.9.0/bin/npm
    bun: 1.0.2 - ~/.bun/bin/bun
  IDEs:
    VSCode: 1.89.1 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.44.1 => 1.44.1

[mxschmitt]

Looks like the website (AkamaiGHost) has a web-application firewall in place which detects us as a bot and closes the connection when sending a request with a specific User-Agent:

import https from 'https';
import url from 'url';

async function main() {
  const parsed = url.parse('https://lppool.catalogsites.net/lf?source=url[file:/MasterSite/CD61/Email/_template_2021/bedding_snob_logo.png]&sink=format[png],quality[96]');
  const options = {
    hostname: parsed.hostname,
    path: parsed.path,
    headers: {
      // Removing the User-Agent will cause the request to end up in 403.
      // With the User-Agent only "request finished" is logged.
      'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36',
    }
  };

  const req = https.request(options, res => {
    console.log('res.statusCode', res.statusCode);
  });
  console.log('sending request');
  req.on('error', e => console.error(e));
  req.on('close', () => console.log('request closed'));
  req.on('finish', () => console.log('request finished'));

  req.end();
}

main();

This also happens when using normal http client from Node.js but doesn't when using fetch() from Node.js.

[yury-s]

route.fetch() then route.fulfill() should behave exactly the same as route.continue() and should not time out.

reques.fetch() only sends headers available on the request when it is intercepted (you can see them in route.request().headers()). In particular, it doesn't include Connection: keep-alive header. If the server's response depends on it for some reason, you can add it manually.

[dpacmittal]

But then why does route.continue() send a Connection: keep-alive header? Is this somehow related to the AcceptCHFrame bug (#10376) because request.fetch() seems to be sending only the ch headers?

[yury-s]

But then why does route.continue() send a Connection: keep-alive header?

In Chromium some headers are added in the Blink layer while others are added in the Network service. The interception sits in between and only sees headers sent by Blink. Changing that in the browser is possible but would require significant redesign of the existing approach.

The other issue you mentioned seems unrelated.

[pavelfeldman]

Why was this issue closed?

Thank you for your contribution to our project. This issue has been closed due to its limited upvotes and recent activity, and insufficient feedback for us to effectively act upon. Our priority is to focus on bugs that reflect higher user engagement and have actionable feedback, to ensure our bug database stays manageable.

Should you feel this closure was in error, please create a new issue and reference this one. We're open to revisiting it given increased support or additional clarity. Your understanding and cooperation are greatly appreciated.