83 Commits

Author SHA1 Message Date
Simon Knott
a8df750a48
fix(har): account for reused sockets (#33087)
Closes https://github.com/microsoft/playwright/issues/32960

If the socket is reused, the connect and DNS timings are set to -1,
because that timing doesn't apply to the current request. The time
between request start and the socket being free is counted as `blocked`.
2024-10-14 17:22:29 +02:00
Simon Knott
042161e1ce
Reapply "fix(har timing): record connect timing for proxied connections" (#32855) (#33003)
This reapplies what we reverted in
https://github.com/microsoft/playwright/pull/32989.

Max and me debugged this, and found that the test failures come from
SOCKS proxy now preferring IPv6 over IPv4. We've updated the tests and
made sure that this doesn't mask any breaking change.

I'm enabling CQ1 to make sure we don't oversee any other CI failures.
2024-10-08 14:17:50 +02:00
Simon Knott
1b589c4bd3
Revert "fix(har timing): record connect timing for proxied connections" (#32989)
Reverts microsoft/playwright#32855. This broke two tests on main, and we
don't yet know how to fix it other than downgrading.
2024-10-08 10:13:21 +02:00
Simon Knott
9a6f03eb87
fix(fetch): listener leaks on Socket (#32956)
Closes https://github.com/microsoft/playwright/issues/32951

`node:http` reuses TCP Sockets under the hood. We weren't cleaning up
our listeners, leading to the `MaxListenersExceededWarning`.

This PR adds cleanup logic. It also raises the warning threshhold, so
that it doesn't trigger until there's 100 concurrent requests over the
same socket.
2024-10-07 18:43:25 +02:00
Simon Knott
de4a4d1ce1
fix(har timing): record connect timing for proxied connections (#32855)
Fixes a bug discovered in
https://github.com/microsoft/playwright/pull/32647. When using http
proxy, the `connect` event isn't emitted so we don't populate
`tcpConnectionAt`. The updated version of `https-proxy-agent` emits a
`proxyConnect` as a replacement, so this PR updates and listens to that
event.
For socks proxies, the `on("socket")` event is emitted once the SOCKS
connection is established, which is the equivalent of having a TCP
connection available.

---------

Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2024-10-07 09:59:13 +02:00
Max Schmitt
daac0ddd24
fix(fetch): fallback to given URL if baseURL is invalid (#32911) 2024-10-01 22:43:32 +02:00
Max Schmitt
d4eecafa8a
test: listen always on 127.0.0.1 for client certificate tests (#32677) 2024-09-18 17:09:08 +02:00
Simon Knott
825df6c074
feat(har): record serverIPAddress for API requests (#32660)
Discovered working on
https://github.com/microsoft/playwright/pull/32658. We're recording the
remote server address for browser requests, but not for API requests.
This PR adds that for API requests.
2024-09-18 14:51:42 +02:00
Simon Knott
8a97050822
feat(har): record securityDetails for API Requests (#32664)
While working on https://github.com/microsoft/playwright/pull/32658 I
discovered that we're recording `securityDetails` for browser requests,
but not for API requests. This PR fixes that.
2024-09-18 08:18:47 +02:00
Simon Knott
751b939d3a
feat(fetch): record timings (#32613)
Related to https://github.com/microsoft/playwright/issues/19621

Adds some instrumentation to collect timings for `APIRequestContext`
requests and adds them to the HAR trace. Doesn't yet expose them via an
API, but makes our `Duration` field in the trace viewer show a nice
duration:

<img width="1392" alt="Screenshot 2024-09-14 at 11 46 04"
src="https://github.com/user-attachments/assets/8020382d-9494-4634-9cfd-22b6f4a1d770">


I'm gonna add it to our API in a separate PR.

---------

Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2024-09-17 16:11:21 +02:00
Max Schmitt
21d162c945
feat(client-certificates): add support for proxies (#32611)
Fixes https://github.com/microsoft/playwright/issues/32370
2024-09-16 17:57:33 +02:00
Yury Semikhatsky
34876e9291
chore: cookies in intercepted bidi requests (#32623) 2024-09-13 18:29:35 -07:00
Max Schmitt
79cba7d704
chore: introduce option overrides on context/browser (#32606) 2024-09-13 17:34:34 +02:00
Max Schmitt
e5d6ee5bd8
chore: merge fetch params on server side (#32518)
https://github.com/microsoft/playwright-python/pull/2546#discussion_r1750090592
2024-09-09 22:28:08 +02:00
Max Schmitt
faf4853259
chore: validate client-certificates on context creation (#32168) 2024-08-19 09:02:14 +02:00
Kuba Janik
308381eeae
feat: allow URLSearchParams and string as params in APIRequestContext (#32120) 2024-08-12 14:22:03 -07:00
Max Schmitt
613ccb8d5b
chore(client-certificates): rewrite error for unsupported PFX errors (#32008) 2024-08-05 14:42:29 +02:00
Max Schmitt
f17de8222f
chore: run client-certificate tests in service mode (#31973) 2024-08-02 08:34:28 +02:00
Yury Semikhatsky
db0980a850
chore(fetch): include response text into failOnStatusCode errors (#31978)
Fixes https://github.com/microsoft/playwright/issues/31834
2024-08-01 17:53:43 -07:00
Max Schmitt
7e7319da7d
fix(client-certificates): don't use proxy when using BrowserContext.request (#31898) 2024-07-29 16:44:53 +02:00
Pavel Feldman
c74843a914
chore(fetch): pass flush and finishFlush options to brotli (#31833)
Fixes https://github.com/microsoft/playwright/issues/31814
2024-07-23 18:28:34 -07:00
Max Schmitt
c4862c022c
chore: client certificates api review (#31826) 2024-07-23 22:56:36 +02:00
Max Schmitt
708def8804
fix(client-certificates): keep ignoreHTTPSErrors false by default (#31760) 2024-07-18 22:37:11 +02:00
Max Schmitt
9569cb5c1e
feat: support client certificates (#31529)
Signed-off-by: Max Schmitt <max@schmitt.mx>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2024-07-12 11:42:24 +02:00
Yury Semikhatsky
95fc2b8a8b
feat(fetch): maxRetries for fetch (#31386)
Fixes https://github.com/microsoft/playwright/issues/30978
2024-06-19 18:10:14 -07:00
Yury Semikhatsky
6067b78f88
chore: http credentials send immeidately/unauthorized enum (#31076)
Reference https://github.com/microsoft/playwright-internal/issues/205
Reference https://github.com/microsoft/playwright/issues/30534
2024-05-30 10:19:56 -07:00
Max Schmitt
a93ad3dade
fix(fetch): allow UTF-8 in Location header (#30904) 2024-05-21 09:15:33 +02:00
Dmitry Gozman
776b04e5ea
feat: APIRequestContext dispose reason (#30765)
Similarly to page.close, we pass test-runner specific reason to
facilitate better error messages.

```
  1) a.test.ts:10:11 › test

    Error: apiRequestContext.fetch: Fixture { request } from beforeAll cannot be reused in a test.
      - Recommended fix: use a separate { request } in the test.
      - Alternatively, manually create APIRequestContext in beforeAll and dispose it in afterAll.
    See https://playwright.dev/docs/api-testing#sending-api-requests-from-ui-tests for more details.

       9 |
      10 |       test('test', async () => {
    > 11 |         await context.fetch('http://example.com');
         |                       ^
      12 |       });
      13 |
```

Closes #29260.
2024-05-13 18:51:30 -07:00
Yury Semikhatsky
d5b387159a
feat(fetch): sendImmediately (#30627)
Fixes https://github.com/microsoft/playwright/issues/30534
2024-05-02 16:30:12 -07:00
Pavel Feldman
0f2de59b7c
chore: remove graceful browser.close cleanup (#27916) 2023-11-01 20:17:10 -07:00
Playwright Service
5262e5ab35
feat(chromium-tip-of-tree): roll to r1159 (#27605) 2023-10-17 22:41:23 +02:00
Pavel Feldman
a54dbfdadf
chore: plumb the target close reason when test fails (#27640) 2023-10-16 20:32:13 -07:00
Yury Semikhatsky
4e845e7b1d
fix(fetch): JSON.stringify on client (#27644)
Fixes https://github.com/microsoft/playwright/issues/27602
2023-10-16 16:33:49 -07:00
Yury Semikhatsky
88038f1b00
fix: do not throw from fetch when response has invalid cookie (#27192)
Cookie value is limited by 4096 characters in the browsers. If
setCookies failed we try setting each cookie individually just in case
only some of them are bad.

Fixes https://github.com/microsoft/playwright/issues/27165
2023-09-19 16:18:16 -07:00
Yury Semikhatsky
700b310150
fix(fetch): restore from empty storage state (#27025)
Fixes #26833
2023-09-12 13:11:18 -07:00
Yury Semikhatsky
501ed32856
fix(fetch): update host header on redirect (#26750)
Fixes https://github.com/microsoft/playwright/issues/26743
2023-08-28 12:42:50 -07:00
Yury Semikhatsky
59d5198d17
fix(fetch): handle negative max-age and expires attributes (#24311)
Fixes #24221
2023-07-20 15:42:52 -07:00
Pavel Feldman
bd090e67df
lint: prevent object string concatenation (#23172) 2023-05-19 15:17:43 -07:00
Yury Semikhatsky
75e106745f
fix(fetch): support proxy.bypass (#22914)
Fixes #21916
2023-05-09 14:51:49 -07:00
Sébastien Richert
591e4ea976
feat: Restrain sending http credentials on a specific origin (#20374)
For security purpose, we would like to restrain sending HTTP credentials
to only the specified server. The idea is to give the ability to specify
a origin (scheme://host:port) additionally to current pair
username/password. When an authorization response is received from
servers, the credentials are sent only if the server origin in the
request matches case insensitive the specified origin.
2023-03-27 08:52:00 -07:00
Dmitry Gozman
0794cb1486
fix(fetch): preserve case in header names (#21543)
Fixes #21492.
2023-03-10 08:58:12 -08:00
Max Schmitt
ae369144b6
chore: consolidate http/https fetching (#21104)
Fixes https://github.com/microsoft/playwright/issues/20784
Supersedes https://github.com/microsoft/playwright/pull/21076
2023-02-22 17:09:56 +01:00
Pavel Feldman
d1c161ce99
chore: merge the util exports (#20110) 2023-01-13 13:50:38 -08:00
Max Schmitt
c339e1615b
fix(fetch): handle invalid redirect URL (#19890)
Fixes https://github.com/microsoft/playwright/issues/19879.

This part is then similar to how node-fetch is doing it:
55a4870ae5/src/index.js (L152-L159)

node-fetch also throws as of today with this URL. Before in Python it
was stalling, because the error was written to stdout and on Windows the
stdout wasn't working. On Node.js it ended up in an unhandled exception.
2023-01-06 10:22:17 -08:00
Yury Semikhatsky
10ccfa9517
feat(fetch): happy eyeballs (#19902)
Fixes #18790
2023-01-05 14:39:49 -08:00
Yury Semikhatsky
d3f41eaa47
fix(fetch): cookie with domain=localhost (#18998)
Fixes https://github.com/microsoft/playwright/issues/18362
2022-11-23 09:22:49 -08:00
Dmitry Gozman
503f8f51dc
fix(reuse): stop pending operations upon reuse/disconnect (#18997) 2022-11-22 15:21:20 -08:00
Max Schmitt
be67189a54
fix(fetch): throw on unexpected end of file brotli requests (#18223)
https://github.com/microsoft/playwright/issues/18190
2022-10-24 12:51:45 -07:00
Yury Semikhatsky
9b35a8071f
fix(fetch): support SameSite attribute (#17748)
Fixes https://github.com/microsoft/playwright/issues/17398
2022-09-30 15:01:59 -07:00
Pavel Feldman
df143031e7
chore: move protocol and trace types into the top-level packages (#17486) 2022-09-20 18:41:51 -07:00