1493 Commits

Author SHA1 Message Date
Dmitry Gozman
10ce7af411
fix(steps): step should not get an unrelated error (#22773)
Previously, we would use any error that was added during the step
execution as an error for this particular step.

This produces false positives, for example failing `page.click` call
that happened during `context` teardown was producing an error and
marking teardown is failed. However, in reality, the test itself has
failed, while teardown has not.

New approach uses test step hierarchy to inherit errors from child steps
to the parent step. This does not regress the original fix where
`expect.soft` errors are surfaced in the parent step.

See also #19973 that introduced the original logic.
2023-05-02 18:50:00 -07:00
Dmitry Gozman
dee0f2791e
test: unflake "should give enough time for fixture teardown" (#22763) 2023-05-02 17:40:10 -07:00
Dmitry Gozman
8f09935e81
fix(test runner): separate test fixture scope for beforeAll/afterAll hooks (#22746)
There was a single test fixture scope that covers all hooks, modifiers
and test function. Now beforeAll-like modifiers, beforeAll and afterAll
hooks get a scope each.

Fixes #22256.
2023-05-02 11:04:51 -07:00
Pavel Feldman
5ad75f92f7
test: unflake basic timeout test (#22745) 2023-05-02 09:14:57 -07:00
Sander
5b69c4cf52
chore: bump vite to 4.3.3 (#22711) 2023-05-01 15:19:49 -07:00
Pavel Feldman
70011c82f7
chore: unflake ui tests (3) (#22741) 2023-05-01 15:18:34 -07:00
Yury Semikhatsky
a4e90f20dc
fix(runner): do not hang on worker exit before tests (#22742) 2023-05-01 14:54:48 -07:00
Dmitry Gozman
fcd966c4e5
chore: make _setupArtifacts a worker-scoped fixture (#22739)
This should unblock having separate test-fixture scopes for hooks and
test.
2023-05-01 13:53:15 -07:00
Yury Semikhatsky
59079d94ca
fix: call onExit when merging reports (#22718) 2023-05-01 12:58:40 -07:00
Pavel Feldman
297fea0826
chore: purify the junit reporter (#22624) 2023-05-01 09:15:08 -07:00
Alex Neo
26cad0b31f
feat(html): matching labels with dash&spec symbol (#22709)
Fixes #22700
2023-05-01 09:13:30 -07:00
Pavel Feldman
a01df2ff5b
chore: render live trace for the serial mode (#22715)
Fixes https://github.com/microsoft/playwright/issues/22655
2023-04-28 17:47:57 -07:00
Dmitry Gozman
dbb218a9d5
feat: project.teardown that runs after all dependents have finished (#22696)
This replicates globalTeardown in the deps world.

Fixes #21914.
2023-04-28 14:27:08 -07:00
Pavel Feldman
2a675026de
chore: add image diff test to ui mode (#22637) 2023-04-28 14:18:46 -07:00
Pavel Feldman
e9373dfb6e
chore: make client-side instrumentation non-nullable (#22694) 2023-04-28 08:57:43 -07:00
Pavel Feldman
b555d33e38
chore: roll the stable test runner (#22701) 2023-04-28 08:57:19 -07:00
Alex Neo
7937699b28
fix(html): label not matched if first with describe (#22680) 2023-04-27 20:04:54 -07:00
Yury Semikhatsky
f37f5fc61c
chore: comma separated reporter names, reporter arg from config (#22693) 2023-04-27 15:16:18 -07:00
Dmitry Gozman
223baa3393
fix(html): keep tests in the declaration order (#22690)
Fixes #22143.
2023-04-27 13:54:15 -07:00
Yury Semikhatsky
82670147b4
chore: support multiple merged reports (#22672) 2023-04-27 09:15:24 -07:00
Max Schmitt
7b27d70d8a
feat(web-server): add stdout: "pipe"|"ignore" option (#22564)
Fixes https://github.com/microsoft/playwright/issues/22454
2023-04-26 23:39:42 +02:00
Pavel Feldman
7fdd7a20fb
chore: fix soft after poll (#22642) 2023-04-26 08:50:10 -07:00
Pavel Feldman
6b487ff49d
chore: remove cli from playwright-core (#22612)
Fixes https://github.com/microsoft/playwright/issues/22599
2023-04-25 11:19:37 -07:00
Pavel Feldman
a1007bbe2c
chore: introduce expect.configure (#22533) 2023-04-25 10:29:56 -07:00
Yury Semikhatsky
24478be565
feat: preserve attachments in blob reports (#22605)
Attachments with `path` are stored as `resources/sha1.ext` zip entries
and extracted under `report-dir/tmp` when merging. This way normal
fs.readFile keeps working as before even thought the file path is
different. The clients should rely on `attachment.name` instead of
`attachment.path` when deriving user visible titles in the UI. If this
turns out not to be the case we can reconsider later.

#10437
2023-04-24 17:34:09 -07:00
Pavel Feldman
bbc47ba315
chore: ensure web assertions are merged in trace (#22544) 2023-04-21 10:07:23 -07:00
Pavel Feldman
d95268ffc0
chore: remove refined title (#22541) 2023-04-20 20:34:07 -07:00
Pavel Feldman
d9a7aa584c
chore: bump timeouts in ui mode tests (#22526) 2023-04-20 16:47:08 -07:00
Yury Semikhatsky
9ea9adf8ff
test: mark reporter-blob tests as slow (#22530)
They run several playwright instances for each shard and then open
merged html report which takes considerable time
2023-04-20 09:36:38 -07:00
Pavel Feldman
0c70f6900e
chore: fix ui mode w/ multiple contexts (#22514)
Fixes: https://github.com/microsoft/playwright/issues/21895
2023-04-20 08:19:00 -07:00
Pavel Feldman
cdaccdeaf7
chore: speculative change to fix the ui mode tests stress (#22510) 2023-04-20 08:18:09 -07:00
Pavel Feldman
8f14517de2
chore: allow collapsing all (#22512)
Fixes https://github.com/microsoft/playwright/issues/21915
2023-04-19 18:16:18 -07:00
Dmitry Gozman
957ec0067f
chore: separate concerns of testinfo helpers (#22491) 2023-04-19 17:31:07 -07:00
Pavel Feldman
a45f04568b
chore: rename watchmode to uimode tsx (#22511) 2023-04-19 16:51:42 -07:00
Pavel Feldman
e7b9c08833
chore: should not spill toPass between tests (#22473) 2023-04-19 14:45:58 -07:00
Dmitry Gozman
bf0fab4927
fix(ui mode): ignore repeatEach (#22505)
Fixes #22498.
2023-04-19 14:16:12 -07:00
Max Schmitt
49c9284bc7
Revert "feat(html reporter): render multiple annotations of the same type together (#21580)" (#22466)
Fixes https://github.com/microsoft/playwright/issues/22323
2023-04-18 20:43:07 +02:00
mindaugasm
be79ee0450
feat(html-report): add attachmentsBaseURL option (#22212)
Fixes https://github.com/microsoft/playwright/issues/21636
2023-04-18 11:25:11 -07:00
Yury Semikhatsky
2ea214d6f0
fix: preserve trace if context if closed manually during test run (#22442)
Reference #22122, #22120
2023-04-18 09:02:33 -07:00
Pavel Feldman
8d69fbacf7
chore: introduce ct-core to hard-depend on vite (#22437)
Fixes https://github.com/microsoft/playwright/issues/22233
2023-04-17 16:19:21 -07:00
Dmitry Gozman
bf661535a6
fix(testMatch): do not count test.ts and spec.ts as test files by default (#22440) 2023-04-17 12:57:33 -07:00
Andrey Lushnikov
8bb708be70
test: unflake inspector-cli tests (#22347)
This patch:
- changes the `childProcess` fixture to reliably SIGKILL all descendants
  (children and grand-children, regardless of their process group).
This is achieved using the `ps` command to build the process tree, and
then send
  `SIGKILL` to the descendant process groups.
- changes the `runCLI` fixture to **not** auto-close codegen by default;
  the `childProcess` fixture will clean up all processes. This makes
sure that all `runCLI.waitFor()` commands actually wait until the
necessary
  output.
- for a handful of tests that do actually want to auto-close codegen,
  introduce an optional `autoCloseWhen` flag for the `runCLI` fixture
that makes sure to close the codegen once a certain output was reached.
2023-04-12 09:37:24 -07:00
Pavel Feldman
c5aeab4d7e
revert(20509, 20596): expect.toPass is broken with these (#22254)
Reverts https://github.com/microsoft/playwright/pull/20509 and
https://github.com/microsoft/playwright/pull/20596
Fixes #22215
2023-04-10 17:39:49 -07:00
Yury Semikhatsky
2af3f486c4
fix: do not load trace data for passing tests (#22311)
Fixes #22122
Fixes #22120
2023-04-10 13:29:55 -07:00
Yury Semikhatsky
1efa8de526
test: fix reporter blob tests on bots (#22281) 2023-04-07 17:48:54 -07:00
Yury Semikhatsky
8d2502ee62
test: make page-leaks tests pass in electron (#22277)
They've been failing
https://devops.playwright.dev/flakiness.html#filter_spec=page%2Fpage-leaks.spec.ts&test_parameter_filters=%5B%5D&timestamp=1680901928195
2023-04-07 17:20:49 -07:00
Pavel Feldman
eed5b4c83b
chore: process stdio buffers (#22270)
Fixes https://github.com/microsoft/playwright/issues/22265
2023-04-07 13:50:15 -07:00
Yury Semikhatsky
d59e0e10ce
feat: blob reporter (#22244)
#10437
2023-04-07 13:47:52 -07:00
Pavel Feldman
159e71982e
chore: render failed steps in the basic reporters (#22200)
Fixes #20532
2023-04-05 13:03:42 -07:00
Johannes Loher
bd698efaef
fix(webServer): follow relative redirects when checking the url (#22035)
Fixes https://github.com/microsoft/playwright/issues/22144
2023-04-05 10:39:35 +02:00