125 Commits

Author SHA1 Message Date
Yury Semikhatsky
137070d889
Revert "chore: hide setup, store, TestProject.setupMatch, storageStat… (#19756)
…eName (#19442)"

This reverts commit 92dd734e0471cc1b4724ad965e4389ed5235a9f1.
2022-12-28 15:39:31 -08:00
Pavel Feldman
233664bd30
chore: report more fatal errors via reporter (#19640) 2022-12-22 17:31:02 -08:00
Pavel Feldman
675f0eb4a0
chore: report error location for fatal errors (#19610) 2022-12-21 09:36:59 -08:00
Yury Semikhatsky
92dd734e04
chore: hide setup, store, TestProject.setupMatch, storageStateName (#19442) 2022-12-13 22:48:38 -08:00
Yury Semikhatsky
beb15bb504
chore: rename setup => setupMatch (#19309) 2022-12-06 14:55:22 -08:00
Yury Semikhatsky
8660288518
chore: use isProjectSetup instead of set (#19285) 2022-12-06 08:59:28 -08:00
Yury Semikhatsky
f9ef18912d
feat: use setup() instead of test() for project setup (#19281) 2022-12-05 18:15:01 -08:00
Dmitry Gozman
b1b21bdac5
fix(shards): zero-sized shard should not include all tests (#19148)
Fixes #19073.
2022-11-29 16:02:11 -08:00
Yury Semikhatsky
31d45ad4dc
chore(runner): add all setup files in separate pass (#18978) 2022-11-22 16:22:48 -08:00
Yury Semikhatsky
2dc51f6c46
fix(runner): run all setup files when there is test.only (#18967) 2022-11-21 16:33:23 -08:00
Yury Semikhatsky
e1189a96b6
feat(runner): run all setup files if none matched the filter (#18922)
The behavior regarding filters (both in config, command line and .only)
is the following:
- if some of tests match and none of setup match then we'll run all
setup files and all matching tests
- otherwise the filters apply to setup files the same way as to regular
tests
2022-11-21 09:23:28 -08:00
Yury Semikhatsky
c0d0f54a12
feat(runner): allow filtering setup and tests (#18900)
Running `npx playwright test file:123` will have the following behavior
- if only test files match then only matching subset of tests will run
but all setup files will run as well
- if only setup files match the filter then only those setup tests will
run
- if both setup and test files match an error will be thrown
2022-11-18 11:35:29 -08:00
Yury Semikhatsky
0f4b67bc6d
test(cli): setup files are in list-files and test --list output (#18890) 2022-11-17 16:31:04 -08:00
Pavel Feldman
b8bd4f7a48
chore: serialize config as a part of list-files (#18649) 2022-11-08 12:05:00 -08:00
Yury Semikhatsky
1d2fc1e963
feat(runner): project.setup (#18486) 2022-11-01 23:44:30 -07:00
Yury Semikhatsky
7337dd4e28
chore: revert project.{stage,run} (#18462) 2022-10-31 14:04:24 -07:00
Yury Semikhatsky
63c41f88cd
chore: remove stale TODOs (#18331) 2022-10-25 15:21:04 -07:00
Max Schmitt
af38449f42
fix(pwtest): fix focused line with dirname specified (#18189) 2022-10-19 15:05:59 -07:00
Yury Semikhatsky
11eb719d13
feat(runner): project run: "always" (#18160)
Projects marked with `run: 'always'` are non shard-able and run after
failures.
2022-10-18 17:18:45 -07:00
Yury Semikhatsky
08a3a269cd
feat(runner): project.canShard (#18037) 2022-10-12 14:34:22 -07:00
Yury Semikhatsky
3b8f63d703
feat(runner): project.stopOnFailure (#18009) 2022-10-11 17:04:01 -07:00
Yury Semikhatsky
3592269caf
feat(runner): project.stage (#17971) 2022-10-10 17:56:18 -07:00
Yury Semikhatsky
2d72d0ba03
chore: remove config.groups (#17974) 2022-10-10 16:42:48 -07:00
Yury Semikhatsky
ef32cab423
chore: turn RunPhase in a class with helper methods (#17721) 2022-09-30 09:12:06 -07:00
Yury Semikhatsky
51966bc045
fix: --grep and --grep-invert should intersect with config (#17716)
Fixes https://github.com/microsoft/playwright/issues/17405
2022-09-29 16:39:21 -07:00
Yury Semikhatsky
43ccd7bc96
chore(runner): remove watch mode (#17682) 2022-09-29 10:20:21 -07:00
Yury Semikhatsky
9f17ee6871
feat: group filter (#17646) 2022-09-28 18:45:01 -07:00
Yury Semikhatsky
10d7c60abf
feat(runner): project execution schedule (#17548) 2022-09-23 20:01:27 -07:00
Andrey Lushnikov
d431958603
chore: move container integration back to playwright-core (#17487) 2022-09-21 12:45:43 -07:00
Pavel Feldman
872bcc9734
chore: expose driver protocol on the pw object (#17363) 2022-09-15 15:53:18 -07:00
Pavel Feldman
854c783019
chore: project.id, configFile in reporter apis (#17346) 2022-09-14 14:56:28 -07:00
Andrey Lushnikov
e295eea450
fix(ignoreSnapshots): print a notice when ignoreSnapshots option is on (#17317)
<img width="1161" alt="image"
src="https://user-images.githubusercontent.com/746130/190032155-ae454c3e-1a7d-4a64-8cd6-bb27f9075ef3.png">
2022-09-14 09:16:41 -07:00
Andrey Lushnikov
dfcd2a273d
fix(docker): do not pollute stdout when used with JSON reporter (#17315)
This patch moves parts of docker configuration to a plugin so that
it can rely on repoter for stdout.

Drive-by: provide all plugins with reporter in the `setup` callback.
2022-09-13 17:05:37 -07:00
Andrey Lushnikov
fafd9837ba
feat: introduce the --ignore-snapshots option (#17004)
This patch introduces `--ignore-snapshots` Playwright Test CLI option,
and `ignoreSnapshots` configuration argument.
2022-09-01 05:34:36 -07:00
Yury Semikhatsky
11dfd31dd9
feat: --pass-with-no-tests option (#16902) 2022-08-29 15:46:34 -07:00
Dmitry Gozman
baa2ef2700
fix(test runner): show interrupted as yellow (#16396)
Drive-by: fix unreadable character in "duplicate titles" error.
2022-08-09 21:17:30 -07:00
Dmitry Gozman
f6d94f0ac9
chore: update "X fatal errors" message (#16325)
Also, exlude certain errors from triggering this message:
- `no tests found`
- `duplicate test titles are not allowed`
- `--forbid-only found a focused test`
- `Timed out waiting 3600s for the entire test run`
2022-08-05 21:21:43 -07:00
Dmitry Gozman
7a86e140f5
chore: remove TestProject.projectSetup (#16321)
There are better ideas to address this issue in more general manner.
2022-08-05 15:24:30 -07:00
Dmitry Gozman
1f40e3d22b
chore: make Reporter.onExit an internal method (#16280) 2022-08-05 13:41:00 -07:00
Pavel Feldman
7645ac25a0
chore: watch mode straw man (#16127) 2022-08-04 08:09:54 -07:00
Dmitry Gozman
3112edb4ca
feat(test runner): TestProject.projectSetup (#16063)
`projectSetup` is a project-scoped alternative to `globalSetup`.
It is only executed if at least one test from the project is scheduled to run.
2022-08-01 09:01:23 -07:00
Pavel Feldman
d73f9b7b88
chore: experimental test server implementation (#16033) 2022-07-29 11:40:33 -07:00
Pavel Feldman
8c190c3e09
feat(testId): expose persistent test id (#16003) 2022-07-27 20:17:19 -07:00
Dmitry Gozman
cd9dccbe27
fix(test runner): serial suites inside parallel suite should run in parallel (#15769) 2022-07-26 13:38:25 -07:00
Dmitry Gozman
f88b1e9cde
feat(reporters): separate onExit from onEnd (#15926)
This is now used by html reporter to open UI.
2022-07-25 13:20:33 -07:00
Dmitry Gozman
f954891491
fix(runner): globalTeardown without globalSetup should work (#15814) 2022-07-20 12:41:35 -07:00
Ross Wollman
799d4703bc
feature(test-runner): multiple web servers (#15388)
Fixes #8206.

Since #8206 is a long-awaited (~ 1 year old), popular (~ 45 reactions, frequently requested in community channels, etc.), this PR aims to unblock folks.

Notably, we do not innovate on the `webServer` API, despite knowing we're not in love with it. We'll save the innovation for either Plugins or a new `LaunchConfigs` option. (We haven't yet arrived at a Plugin API we like, and instead of launching a new option guessing what the "better" launchConfig API would be, let's wait and see how folks use this new Array-variant of `webServer` which—despite its name—can be used for non-Web Server launches!
2022-07-07 15:27:21 -07:00
Dmitry Gozman
c02d83fb11
fix(runner): filter only after duplicate titles check (#15401) 2022-07-06 12:51:26 -07:00
Pavel Feldman
41529bb1a5
chore: respect sigint in global setup (#14805) 2022-06-12 13:06:00 -07:00
Dmitry Gozman
a7500c18d6
feat(test runner): allow serial suites inside parallel (#14530)
Also works for `fullyParallel` mode.
2022-05-31 15:24:20 -07:00