Dmitry Gozman
e2b092c1a0
feat(html reporter): show log for pw:api and expect steps ( #8692 )
2021-09-03 13:08:17 -07:00
Dmitry Gozman
2b4a93972d
test: use ToT test runner for non-test-runner tests ( #8671 )
2021-09-03 11:22:25 -07:00
Dmitry Gozman
620712a5d9
feat(expect): support array of RegExp objects in toHaveText/toHaveClass ( #8667 )
2021-09-02 15:48:04 -07:00
Dmitry Gozman
e691b649de
feat(test runner): describe.parallel ( #8662 )
2021-09-02 15:42:07 -07:00
Fokke Zandbergen
0ae38b5aec
fix(test-runner): WebServer: use socket for reuseExistingServer to detect servers that aren't using SO_REUSEADDR ( #8537 )
2021-09-02 18:39:41 +02:00
Yury Semikhatsky
bafa426231
feat(runner): support multiple names in project filter ( #8600 )
2021-09-02 09:29:55 -07:00
Dmitry Gozman
b1260602ac
docs: update global setup docs ( #8637 )
...
Changing example to "authenticate once", and also updating the auth doc.
Adding a test with the same setup.
2021-09-01 15:35:46 -07:00
Dmitry Gozman
4f4cf448c2
fix(test runner): generate unique outputDir for beforeAll/afterAll ( #8633 )
2021-09-01 13:41:35 -07:00
Dmitry Gozman
1a9215a6ec
fix(html reporter): do not throw when attachment is actually missing ( #8630 )
2021-09-01 12:20:28 -07:00
Dmitry Gozman
b8f8ca7493
feat(tracing): tracing.{start,stop}Chunk instead of tracing._export ( #8521 )
2021-08-31 17:03:31 -07:00
Dmitry Gozman
bee8ed117b
feat(html reporter): preview source code, steps and step errors ( #8598 )
2021-08-31 16:34:52 -07:00
Dmitry Gozman
900362ec0b
fix(test runner): report unhandled rejections during worker teardown ( #8592 )
2021-08-31 10:50:30 -07:00
Dmitry Gozman
37a897c9b4
fix(test runner): report beforeAll timeout instead of hanging ( #8529 )
...
We used to not report fatal error and hang forever because worker
did not run any tests but also did not report any errors.
Also properly show stack-less errors.
2021-08-28 07:19:45 -07:00
Michael Rienstra
ed0c47a06f
fix(test-runner): minor error message typo ( #8489 )
2021-08-27 10:15:16 +02:00
Dmitry Gozman
a479cb6f52
fix(test runner): add an overload for test.skip(title, fn) ( #8454 )
...
We shipped this feature, but forgot to add the right overload to d.ts.
2021-08-25 14:36:36 -07:00
Dmitry Gozman
de85d8bb83
fix(test runner): do not special case test.fail ( #8447 )
...
This makes `test.fail` tests considered as passing when they actually fail:
- Stop restarting the worker.
- Retry when it passes instead of a fail.
- Behaves similar to regular tests in a `describe.serial` suite.
2021-08-25 12:19:50 -07:00
Dmitry Gozman
75fb77355a
test(test runner): check retries with beforeAll failure and multiple tests ( #8413 )
2021-08-24 12:22:16 -07:00
Joel Einbinder
a8a3799e9d
fix(test-runner): use describe name in output dir ( #8282 )
2021-08-24 10:33:40 -04:00
Dmitry Gozman
f9b87268a7
fix(test runner): afterAll error should not mask beforeAll error ( #8358 )
2021-08-23 09:21:40 -07:00
Pavel Feldman
0997c13151
fix(test-runner): do not attach non-existent diff ( #8297 )
2021-08-20 13:40:27 -07:00
Dmitry Gozman
e5be2c9205
feat(test runner): show failure details for flaky test runs ( #8332 )
...
Currently, we just say "foo.spec.ts > my test" is flaky, but do not
show how exactly the failed run went.
2021-08-19 18:20:53 -07:00
Dmitry Gozman
70eca0401d
fix(test runner): avoid internal error for step end without begin ( #8327 )
...
fix(test runner): avoid internal error for step end without begin
Consider the following scenario:
- Test finishes and starts tearing down fixtures.
- Fixture teardown starts a step S and then times out.
- We declare the test finished (with timeout).
- Dispatcher shuts down the worker and spins a new one for a retry.
Additionally, it clears steps information for the test to be
ready for the new retry. Step S information is lost.
- Meanwhile, during worker teardown, the step S does
actually finish (usually with an error), and we send stepEnd for S.
- Dispatcher does not know what to do with step S end and
prints an internal error.
The fix is to ignore certain messages from the shutting down worker that failed.
2021-08-19 17:04:09 -07:00
Pavel Feldman
710cec80a0
feat(test-runner): render step titles ( #8270 )
2021-08-17 16:41:36 -07:00
Pavel Feldman
97ba4f22f3
feat(test-runner): report nested steps ( #8266 )
2021-08-17 13:57:26 -07:00
Dmitry Gozman
2aff06ec73
fix(test runner): make sure tracing is not running on non-retries ( #8232 )
...
When sharing a context between tests and using `'on-first-retry'` we
could end up with tracing still running in non-retried tests. That's
extra overhead without a reason.
2021-08-16 16:46:35 -07:00
Dmitry Gozman
d32d50a906
fix(test runner): make sure we always teardown all fixtures ( #8158 )
...
Even if one of the fixtures throws, we should teardown all of them
so that we can run afterAll hooks.
2021-08-12 09:08:56 -07:00
Pavel Feldman
d2d71c4cdb
fix(reporter): group fixture initialization under before hooks ( #8072 )
2021-08-12 07:58:00 -07:00
Dmitry Gozman
44cdda43fe
feat(test runner): show stdio for failures in terminal reporters ( #8150 )
2021-08-11 16:44:19 -07:00
Dmitry Gozman
052e0e197c
fix(types): allow specifying scope when overriding fixtures ( #8139 )
...
Otherwise it show a confusing error.
2021-08-11 10:44:15 -07:00
Dmitry Gozman
a5e0965087
feat(test runner): implement test.describe.serial ( #8132 )
2021-08-10 21:26:45 -07:00
Joel Einbinder
64da74fba8
feat(test-runner): allow non-ascii characters in the output dir path ( #8093 )
2021-08-10 21:24:35 -07:00
Dmitry Gozman
c9c1ea6546
fix(test runner): disallow use(workerFixture) in describes ( #8119 )
...
Using a worker fixture forces a new worker. This might be unexpected
when part of the test file runs in one worker, and another runs
in another worker. Top-level use of worker fixtures is still fine.
2021-08-10 16:32:32 -07:00
Dmitry Gozman
9d6c7cdf20
fix(test runner): ensure we run after hooks after failures ( #8102 )
2021-08-10 10:54:05 -07:00
Dmitry Gozman
2744cd6c9a
feat(test runner): remove createContext fixture ( #8109 )
...
We can now use `browser.newContext()` or `browser.newPage()` instead.
2021-08-10 09:26:36 -07:00
Dmitry Gozman
3bf3318350
fix(test runner): make sure options, trace and screenshot apply to all contexts ( #8083 )
...
- Uses some auto fixtures to set default options and instrumentation on BrowserType.
- Moves screenshot, trace and video to worker-scoped fixtures.
- Throws in page/context when used from beforeAll/afterAll.
- Plumbs around BrowserType to be accessible from Browser and BrowserContext.
2021-08-09 18:09:11 -07:00
Dmitry Gozman
e638c4597f
fix(test runner): do not swallow afterAll failure ( #8099 )
2021-08-09 14:21:53 -07:00
Dmitry Gozman
87548f94c1
feat(test runner): support test fixtures in beforeAll/afterAll ( #8082 )
...
Each hook gets its own test scope. This is not too useful for
object fixtures like `page` (although one can use a page in
`beforeAll` to save storage state), but much more useful for option
fixtures like `viewport`.
2021-08-09 13:26:33 -07:00
Dmitry Gozman
41949e559e
Revert "feat(test runner): file scope fixtures ( #7969 )" ( #8081 )
...
This reverts commit 1bbf86d060664bd605e615ca2af3f2e7df12ba9b,
leaving small improvements around.
2021-08-09 12:33:16 -07:00
Joel Einbinder
91394b257c
test(test-runner): add tests to show that grep is case insensitive ( #8091 )
2021-08-09 14:05:51 -04:00
Pavel Feldman
290f601dae
feat(expect): introduce explicit default async expect timeout ( #8071 )
2021-08-07 22:08:56 -07:00
Pavel Feldman
3424f59e67
chore(expect): polish matcher names, remote arguable ones ( #8060 )
2021-08-06 16:58:42 -07:00
Dmitry Gozman
5f297b6894
feat(junit reporter): add attachments to stdout ( #8059 )
...
`JUnitReporter` follows the common format for attachments in JUnit reports,
recognized by GitLab and Jenkins among others.
2021-08-06 15:47:54 -07:00
Dmitry Gozman
d846c05619
fix(test runner): make obtainWorker() resolve with null when stopping ( #8018 )
...
This ensures that we properly exit from `Dispatcher.run()`, print
epilogue and set the right exit code.
2021-08-05 15:00:00 -07:00
Dmitry Gozman
1bbf86d060
feat(test runner): file scope fixtures ( #7969 )
...
These are reset after running tests from a single file.
2021-08-04 21:11:02 -07:00
Dmitry Gozman
ea4f42b7ed
feat(report): add video to attachments ( #7976 )
2021-08-03 17:07:57 -07:00
Max Schmitt
385d489b35
feat(test-runner): re-enable web server ( #7906 )
...
Co-authored-by: Joel Einbinder <joel.einbinder@gmail.com>
2021-08-03 23:24:14 +02:00
Pavel Feldman
3b34e57ee4
feat(test-runner): introduce test.step ( #7956 )
2021-08-02 22:11:37 -07:00
Pavel Feldman
5803035c1b
feat(test-runner): introduce steps ( #7952 )
2021-08-02 17:17:20 -07:00
Pavel Feldman
af30d267b6
feat(test-runner): experiemental expect plumbing ( #7926 )
2021-07-30 16:07:02 -07:00
Dmitry Gozman
081b8683a3
feat(test runner): expect(locator) matchers to show a nice error on timeout ( #7935 )
2021-07-30 13:12:49 -07:00
