Dmitry Gozman
bee8ed117b
feat(html reporter): preview source code, steps and step errors ( #8598 )
2021-08-31 16:34:52 -07:00
Dmitry Gozman
900362ec0b
fix(test runner): report unhandled rejections during worker teardown ( #8592 )
2021-08-31 10:50:30 -07:00
Dmitry Gozman
37a897c9b4
fix(test runner): report beforeAll timeout instead of hanging ( #8529 )
...
We used to not report fatal error and hang forever because worker
did not run any tests but also did not report any errors.
Also properly show stack-less errors.
2021-08-28 07:19:45 -07:00
Michael Rienstra
ed0c47a06f
fix(test-runner): minor error message typo ( #8489 )
2021-08-27 10:15:16 +02:00
Dmitry Gozman
a479cb6f52
fix(test runner): add an overload for test.skip(title, fn) ( #8454 )
...
We shipped this feature, but forgot to add the right overload to d.ts.
2021-08-25 14:36:36 -07:00
Dmitry Gozman
de85d8bb83
fix(test runner): do not special case test.fail ( #8447 )
...
This makes `test.fail` tests considered as passing when they actually fail:
- Stop restarting the worker.
- Retry when it passes instead of a fail.
- Behaves similar to regular tests in a `describe.serial` suite.
2021-08-25 12:19:50 -07:00
Dmitry Gozman
75fb77355a
test(test runner): check retries with beforeAll failure and multiple tests ( #8413 )
2021-08-24 12:22:16 -07:00
Joel Einbinder
a8a3799e9d
fix(test-runner): use describe name in output dir ( #8282 )
2021-08-24 10:33:40 -04:00
Dmitry Gozman
f9b87268a7
fix(test runner): afterAll error should not mask beforeAll error ( #8358 )
2021-08-23 09:21:40 -07:00
Pavel Feldman
0997c13151
fix(test-runner): do not attach non-existent diff ( #8297 )
2021-08-20 13:40:27 -07:00
Dmitry Gozman
e5be2c9205
feat(test runner): show failure details for flaky test runs ( #8332 )
...
Currently, we just say "foo.spec.ts > my test" is flaky, but do not
show how exactly the failed run went.
2021-08-19 18:20:53 -07:00
Dmitry Gozman
70eca0401d
fix(test runner): avoid internal error for step end without begin ( #8327 )
...
fix(test runner): avoid internal error for step end without begin
Consider the following scenario:
- Test finishes and starts tearing down fixtures.
- Fixture teardown starts a step S and then times out.
- We declare the test finished (with timeout).
- Dispatcher shuts down the worker and spins a new one for a retry.
Additionally, it clears steps information for the test to be
ready for the new retry. Step S information is lost.
- Meanwhile, during worker teardown, the step S does
actually finish (usually with an error), and we send stepEnd for S.
- Dispatcher does not know what to do with step S end and
prints an internal error.
The fix is to ignore certain messages from the shutting down worker that failed.
2021-08-19 17:04:09 -07:00
Pavel Feldman
710cec80a0
feat(test-runner): render step titles ( #8270 )
2021-08-17 16:41:36 -07:00
Pavel Feldman
97ba4f22f3
feat(test-runner): report nested steps ( #8266 )
2021-08-17 13:57:26 -07:00
Dmitry Gozman
2aff06ec73
fix(test runner): make sure tracing is not running on non-retries ( #8232 )
...
When sharing a context between tests and using `'on-first-retry'` we
could end up with tracing still running in non-retried tests. That's
extra overhead without a reason.
2021-08-16 16:46:35 -07:00
Dmitry Gozman
d32d50a906
fix(test runner): make sure we always teardown all fixtures ( #8158 )
...
Even if one of the fixtures throws, we should teardown all of them
so that we can run afterAll hooks.
2021-08-12 09:08:56 -07:00
Pavel Feldman
d2d71c4cdb
fix(reporter): group fixture initialization under before hooks ( #8072 )
2021-08-12 07:58:00 -07:00
Dmitry Gozman
44cdda43fe
feat(test runner): show stdio for failures in terminal reporters ( #8150 )
2021-08-11 16:44:19 -07:00
Dmitry Gozman
052e0e197c
fix(types): allow specifying scope when overriding fixtures ( #8139 )
...
Otherwise it show a confusing error.
2021-08-11 10:44:15 -07:00
Dmitry Gozman
a5e0965087
feat(test runner): implement test.describe.serial ( #8132 )
2021-08-10 21:26:45 -07:00
Joel Einbinder
64da74fba8
feat(test-runner): allow non-ascii characters in the output dir path ( #8093 )
2021-08-10 21:24:35 -07:00
Dmitry Gozman
c9c1ea6546
fix(test runner): disallow use(workerFixture) in describes ( #8119 )
...
Using a worker fixture forces a new worker. This might be unexpected
when part of the test file runs in one worker, and another runs
in another worker. Top-level use of worker fixtures is still fine.
2021-08-10 16:32:32 -07:00
Dmitry Gozman
9d6c7cdf20
fix(test runner): ensure we run after hooks after failures ( #8102 )
2021-08-10 10:54:05 -07:00
Dmitry Gozman
2744cd6c9a
feat(test runner): remove createContext fixture ( #8109 )
...
We can now use `browser.newContext()` or `browser.newPage()` instead.
2021-08-10 09:26:36 -07:00
Dmitry Gozman
3bf3318350
fix(test runner): make sure options, trace and screenshot apply to all contexts ( #8083 )
...
- Uses some auto fixtures to set default options and instrumentation on BrowserType.
- Moves screenshot, trace and video to worker-scoped fixtures.
- Throws in page/context when used from beforeAll/afterAll.
- Plumbs around BrowserType to be accessible from Browser and BrowserContext.
2021-08-09 18:09:11 -07:00
Dmitry Gozman
e638c4597f
fix(test runner): do not swallow afterAll failure ( #8099 )
2021-08-09 14:21:53 -07:00
Dmitry Gozman
87548f94c1
feat(test runner): support test fixtures in beforeAll/afterAll ( #8082 )
...
Each hook gets its own test scope. This is not too useful for
object fixtures like `page` (although one can use a page in
`beforeAll` to save storage state), but much more useful for option
fixtures like `viewport`.
2021-08-09 13:26:33 -07:00
Dmitry Gozman
41949e559e
Revert "feat(test runner): file scope fixtures ( #7969 )" ( #8081 )
...
This reverts commit 1bbf86d060664bd605e615ca2af3f2e7df12ba9b,
leaving small improvements around.
2021-08-09 12:33:16 -07:00
Joel Einbinder
91394b257c
test(test-runner): add tests to show that grep is case insensitive ( #8091 )
2021-08-09 14:05:51 -04:00
Pavel Feldman
290f601dae
feat(expect): introduce explicit default async expect timeout ( #8071 )
2021-08-07 22:08:56 -07:00
Pavel Feldman
3424f59e67
chore(expect): polish matcher names, remote arguable ones ( #8060 )
2021-08-06 16:58:42 -07:00
Dmitry Gozman
5f297b6894
feat(junit reporter): add attachments to stdout ( #8059 )
...
`JUnitReporter` follows the common format for attachments in JUnit reports,
recognized by GitLab and Jenkins among others.
2021-08-06 15:47:54 -07:00
Dmitry Gozman
d846c05619
fix(test runner): make obtainWorker() resolve with null when stopping ( #8018 )
...
This ensures that we properly exit from `Dispatcher.run()`, print
epilogue and set the right exit code.
2021-08-05 15:00:00 -07:00
Dmitry Gozman
1bbf86d060
feat(test runner): file scope fixtures ( #7969 )
...
These are reset after running tests from a single file.
2021-08-04 21:11:02 -07:00
Dmitry Gozman
ea4f42b7ed
feat(report): add video to attachments ( #7976 )
2021-08-03 17:07:57 -07:00
Max Schmitt
385d489b35
feat(test-runner): re-enable web server ( #7906 )
...
Co-authored-by: Joel Einbinder <joel.einbinder@gmail.com>
2021-08-03 23:24:14 +02:00
Pavel Feldman
3b34e57ee4
feat(test-runner): introduce test.step ( #7956 )
2021-08-02 22:11:37 -07:00
Pavel Feldman
5803035c1b
feat(test-runner): introduce steps ( #7952 )
2021-08-02 17:17:20 -07:00
Pavel Feldman
af30d267b6
feat(test-runner): experiemental expect plumbing ( #7926 )
2021-07-30 16:07:02 -07:00
Dmitry Gozman
081b8683a3
feat(test runner): expect(locator) matchers to show a nice error on timeout ( #7935 )
2021-07-30 13:12:49 -07:00
Dmitry Gozman
5a3ebfc9f9
fix(test runner): dot reporter incorrectly splits by 80 ( #7925 )
2021-07-30 01:34:28 -07:00
Pavel Feldman
4163cec93b
feat(test-runner): introduce actionTimeout and navigationTimeout ( #7919 )
2021-07-29 21:03:50 -07:00
Dmitry Gozman
34c0c342fa
fix(test runner): make test order stable when fixtures are changing ( #7923 )
...
We used to sort based on workerHash, and that changes depending on
the exact worker fixtures list. Now we replace workerHash with
an ordinal when constructing the TestGroup list to preserve the
natural order.
2021-07-29 18:27:47 -07:00
Dmitry Gozman
40901e8b9a
feat(test runner): test.skip(title, testFunction) syntax ( #7922 )
2021-07-29 14:33:37 -07:00
Dmitry Gozman
dd0b089d13
feat(test runner): createContext fixture for multi-context scenarios ( #7779 )
2021-07-29 14:03:58 -07:00
Joel Einbinder
626dd23ce1
test(test-runner): unflake override-timeout.spec.ts ( #7869 )
2021-07-29 13:34:46 -07:00
Dmitry Gozman
82d2ec0b2d
test: unflake "max-failures should stop workers" ( #7918 )
2021-07-29 13:32:44 -07:00
Pavel Feldman
1807142eb7
feat(expect): even more matchers ( #7902 )
2021-07-29 07:33:19 -07:00
Pavel Feldman
3187ffdebf
feat(expect): add more matchers ( #7891 )
2021-07-28 15:44:44 -07:00
Dmitry Gozman
3f0485486d
feat(test runner): show errors from interrupted tests when available ( #7874 )
...
This shows the exact operation that is timing out (like click) when
user hits Ctrl+C.
2021-07-28 15:43:37 -07:00
