28 Commits

Author SHA1 Message Date
Dmitry Gozman
0d54afab9c
feat(test runner): show beforeAll/afterAll hooks similar to tests (#10923)
Reporters now get notified about hooks start/end/steps.
2021-12-15 10:39:49 -08:00
Dmitry Gozman
9491f6652d
fix(steps): do not show unnecessary steps for internal waitForEventInfo (#10889) 2021-12-13 13:32:53 -08:00
Dmitry Gozman
de6af16744
test: add reporter test with grep (#10678) 2021-12-02 11:07:45 -08:00
Dmitry Gozman
4bb123d4b7
fix(test runner): route more errors to reporter (#10263)
For example, top-level errors in files or global setup issues.
2021-11-12 06:47:41 -08:00
Pavel Feldman
6a3e08d1ac
fix(reporter): fix locator stacks to hide internal detail (#9693) 2021-10-21 16:10:47 -07:00
Dmitry Gozman
6d554a5e30
feat(reporter): expose more apis (#9603) 2021-10-19 08:38:04 -07:00
Pavel Feldman
e37660b068
feat(api): expose step location (#9602) 2021-10-18 21:06:18 -07:00
Pavel Feldman
241411ad42
chore: render expect in trace viewer (#9141) 2021-09-27 09:19:59 -07:00
Pavel Feldman
0908dc98c8
chore: migrate expect(locator).toMatchText to protocol (#9117) 2021-09-23 16:46:46 -07:00
Pavel Feldman
63ff405e6e
fix(inspector): stop on all snapshottable actions (#8990) 2021-09-17 15:24:15 -07:00
Pavel Feldman
14bc663742
chrome: don't nest pw:api steps (#8932) 2021-09-15 11:34:23 -07:00
Pavel Feldman
b76e993951
chore: remove log from htmlreport (#8898) 2021-09-13 18:07:15 -07:00
Dmitry Gozman
440651e05c
test(test runner): calling test.step() from fixture (#8899) 2021-09-13 17:49:58 -07:00
Dmitry Gozman
e2b092c1a0
feat(html reporter): show log for pw:api and expect steps (#8692) 2021-09-03 13:08:17 -07:00
Dmitry Gozman
bee8ed117b
feat(html reporter): preview source code, steps and step errors (#8598) 2021-08-31 16:34:52 -07:00
Dmitry Gozman
70eca0401d
fix(test runner): avoid internal error for step end without begin (#8327)
fix(test runner): avoid internal error for step end without begin

Consider the following scenario:
- Test finishes and starts tearing down fixtures.
- Fixture teardown starts a step S and then times out.
- We declare the test finished (with timeout).
- Dispatcher shuts down the worker and spins a new one for a retry.
  Additionally, it clears steps information for the test to be
  ready for the new retry. Step S information is lost.
- Meanwhile, during worker teardown, the step S does
  actually finish (usually with an error), and we send stepEnd for S.
- Dispatcher does not know what to do with step S end and
  prints an internal error.

The fix is to ignore certain messages from the shutting down worker that failed.
2021-08-19 17:04:09 -07:00
Pavel Feldman
710cec80a0
feat(test-runner): render step titles (#8270) 2021-08-17 16:41:36 -07:00
Pavel Feldman
97ba4f22f3
feat(test-runner): report nested steps (#8266) 2021-08-17 13:57:26 -07:00
Pavel Feldman
d2d71c4cdb
fix(reporter): group fixture initialization under before hooks (#8072) 2021-08-12 07:58:00 -07:00
Dmitry Gozman
3bf3318350
fix(test runner): make sure options, trace and screenshot apply to all contexts (#8083)
- Uses some auto fixtures to set default options and instrumentation on BrowserType.
- Moves screenshot, trace and video to worker-scoped fixtures.
- Throws in page/context when used from beforeAll/afterAll.
- Plumbs around BrowserType to be accessible from Browser and BrowserContext.
2021-08-09 18:09:11 -07:00
Pavel Feldman
3b34e57ee4
feat(test-runner): introduce test.step (#7956) 2021-08-02 22:11:37 -07:00
Pavel Feldman
5803035c1b
feat(test-runner): introduce steps (#7952) 2021-08-02 17:17:20 -07:00
Pavel Feldman
af30d267b6
feat(test-runner): experiemental expect plumbing (#7926) 2021-07-30 16:07:02 -07:00
Dmitry Gozman
cb978848d9
test(test runner): check custom reporter with .only tests (#7862) 2021-07-27 10:37:18 -07:00
Joel Einbinder
051dc332a6
feat(test-runner): use require to resolve reporters (#7749) 2021-07-20 15:03:01 -05:00
Dmitry Gozman
66ea613c4d
feat(test-runner): small changes to Reporter api (#7709)
- `TestResult.startTime`
- `Suite.location` is optional now
- `Test.status()` renamed to `Test.outcome()` to differentiate against a
  `Test.expectedStatus` and `TestResult.status` of the different type.
2021-07-18 17:40:59 -07:00
Dmitry Gozman
18be5f5319
feat(test-runner): suite per project (#7688)
This makes our suite structure the following:
```
Root(title='') > Project(title=projectName) > File(title=relativeFilePath) > ...suites > test
```

Removed `fullTitle()` because it is not used directly by anyone.
Default reporters now report each test as
```
[project-name] › relative/file/path.spec.ts:42:42 › suite subsuite test title
```
2021-07-16 15:23:50 -07:00
Dmitry Gozman
8b2dd2e3d1
chore(test-runner): remove the notion of Spec (#7661)
We now have Suites and Tests. When running multiple projects the whole
suite is cloned for each project. Same happens for repeatEach.

This simplifies the reporters API, but there is still room for improvement.

JSON reporter continues to produce old json output.
2021-07-15 22:02:10 -07:00