20 Commits

Author SHA1 Message Date
Joel Einbinder
051dc332a6
feat(test-runner): use require to resolve reporters (#7749) 2021-07-20 15:03:01 -05:00
Joel Einbinder
d72efbe181
feat(test-runner): smaller error message paths (#7384) 2021-07-19 12:20:24 -05:00
Dmitry Gozman
66ea613c4d
feat(test-runner): small changes to Reporter api (#7709)
- `TestResult.startTime`
- `Suite.location` is optional now
- `Test.status()` renamed to `Test.outcome()` to differentiate against a
  `Test.expectedStatus` and `TestResult.status` of the different type.
2021-07-18 17:40:59 -07:00
Dmitry Gozman
167db03f05
feat(test-runner): export reporter api as @playwright/test/reporter (#7692) 2021-07-16 21:15:03 -07:00
Dmitry Gozman
18be5f5319
feat(test-runner): suite per project (#7688)
This makes our suite structure the following:
```
Root(title='') > Project(title=projectName) > File(title=relativeFilePath) > ...suites > test
```

Removed `fullTitle()` because it is not used directly by anyone.
Default reporters now report each test as
```
[project-name] › relative/file/path.spec.ts:42:42 › suite subsuite test title
```
2021-07-16 15:23:50 -07:00
Dmitry Gozman
31572fc372
chore(test-runner): misc changes to reporter api (#7664)
- `Location` with `file`, `line` and `column`.
- `fullTitle` does not include project name.
- `titlePath` method.
- All methods of `Reporter` are optional.
- Removed `Test.skipped` property that is superseeded by `Test.status()`.
- Replaced `Suite.findTest()` with `Suite.allTests()`.
- Removed `Test.suite` property.
2021-07-16 12:40:33 -07:00
Max Schmitt
6cc2fe178e
feat(test-runner): migrate to launch config/server (#7603) 2021-07-15 01:19:45 +02:00
Joel Einbinder
eb31b9e4a9
feat(test-runner): support esm modules in more places (#7542) 2021-07-12 11:59:58 -05:00
Max Schmitt
98bcf26656
feat(test-runner): add webServer (#7368) 2021-07-07 20:19:42 +02:00
Joel Einbinder
7629587914
fix(test-runner): work with .mjs files (#7373) 2021-06-29 15:28:41 -07:00
Joel Einbinder
368880962f
fix(test-runner): better test match glob (#7382) 2021-06-29 11:49:50 -07:00
Dmitry Gozman
4c6fa42810
fix(test runner): expose real stack traces and speed up locations (#7265)
Stop wrapping/prepending error messages so that we do not loose the stack trace. For this, update a few manually thrown errors with better messages (usually including a file path).

Speed up locations by doing manual `sourceMapSupport.wrapCallSite()` for a single call site. Performance gain in the runner process with 100 files x 100 tests each:
- 25% on the fresh run without babel cache;
- 80% on the cached run where babel is almost instant.

Also some obvious cleanups around stack traces (removing unused code).
2021-06-23 10:30:54 -07:00
Max Schmitt
103f8ddd06
Revert "chore: update eslint config (#6840)" partly (#7268)
This reverts commit ec7d37d92f4b6b75039d2343414237d2dbc84847.
2021-06-23 11:08:35 +02:00
Dmitry Gozman
6118d16edd
fix(test runner): properly keep track of requireFile to support helpers/wrappers (#7243)
This fixes an issue where we incorrectly labeled and assigned ids for tests
that declared tests in require'd files or used test wrappers.
See new tests for examples.
2021-06-21 11:25:15 -07:00
Pavel Feldman
ea4eebeb2d
feat(test-runner): document tagging, implement grep-invert (#7227) 2021-06-18 17:56:59 -07:00
Joel Einbinder
4c2a3fb443
fix(test-runner): accept relative paths for outputDir (#7149)
fixes #7124
2021-06-15 13:39:07 -07:00
Dmitry Gozman
742cce3a1d
feat(test runner): configurable reportSlowTests (#7120)
Also splits tests by projects and reports them with nice relative paths.
2021-06-14 22:45:58 -07:00
Pavel Feldman
970bb6a70d
feat(test-runner): allow setting pixel match threshold for project (#7123) 2021-06-14 21:52:10 -07:00
Pavel Feldman
49a8f67c0f
fix(test-runner): resolve global hooks relative to the config dir (#7061) 2021-06-10 22:31:27 -07:00
Dmitry Gozman
f745bf1fbc
chore: bring in folio source (#6923)
- Source now lives at `src/test`.
- Former folio tests live at `tests/playwright-test`.
- We use `src/test/internal.ts` that exposes base test without
  Playwright fixtures for most tests (to avoid modifications for now).
- Test types live in `types/testFoo.d.ts`.
- Stable test runner is installed to `tests/config/test-runner` during `npm install`.
- All deps including test-only are now listed in `package.json`.
  Non-test deps must also be listed in `build_package.js` to get included.
2021-06-06 17:09:53 -07:00