Andrey Lushnikov
e5c9d1e39f
chore: best-effort cleanup for output folders that are mounted ( #12300 )
...
Fixes #12106
2022-02-23 12:55:41 -08:00
Dmitry Gozman
4399623f9f
chore(test runner): make timeout a separate error in TestInfo.errors ( #12315 )
...
This way we control the timeout error message from the runner,
so that later on we can differentiate between test timeout, fixture
timeout and hook timeout.
2022-02-23 12:32:12 -08:00
Dmitry Gozman
114edecd3a
fix(list mode): keep outputDir intact ( #12291 )
2022-02-22 12:50:26 -08:00
Dmitry Gozman
d3c4323021
fix(test runner): improve error message for unexpected calls ( #12240 )
2022-02-18 18:25:18 -08:00
Dmitry Gozman
ee0dd6ec71
fix(test runner): trim full output path ( #12239 )
...
Previously, we only trimmed the "full title" component, but
we should also trim the "relative path" component.
2022-02-18 15:40:36 -08:00
Andrey Lushnikov
0682672242
chore: move comparator logic to playwright-core ( #12232 )
...
This will enable implementation of `toHaveScreenshot` on the
server-side.
Drive-by: drop blink-diff
References #9938
2022-02-18 14:39:17 -08:00
Andrey Lushnikov
b9e86b79ee
chore: refactor toMatchSnapshot implementation ( #12206 )
...
This patch prepares for the `toHaveScreenshot` implementation
by splitting common parts from `toMatchSnapshot`.
Drive-by: fix default extension generation from `.bin` to `.dat`
for unknown buffers.
2022-02-18 11:21:58 -08:00
Dmitry Gozman
edac84d072
feat(remote): send browser/channel/headless in a header ( #12205 )
2022-02-18 07:54:33 -08:00
Dmitry Gozman
34eec7eae7
fix(webServer): route output through reporters ( #12198 )
2022-02-18 07:54:01 -08:00
Andrey Lushnikov
a98babec69
feat(test-runner): introduce pixelCount and pixelRatio options ( #12169 )
...
This patch adds additional options to `toMatchSnapshot` method:
- `pixelCount` - acceptable number of pixels that differ to still
consider images equal. Unset by default.
- `pixelRatio` - acceptable ratio of all image pixels (from 0 to 1) that differ to still
consider images equal. Unset by default.
Fixes #12167 , #10219
2022-02-17 15:44:03 -08:00
Pavel Feldman
51344c6924
chore: parse-skip .tsx ( #12143 )
2022-02-16 15:45:35 -08:00
Andrey Lushnikov
086333cd60
feat(test-runner): support unnamed snapshots ( #12161 )
...
Fixes #9007
2022-02-16 14:22:01 -08:00
Andrey Lushnikov
ef21ce3f56
feat(test-runner): filter out syntax error stack traces ( #12095 )
...
Filter out long stack traces from babel when it fails compilation
due to syntax error in test.
2022-02-14 14:33:14 -08:00
Dmitry Gozman
9814c592d2
fix(test runner): shutdown redundant workers ( #12062 )
2022-02-14 10:57:15 -08:00
Pavel Feldman
96b5831a49
chore: add support private methods in TS ( #12051 )
2022-02-11 14:46:49 -08:00
Dmitry Gozman
d8db785c0a
fix(reporters): correctly handle missing stdout.columns ( #12016 )
...
When columns are not available, do not trim the output.
2022-02-11 08:33:56 -08:00
Pavel Feldman
ea590c6f67
fix(teardown): await teardown in failed test runs ( #12012 )
2022-02-10 12:44:42 -08:00
Ross Wollman
0d42c16a17
fix(test-runner): undefined body crash with manual attachments ( #11995 )
...
The new (as of 1.18) `async testInfo.attach(…)` API handles this
gracefully (and is part of the reason for the new API's existence).
However, for the foreseeable future, it's still possible to manually
push onto the attachments array where we can't validate the contents
until it's too late, so this change ensures more graceful handling in
that case.
Fixes #11565
2022-02-10 12:33:38 -08:00
Pavel Feldman
fbc05b74a7
chore: revert "fix(test-runner): escape backslashes in win cli ( #11763 )" ( #12004 )
2022-02-10 07:02:37 -08:00
Andrey Lushnikov
6904b3294e
fix(test-runner): fix browser initialization in test modifiers ( #11984 )
...
Fixes #11985
2022-02-09 15:30:14 -08:00
Yury Semikhatsky
706c897031
feat(fixtures): respect tracing config for APIRequestContext ( #11954 )
2022-02-09 08:54:09 -08:00
Pavel Feldman
e9e5de2d57
feat(debug): allow preprocessing JS scripts as well ( #11953 )
2022-02-09 07:14:11 -08:00
Dmitry Gozman
19368e93af
feat(test runner): support connectOptions ( #11919 )
...
This allows to specify `connectOptions` in the config that
switch built-in `browser` to be remotely connected.
2022-02-08 20:45:42 -08:00
Dmitry Gozman
5881a46ecf
fix(test runner): skip beforeAll/afterAll when all tests are skipped ( #11952 )
...
There is a corner case where tests were skipped like this:
```js
test.skip('title', () => {});
```
2022-02-08 16:36:30 -08:00
Andrey Lushnikov
3eba252f2e
chore: cut v1.19.0 ( #11944 )
2022-02-08 16:01:44 -08:00
Dmitry Gozman
e92caf01b3
fix(webServer): do not set baseURL equal to webServer.url ( #11951 )
2022-02-08 15:57:36 -08:00
Pavel Feldman
8dff2e35c8
chore: don't throw on no testDir ( #11950 )
2022-02-08 15:27:05 -08:00
Pavel Feldman
d5158e8d24
chore: revert source-map-support to 0.4.18 ( #11945 )
2022-02-08 14:50:20 -08:00
Pavel Feldman
4bcca2c87e
chore: add a parallel mode note ( #11943 )
2022-02-08 14:36:14 -08:00
Dmitry Gozman
8a5c93436d
fix(reporter): do not report parallel tests as slow ( #11921 )
2022-02-07 20:10:13 -08:00
Dmitry Gozman
7912c515a3
fix(fixtures): account for default options being undefined ( #11916 )
2022-02-07 17:11:36 -08:00
Pavel Feldman
9116adc684
chore: allow opt-into the legacy global setup mode ( #11888 )
2022-02-07 10:41:56 -08:00
Yury Semikhatsky
46dfa45b4e
docs: fix parallel test docs build ( #11877 )
2022-02-04 16:44:04 -08:00
Yury Semikhatsky
afc4b505aa
docs: fix duplicate sections ( #11874 )
2022-02-04 13:50:46 -08:00
Dmitry Gozman
d9a8bb057d
fix(test-fail): allow unhandled expects in test.fail ( #11850 )
...
Previously, we would consider this a worker error, but
we make an exception for "expect()" calls.
2022-02-03 17:14:12 -08:00
Pavel Feldman
72424dc904
chore: allow setting reporter via env ( #11848 )
2022-02-03 16:10:39 -08:00
Pavel Feldman
fdda759a9d
feat(parallel): allow setting enclosing scope parallel mode ( #11822 )
2022-02-02 20:44:11 -08:00
Andrey Lushnikov
ba0c7e679b
feat(test-runner): support expect.soft ( #11800 )
...
Soft expects will still fail the test, but will not abort it's execution. As a consequence of this:
- `TestResult` now might have multiple errors, which is reflected with a new `testResult.erros: TestError[]` field.
- `TestInfo` now might have multiple errors as well, which is reflected with a new `testInfo.errors: TestError[]` field.
Fixes #7819
2022-02-02 18:33:51 -08:00
Max Schmitt
c752b28516
chore: pin production dependencies ( #11793 )
2022-02-02 11:14:41 +01:00
Andrey Lushnikov
b0daa7754f
feat: filter stack traces to exclude test runner frames ( #11795 )
...
Before:
```bash
Running 1 test using 1 worker
1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================
Error: expect(received).toBe(expected) // Object.is equality
Expected: 2
Received: 1
2 |
3 | test('should work', async({page}) => {
> 4 | expect(1).toBe(2);
| ^
5 | });
6 |
at Proxy.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/expect.ts:151:30)
at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
at /Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:335:13
at runNextTicks (node:internal/process/task_queues:61:5)
at processImmediate (node:internal/timers:437:9)
at TestInfoImpl._runFn (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:164:7)
at WorkerRunner._runTestWithBeforeHooks (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:317:24)
at TimeoutRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-core/src/utils/async.ts:48:14)
at TestInfoImpl._runWithTimeout (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:151:7)
at WorkerRunner._runTestOrAllHook (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:276:5)
at WorkerRunner._runSuite (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:190:11)
at WorkerRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:137:9)
at process.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/worker.ts:87:5)
```
after:
```
Running 1 test using 1 worker
1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================
Error: expect(received).toBe(expected) // Object.is equality
Expected: 2
Received: 1
2 |
3 | test('should work', async({page}) => {
> 4 | expect(1).toBe(2);
| ^
5 | });
6 |
at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
```
2022-02-01 18:40:44 -08:00
Pavel Feldman
ceb3027bdf
chore(test-runner): force colors in worker processes ( #11799 )
2022-02-01 17:29:15 -08:00
Pavel Feldman
6e2fcc4700
chore: do not expose suite load error ( #11797 )
2022-02-01 15:34:16 -08:00
Pavel Feldman
f875ebe730
fix(test-runner): respect source maps when reporting test files ( #11770 )
2022-02-01 13:04:54 -08:00
Max Schmitt
028afb167b
chore: bump production dependencies ( #11787 )
2022-02-01 21:27:34 +01:00
Pavel Feldman
a3bc911a3d
fix(debug): don't apply infinite timeouts when using debugger ( #11785 )
2022-02-01 11:51:37 -08:00
Caio Agiani
b6b60decdf
fix: typos ( #11789 )
2022-02-01 11:09:41 -08:00
Pavel Feldman
3a5e8184b5
fix(html-reporter): open tests from required file ( #11784 )
2022-02-01 11:01:52 -08:00
Pavel Feldman
c82f2641d7
fet(list-files): report per-project test dir and filters ( #11764 )
2022-02-01 08:08:56 -08:00
Pavel Feldman
c5d852f1bb
fix(test-runner): escape backslashes in win cli ( #11763 )
2022-01-31 17:48:05 -08:00
Andrey Lushnikov
2336692e8a
feat: support clarification message for expect ( #11735 )
...
The clarification message is displayed in the HTML report as the name of the step:
It is also shown in terminal output:
Fixes #7816
2022-01-31 17:14:59 -08:00
