858 Commits

Author SHA1 Message Date
Dmitry Gozman
91da67fab1
test: remove magic headers in ttest (#20867)
Instead, explicitly import from '@playwright/test'.
2023-02-14 19:20:56 -08:00
Pavel Feldman
08be39a80e
fix(watch): fix the tests on win (#20844) 2023-02-14 14:55:49 -08:00
Pavel Feldman
7f0763d789
chore: hide watch mode, it is not ready (#20905) 2023-02-14 13:22:44 -08:00
Pavel Feldman
798696a18a
chore: fix context reuse disposal (#20876)
Fixes: https://github.com/microsoft/playwright/issues/20409
2023-02-14 12:56:47 -08:00
Dmitry Gozman
6d03211439
fix(test runner): --list should ignore '.only' annotations (#20868) 2023-02-13 11:13:30 -08:00
Andrey Lushnikov
fdcd7b549d
chore: mark comparator option back as experimental (#20816)
This reverts commit 303c5998f8b747e94ac80e2e02c53b47ded11322.

Reason for revert: I tried enabling `ssim-cie94` by default on
ionic-framework test suite, and it proves to be overly strict for their
usecase.
2023-02-13 11:11:44 -08:00
Joel Einbinder
997dfa9274
fix(reporter): properly indent multiline attachments (#20847)
Text attachments were only indented on the first line.
2023-02-13 10:28:34 -08:00
Dmitry Gozman
de69b766d9
test: code health in ttest (#20837)
- Remove duplicate tests.
- Remove unused test helpers.
- Print full watch output on failures.
- Unflake some tests.
2023-02-10 20:26:19 -08:00
Dmitry Gozman
789b1c75e6
fix(expect): report received when timedOut during oneShot (#20806) 2023-02-10 14:59:21 -08:00
Pavel Feldman
1ba768bf60
chore: make watch + ct happy (#20804) 2023-02-10 08:33:25 -08:00
Pavel Feldman
4469e57695
chore: always grow component bundle (#20799)
Fixes https://github.com/microsoft/playwright/issues/20581
2023-02-09 18:39:20 -08:00
Pavel Feldman
69c4653c6b
chore: speculative windows cli fix (#20801) 2023-02-09 16:04:07 -08:00
Pavel Feldman
e1f287f255
chore: more watch tests (#20797) 2023-02-09 16:03:54 -08:00
Pavel Feldman
596ed97791
chore: add more watch tests (#20793) 2023-02-09 13:57:00 -08:00
Pavel Feldman
5112abc1d2
chore: fix a racy time-based test (#20783) 2023-02-09 08:49:51 -08:00
Pavel Feldman
b247bfe153
test(watch): start adding tests (#20764) 2023-02-09 08:31:02 -08:00
Pavel Feldman
7a093329fa
chore: do not run all on watch (#20758) 2023-02-08 12:44:51 -08:00
Pavel Feldman
11168efb0e
chore: roll stable-test-runner to ToT (#20754) 2023-02-08 10:02:20 -08:00
Yury Semikhatsky
a93cf767a1
feat: html reporter sharded option (#20737)
Make sharded report feature an opt-in:

```ts
{
   reporter: [['html', { sharded: true }]]
};
```

#10437
2023-02-07 22:21:50 -08:00
Pavel Feldman
4259d4e1d6
chore: implement repeat last run (#20727) 2023-02-07 15:56:39 -08:00
Pavel Feldman
1b941bcf2e
chore: simplify ttests (#20733) 2023-02-07 15:11:44 -08:00
Yury Semikhatsky
f10b29fd5e
feat: show warning when some report shards are missing (#20731)
When some of the report shards are missing still show the report but
display an error in the status line:

![image](https://user-images.githubusercontent.com/9798949/217371112-2317a307-204e-4c11-8d01-42e91fe6ca7a.png)

#10437
2023-02-07 14:22:10 -08:00
Yury Semikhatsky
8f53bf7b41
feat: sharded html report (#20700)
This implementation is based on the [original
PR](https://github.com/microsoft/playwright/pull/19691) by @kevin940726.
It makes the reporter produce single file when there is no sharding and
multiple out-of-line report-x-of-y.zip reports which are automatically
merged together when put in one folder.

References https://github.com/microsoft/playwright/issues/10437

Co-authored-by: Kai Hao <kevin830726@gmail.com>
2023-02-07 10:21:26 -08:00
Andrey Lushnikov
303c5998f8
feat: release "comparator" option from experiment (#20720)
The option defines a comparator to be used to compare images.
Possible values are `"pixelmatch"` and `"ssim-cie94"`.

Note: This reverts commit 8167f8bf548308ad8c6f1188508aadee84f26023.
2023-02-07 08:51:48 -08:00
Pavel Feldman
361ea949aa
chore: respect deps when watching files (#20695) 2023-02-06 17:09:16 -08:00
Pavel Feldman
b6df48758d
chore: collect test dependencies (#20645) 2023-02-06 14:52:40 -08:00
Dmitry Gozman
e64c623e61
feat(expect): strip down Expect types (#20601)
This only leaves:
- documented assertions;
- asymmetric matchers;
- `resolves`/`rejects`;
- `expect.extend()`;
- `expect.getState()` with selected properties.

References #20432.
2023-02-03 15:56:31 -08:00
Sam Chen
340cacf4fd
feat: show diff tab by default (#19820)
Close https://github.com/microsoft/playwright/issues/19803

I had to manually inject `window.playwrightReportBase64` data into
`playwright/packages/html-reporter/index.html` when developing,
wondering if there's any method I don't know.
2023-02-03 10:32:23 -08:00
Andrey Lushnikov
f45f20a8c9
fix: handle not.toPass() with expect.soft inside (#20596)
Fixes #20518
2023-02-03 09:14:05 -08:00
Dmitry Gozman
3180bc9804
fix(types): make most fixtures non-undefined (#20573)
Fixes #20416.
2023-02-02 13:11:50 -08:00
Andrey Lushnikov
4f61105032
fix: swallow expect.soft errors inside successful toPass matcher (#20509)
Fixes #20437
2023-02-02 10:47:20 -08:00
Dmitry Gozman
5fb430a743
fix(test runner): do not print timeout message upon interrupt (#20574)
We should only print "Test was interrupted."

Regressed in #18321.
2023-02-01 19:39:43 -08:00
Dmitry Gozman
f46883e58e
feat(expect): add GenericAssertions documentation (#20564)
References #20432.
2023-02-01 16:55:52 -08:00
Pavel Feldman
421dd884a7
chore: implement --no-deps (#20569) 2023-02-01 16:32:13 -08:00
Pavel Feldman
635b47025e
chore: expose project dependencies api (#20546) 2023-02-01 08:39:07 -08:00
Pavel Feldman
08e4b50ff6
chore: project deps (#20514) 2023-01-31 15:59:13 -08:00
Dmitry Gozman
9c6c31a442
fix(types): update PlaywrightTestConfig types (#20540)
This makes errors more focused on the custom properties.

References #20416.
2023-01-31 15:02:01 -08:00
Yury Semikhatsky
a03f3223c4
fix: trim long expect messages (#20517)
Fixes #20324
2023-01-30 17:24:12 -08:00
chimurai
f72d400952
chore: fix typos (#20473) 2023-01-30 09:53:28 +01:00
Pavel Feldman
2c27bd3b07
chore: remove plugin registration sink (#20460) 2023-01-27 14:36:41 -08:00
Pavel Feldman
f7ff252455
chore: remove addFatalError (#20383) 2023-01-25 17:26:30 -08:00
Pavel Feldman
fe1dd7818d
chore: extract task runner for global setup (#20345) 2023-01-25 15:38:23 -08:00
Dmitry Gozman
d1fb3a2384
fix(test runner): do not send entirely skipped test groups to a worker (#20346)
Move TestCase properties calculation from WorkerMain to suite building
phase.

Fixes #20156.
2023-01-25 12:54:50 -08:00
Dmitry Gozman
b971dd3417
fix(test runner): show deep strack traces during imports (#20305)
Fixes #20239.
2023-01-24 08:49:06 -08:00
Pavel Feldman
147bb6b292
chore: run global setup before onBegin (#20285) 2023-01-23 17:44:23 -08:00
Pavel Feldman
9f31bcfbab
chore: refactor graceful close, rename to host/main (#20283) 2023-01-22 15:04:29 -08:00
Dmitry Gozman
b700c08dc5
feat(screenshots): when actual and expected have different sizes, pad and produce the diff image (#20208)
Also show sizes in the html report to easier spot the size mismatch
issue.

<img width="1030" alt="diff"
src="https://user-images.githubusercontent.com/9881434/213327632-b8fcd69c-8d08-460c-9de1-b5f4f8c56359.png">

Fixes #15802.
2023-01-20 19:41:43 -08:00
Pavel Feldman
7ff27600b4
chore: experimental oop loader (#20269) 2023-01-20 18:24:15 -08:00
Dmitry Gozman
eafa6fda13
fix(expect): poll/toPass should not wait over specified timeout (#20266)
Drive-by: unflake some timeout-dependent tests.
2023-01-20 15:47:24 -08:00
Pavel Feldman
3066ffd577
chore: use fake pool on the runner side (#20241) 2023-01-20 08:36:31 -08:00