1478 Commits

Author SHA1 Message Date
Dmitry Gozman
2c8dd67b0e
fix(test runner): improve error message for non-stable test titles (#23792)
Fixes #23779.
2023-06-19 14:13:11 -07:00
Yury Semikhatsky
f6d86c20f3
chore: run more reporter tests through blob report, some fixes (#23765) 2023-06-16 21:30:55 -07:00
Yury Semikhatsky
426d6dde0e
chore: dedup mergeReport fixture (#23762) 2023-06-16 16:40:55 -07:00
Yury Semikhatsky
11770156eb
feat(html): show number of filtered tests, update total time (#23743) 2023-06-16 09:22:57 -07:00
Yury Semikhatsky
b032e10973
chore: roll stable test runner to jun-15-2023 (#23722) 2023-06-14 18:41:42 -07:00
Yury Semikhatsky
77d322028c
feat(blob): zip .jsonl report files (#23720)
For linux tests without tracing blob-report-1.zip takes 19M, while
unpacked size is 228 MB. That size is counted for GitHub artifact
billing:

<img width="434" alt="image"
src="https://github.com/microsoft/playwright/assets/9798949/5bc32511-6686-4581-a348-acb6a54cd99b">

We zip individual .jsonl reports so that they still have unique names
and can be easily uploaded into the same artifacts directory without
name collisions.
2023-06-14 17:10:39 -07:00
Andrey Lushnikov
3c0fab489b
chore: miscellaneous trace viewer fixes (#23695)
- properly annotate continued requests
- nest `attach` steps inside the related `expect` step
- fix primary-id-to-non-primary-id mapping
- make sure images in trace are not draggable

Fixes #23693

---------

Signed-off-by: Andrey Lushnikov <aslushnikov@gmail.com>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2023-06-14 09:37:19 -07:00
Yury Semikhatsky
dd9a49690b
test: speculative fix for reporter-blob (#23630) 2023-06-09 17:16:45 -07:00
Yury Semikhatsky
abdfe264fa
chore: run html reporter tests with merged report (#23626) 2023-06-09 15:41:15 -07:00
Yury Semikhatsky
400c7cd529
fix: serialize attachment to base64 in tele reporter (#23590) 2023-06-09 11:52:18 -07:00
Yury Semikhatsky
ded4a294f4
fix(merge): populate TestResult.stdout/stderr when merging (#23587) 2023-06-07 23:00:57 -07:00
Yury Semikhatsky
843034d83d
chore: roll stable test runner to 1.35.0-alpha-jun-7-2023 (#23585)
This fixes following error in the merged reports:

```
(index):9825 TypeError: Cannot read properties of null (reading 'toFixed')
    at msToString ((index):18917:15)
    at TestFilesView ((index):19373:9)
    at Uh ((index):9372:7)
    at kj ((index):10444:7)
    at Uk ((index):12508:86)
    at Tk ((index):12134:11)
```
2023-06-07 15:54:14 -07:00
Andrey Lushnikov
0f4090472c
feat: add maskColor option to the toHaveScreenshot method (#23555) 2023-06-06 17:15:55 -07:00
Yury Semikhatsky
874f4525b4
fix(merger): total time is sum of shard total times (#23534) 2023-06-05 19:10:41 -07:00
Yury Semikhatsky
ceaa29cec1
chore: clear message for no reports (#23492) 2023-06-05 17:20:54 -07:00
Dmitry Gozman
6b687b6d69
fix(toHaveScreenshot): attach newly created missing expectations (#23528)
Fixes #23090.
2023-06-05 14:57:06 -07:00
Kristo Jorgenson
d5d155df1f
fix(typescript): allow directory imports (#23254)
This updates previous work in #22887 to align more fully with
`--moduleResolution=bundler`, allowing index files to be imported with
the /index extension

---------

Signed-off-by: Kristo Jorgenson <kristojorg@users.noreply.github.com>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2023-06-05 08:58:25 -07:00
Yury Semikhatsky
9cd49d5dd5
test: unflake "should stop tracing on requestContext.dispose()" (#23489)
Use Chromium instead of Firefox as it navigates faster and remove
timeout from the slow request handler, just make it hang.

Fixes #23116
2023-06-02 16:37:50 -07:00
Max Schmitt
3c2a8fa306
chore: enable no-floating-promises ESLint rule for tests (#23376)
https://github.com/microsoft/playwright/issues/23339
2023-06-02 21:59:12 +02:00
Max Schmitt
af893a1019
chore: inherit apiRequestContext timeout from actionTimeout (#23481)
Fixes https://github.com/microsoft/playwright/issues/23344
2023-06-02 21:36:09 +02:00
Max Schmitt
a97bdd0016
chore: update typescript to 5.1.3 (#23461)
https://devblogs.microsoft.com/typescript/announcing-typescript-5-1/
2023-06-02 18:57:09 +02:00
Pavel Feldman
d1666d2dde
chore: include test-end-screenshot in a trace (#23457)
Fixes https://github.com/microsoft/playwright/issues/23222
2023-06-01 20:29:32 -07:00
Pavel Feldman
96b2247e28
chore: allow marking scripts as external for transform (#23449)
Fixes https://github.com/microsoft/playwright/issues/22874
2023-06-01 20:28:49 -07:00
Pavel Feldman
84942aa992
chore: render successful toPass as such (#23411)
Fixes https://github.com/microsoft/playwright/issues/23302
2023-06-01 13:22:08 -07:00
Dmitry Gozman
5d5314e006
test: make sure trace is recorded for custom fixtures (#23418)
References #23220.
2023-05-31 21:50:20 -07:00
Pavel Feldman
7ad03027fb
chore: remove output dir before each test (#23380) 2023-05-30 16:03:50 -07:00
Yury Semikhatsky
624e88e504
chore: roll stable test runner to May 26 (#23308) 2023-05-26 09:19:23 -07:00
Yury Semikhatsky
57e496c978
test: use platform specific path separator (#23311)
The test has been failing recently
https://devops.playwright.dev/flakiness.html#filter_spec=should+list+files&timestamp=1685115453114
2023-05-26 09:18:53 -07:00
Dmitry Gozman
fa86f2aee0
chore: createHttpServer that destroys sockets upon close (#23294)
This avoids the server hanging on close.
2023-05-26 07:03:41 -07:00
Yury Semikhatsky
6a2d07401e
chore: roll stable test runner to May 25 (#23289) 2023-05-25 15:39:51 -07:00
Pavel Feldman
f879ac7e44
chore: report output dir as a part of the list files (#23283) 2023-05-25 15:38:35 -07:00
Dmitry Gozman
00b34dddb2
chore: move TestServer under test/ (#23287) 2023-05-25 15:11:16 -07:00
Tomáš Hübelbauer
e550df060e
Enhance the forbidOnly mode message to guide the user towards the configuration option (#23146)
Hi, I am putting this PR out as a feeler to see if there's interested in
improving this error message, but the copy is by no means final and I am
open to improvement suggestions.

My intention here is to:
- Explain what a "focused item" is - that we're talking about a test and
it being focused is most likely down it using `only`
  
Are there other types of "items"? Are there other ways to make them
focused other than `only`?

- Explain why we're even in focused mode and how to control it

The default scaffolded Playwright config file includes a forbidMode
expression driven by whether `CI=1` is set.
I ran into this when trying to reproduce a CI issue locally so I had it
set and unknowingly entered focus only mode.
I wasn't aware this mode was a thing because I was using the default
configuration from `npm init` and did not familiarize myself with all
the options in it.

Is there a way to tell if we're in a TypeScript or JavaScript project in
this function? I would use that to display the configuration file name
with the right extension.

---------

Signed-off-by: Tomáš Hübelbauer <tomas@hubelbauer.net>
2023-05-25 13:32:49 -07:00
Yury Semikhatsky
dce730c3be
fix: consider project suffix when computing id (#23270) 2023-05-25 10:36:34 -07:00
Dmitry Gozman
70589c0113
test: unflake "beforeAll and afterAll should have a separate timeout" (#23263) 2023-05-24 13:46:28 -07:00
Dmitry Gozman
871032b411
test: unflake some reporter tests (#23264) 2023-05-24 13:28:40 -07:00
Dmitry Gozman
2858ca95d8
tet: unflake some test runner tests (#23240)
- `unhandled rejection during beforeAll should be reported and prevent
more tests` had timeout of 100ms that is too short on busy system;
- `runTSC` is always slow, so mark as such.
2023-05-23 13:31:23 -07:00
Pavel Feldman
fd75b85510 Revert "chore: more tree gardening (#23119)"
This reverts commit e6bc32b0222d628db6191602ff85ff9801a5a7ec.
2023-05-23 12:04:44 -07:00
Pavel Feldman
6cce93b697
chore: explicit server mode control (#23215) 2023-05-23 10:56:37 -07:00
Pavel Feldman
b814e8a5f1
chore: bring back per test artifacts (#23153) 2023-05-23 09:36:35 -07:00
Yury Semikhatsky
10b7cb3979
chore: blob report project suffix (#23212) 2023-05-22 17:54:37 -07:00
Pavel Feldman
b9e7a91368
fix(html): fix the filter to respect status (#23208) 2023-05-22 15:35:19 -07:00
Pavel Feldman
ee3864913a
chore: allow stub JSX instances in type module (#23211)
Fixes https://github.com/microsoft/playwright/issues/23207
2023-05-22 15:34:50 -07:00
Pavel Feldman
8fee175896
test(filter): add a filter by location test (#23170) 2023-05-19 13:35:11 -07:00
Yury Semikhatsky
5004f908d9
devops: do not upload test-results artifacts (#23168) 2023-05-19 13:06:42 -07:00
Pavel Feldman
49370cb6fd
chore: fix file view padding (#23166) 2023-05-19 11:07:49 -07:00
Pavel Feldman
2697e93663
chore: do not annotate actions after failed ones as timed out (#23148) 2023-05-18 15:52:44 -07:00
Dmitry Gozman
ab7e794bf7
feat(shard): introduce mode: 'default' (#23023)
This mode allows a suite to opt-out from parallelism. Useful to setup
multiple suites running in parallel, with each suite not being sharded.

References #22891.
2023-05-18 13:07:22 -07:00
Pavel Feldman
969e5ff1aa
test: add attachment tests (#23143) 2023-05-18 11:48:53 -07:00
Pavel Feldman
2501bbb715
test(ui-mode): make retries (#23136) 2023-05-18 11:28:28 -07:00