1519 Commits

Author SHA1 Message Date
Pavel Feldman
6d5889a52c
chore: revert the matcherResult in API (#32524) 2024-09-09 16:44:32 -07:00
Dmitry Gozman
ae118674b8
fix(test runner): allow directory imports with path mapping (#32491)
We now hopefully align with `moduleResolution: bundler` tsconfig option,
allowing directory imports in every scenario, and allowing proper module
imports when not going through the type mapping.

This regressed in #32078. Fixes #32480, fixes #31811.
2024-09-09 14:01:20 -07:00
Dmitry Gozman
6bb005db85
fix(test runner): improve error message when not able to parse tsconfig (#32526) 2024-09-09 14:01:02 -07:00
Dmitry Gozman
e6c5b6054d
test: fix project filter test (#32525)
It was erroneously passing projects separate by comma, which never
worked.
2024-09-09 14:00:51 -07:00
Pavel Feldman
9a313eecc9
chore: roll expect and move it to third party (#32458) 2024-09-09 13:12:20 -07:00
Dmitry Gozman
1402dee9e6
Revert "fix(test runner): align with typescript behaviour for resolving index.js and package.json through path mapping (#32078)" (#32492)
This reverts commit effb1ae2349aeef1c0bfb0b6cd886b9b4b32c8e5.

This broke path mapping into directories in ESM mode. References #32480.
2024-09-06 12:08:10 -07:00
Simon Knott
0e3f4736cc
fix(test runner): always show all projects in selection (#32450)
Follow-up to
https://github.com/microsoft/playwright/pull/32156#discussion_r1741628770,
alternative solution to
https://github.com/microsoft/playwright/pull/32425.

Ensures we always show all projects in the watch mode project selector
by performing the initial `listTests` without any filters, and using its
result for the project selector.
2024-09-06 11:35:20 +02:00
Pavel Feldman
a52eb0c9a0
chore: expose matcherResult on TestError (#32455) 2024-09-05 21:36:51 -07:00
Dmitry Gozman
1ba3db6864
chore: implement clear-cache through plugins (#32471)
Also switches it to task runner.
2024-09-05 13:50:16 -07:00
Dmitry Gozman
d4c77ce260
chore: make find-related-test-files work through plugins (#32465)
Also switches it to the task runner.
2024-09-05 06:52:11 -07:00
Simon Knott
29f1541b14
chore: roll stable test runner to 1.47 beta (#32469) 2024-09-05 13:44:22 +02:00
Dmitry Gozman
91012833c6
chore: move 'dev-server' extensibility point to plugin (#32448)
Instead of plumbing it through a custom unspecified config field, make
it a part of plugin interface.

Additionally, use task runner for starting/stopping dev server.
2024-09-05 02:22:27 -07:00
Kuba Janik
ee91bdc585
feat(ui-mode): display list of query params in request tab (#32443) 2024-09-04 07:54:44 -07:00
Dmitry Gozman
60631409d6
chore: make dev server only use public config (#32441)
In preparation to make it a part of a plugin.
2024-09-04 01:29:55 -07:00
Yury Semikhatsky
565aed6c39
Revert "chore: enforce tags format via typescript types (#32384)" (#32431)
After API review we decided to revert it:
* VSCode extension and UI mode users already get the (runtime) error if
the tag is not prefixed
* The typescript error message is not very nice
* The type change would break those clients that generate tests with
tags passed as string

This reverts commit 90e7b9ebacbd597b7380522001eb6d17ee9c3d86.
2024-09-03 10:07:08 -07:00
Simon Knott
201bad75d3
chore(test runner): rebase watch mode onto TestServerConnection (#32156)
Closes https://github.com/microsoft/playwright/issues/32076.

This PR rewrites `watchMode.ts` to use `TestServer` under the hood. It's
essentially a complete rewrite, so don't pay too much attention on the
old implementation. Note that there's no changes to tests, so all
behaviour we have specced out there still works.

To make this work without a superfluous WebSocket connection, I had to
refactor `TestServerConnection` a little. Originally, I pulled this into
a [separate PR](https://github.com/microsoft/playwright/pull/32132), but
then realised how small the refactoring is. So it's in this PR now. Let
me know if you'd like to land it separately.
2024-09-03 15:15:44 +02:00
Simon Knott
f62f85ba51
fix(test runner): fix types to allow calling custom matchers on expect.poll (#32407)
The `'should support custom matchers'` test asserts that the
functionality works, but it was a type error. This PR updates the types
so that it's allowed.

Closes https://github.com/microsoft/playwright/issues/32408

---------

Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2024-09-02 13:42:15 +02:00
Simon Knott
cf8c14f884
feat(html reporter): open html attachments in new tab (#32389)
Closes https://github.com/microsoft/playwright/issues/32281.

HTML attachments get a linkified name that opens the attachment in a new
tab.
2024-09-02 08:35:53 +02:00
Kuba Janik
a6b320e362
fix(ui-mode): format request body when headers are lower case (#32395)
Resolves https://github.com/microsoft/playwright/issues/32396

Currently, the request body is not formatted when content type header is
lower case (`content-type`). Even though the value is
`application/json`.

It happens because we are looking only for `Content-Type` header
ignoring headers that are lower case.

<img width="674" alt="363197933-5178ec23-b9cf-46b5-8284-e8d4d730b036"
src="https://github.com/user-attachments/assets/0ef01b52-7dd8-4f33-b836-9adb86f94cc9">
2024-08-30 16:21:51 +02:00
Simon Knott
ed5c21b827
fix(ui): respect --output param (#32351)
Closes https://github.com/microsoft/playwright/issues/32331

We're already passing the `outputDir` param to the UI, but the UI isn't
passing it back to the TestServer. This PR fixes that. I've added it to
`listTests`, which is requires to that
`TestServerDispatcher#_ignoredProjectOutputs` is populated with the
correct output dir. And i've added it to `runGlobalSetup`, which is what
the bug report was about.
2024-08-30 08:29:49 +02:00
Yury Semikhatsky
90e7b9ebac
chore: enforce tags format via typescript types (#32384)
Leverage [template literal
types](https://www.typescriptlang.org/docs/handbook/2/template-literal-types.html).

Fixes https://github.com/microsoft/playwright/issues/32382
2024-08-29 14:16:49 -07:00
Simon Knott
1511d8643e
fix(test runner): expect.poll error reporting should handle non-expect errors (#32257)
Closes https://github.com/microsoft/playwright/issues/32256

We were expecting all errors to be of type `ExpectError`, but apparently
`expect` propagates rejections in the polling functions right through.
So we also need to handle that case.

I wonder if we have more cases of this. Would it make sense to enable
`useUnknownInCatchVariables` in TypeScript?
2024-08-26 09:39:25 +02:00
Pavel Feldman
9d86bc5336
fix(dupe): render dupe test error indicator (#32303)
Fixes https://github.com/microsoft/playwright/issues/32093
2024-08-23 14:33:37 -07:00
Simon Knott
3fb33e7144
chore(ui): decouple TestServerConnection from websocket transport (#32274)
Preparation for https://github.com/microsoft/playwright/issues/32076.
2024-08-23 14:58:34 +02:00
Simon Knott
5368fd7ca7
fix(only-changed): exit successfully if there were no changes (#32197)
Closes https://github.com/microsoft/playwright/issues/32180

I was briefly wondering if we should output a log line a la "no tests
found", but my understanding is that that's the reporters job - so I
didn't change anything in that regard.
2024-08-22 14:53:00 +02:00
Dmitry Gozman
7758b330b1
fix(ui mode): make sure that reload does correctly restart the webserver (#32263)
Fixes #32103.
2024-08-22 05:48:31 -07:00
Simon Knott
3e6bba0b79
fix(only changed): make only-changed work together with list mode (#32196)
Closes https://github.com/microsoft/playwright/issues/32161

Turns out we were wrong in
https://github.com/microsoft/playwright/pull/31727#discussion_r1685793870!

Adds support for `--only-changed` in combination with `--list` by
removing our code to prevent that.
2024-08-16 17:12:45 +02:00
Dmitry Gozman
c4bb24f02f
feat(test runner): record trace after a test-scoped fixture teardown times out (#32160)
Fixes #30718, fixes #31537.
2024-08-16 08:03:02 -07:00
Dmitry Gozman
e17d1c498b
fix(test runner): timeout in fixture teardown should not prevent other slots (#32157)
When two fixtures have different time slots, timeout in the first one
should not prevent the second one from tearing down.

Similarly, timeout in afterEach hook should not prevent fixture
teardowns.
2024-08-16 04:25:00 -07:00
Dmitry Gozman
1537d3c2de
chore(test runner): make 'debug' an explicit option internally (#32154)
This allows any time slot that has a legitimate timeout of zero to be
updated later on. See test for an example.

Previously, setting timeout to zero at any moment was considered a
"debug mode" and any subsequent timeouts were ignored.
2024-08-16 01:44:37 -07:00
Simon Knott
edd1894ac6
fix(test runner): run project dependencies of --only-changed test files (#32094)
Closes https://github.com/microsoft/playwright/issues/32070. We were
applying `additionalFileMatcher` not just to `filteredProjectSuites`,
but also to `projectSuites`. `projectSuites` is where we take dependency
projects from, though - so `--only-changed` led to empty dependency
projects, resulting in the reported bug.

The fix is to only apply `additionalFileMatcher` on
`filteredProjectSuites`.
2024-08-12 17:26:01 +02:00
Simon Knott
effb1ae234
fix(test runner): align with typescript behaviour for resolving index.js and package.json through path mapping (#32078)
Supercedes https://github.com/microsoft/playwright/pull/31915, closes
https://github.com/microsoft/playwright/issues/31811.

When TypeScript resolves a specifier via path mapping, it does not
interpret `package.json`. If path mapping resolves to a directory, it
only looks at the `index.js` file in that directory if it's in CommonJS
mode.

We need to mirror this in our `esmLoader.ts`.

---------

Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2024-08-12 15:05:19 +02:00
Simon Knott
c8cc4f9c8b
chore(ui): update to react 18 (#32079)
Part of https://github.com/microsoft/playwright/issues/31863. Updates
most of our React usage to React 18. `recorder` doesn't seem to like it
yet. I suspect that some of our code isn't compatible with concurrent
mode, i've investigated that in
https://github.com/microsoft/playwright/pull/32101.

---------

Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2024-08-12 13:50:11 +02:00
Dmitry Gozman
f24e46c367
fix(types): revert type changes made to support TS 5.5 (#32066)
Regressed in #31532. The TS5.5 changes broke chaining of `extend`s where
the first `extend` did not specify any type arguments.

Fixes #32056.
2024-08-08 05:21:48 -07:00
Kuba Janik
7ec3a93db3
feat(ui-mode): add filters to network tab (#31956) 2024-08-06 14:52:35 -07:00
Dmitry Gozman
a54ed48b42
feat(test runner): --tsconfig cli option (#31932)
Introduce `--tsconfig` to specify a single config to be used for all
imported files, instead of looking up tsconfig for each file separately.

Fixes #12829.
2024-08-06 06:55:15 -07:00
Yury Semikhatsky
fef27395a5
chore(trace): do not nest API actions based on time (#31990)
They should be properly nested based on Node.js zones now.
2024-08-05 10:06:14 -07:00
Dmitry Gozman
5c9ce6b9d9
test: unflake various tests (#32014) 2024-08-05 08:29:13 -07:00
Yury Semikhatsky
a828fd5d73
test: ui mode annotations (#31965) 2024-08-01 14:47:50 -07:00
Yury Semikhatsky
1074a765e4
fix(trace): do not place expect into unfinished api calls based on time (#31970)
Fixes https://github.com/microsoft/playwright/issues/31959
2024-08-01 14:14:10 -07:00
Dmitry Gozman
a541751657
feat(ui mode): linkify attachment names and content (#31960)
- Pass `contentType` to the CodeMirror.
- Support `text/markdown` mode.
- Custom mode for non-supported types that linkifies urls.
2024-08-01 09:27:45 -07:00
Yury Semikhatsky
0217defab4
chore(trace-viewer): do not shrink metadata view (#31938)
Avoids the following effect:

![image](https://github.com/user-attachments/assets/694de773-acc0-4266-87f2-eab67a3c7ce2)
2024-07-31 16:37:16 -07:00
Dmitry Gozman
e62a54af7a
fix(test runner): do not revert the transform (#31930)
This allows a dynamic import of a TS file to be processed by Babel.

For some reason, Playwright used to revert the CJS transforms. However,
ESM loader and transforms are always active, so CJS should be too.
2024-07-31 13:17:09 -07:00
Dmitry Gozman
7c55b94280
fix(trace): make sure the correct attachment name is used for downloads (#31928)
When two attachments have the same content sha1, we used the first one's
name for the downloaded file, no matter which one the user clicked to
download. Now we pass the name explicitly.

References #31912.
2024-07-31 06:20:36 -07:00
Simon Knott
8412d973c0
fix(ui): added test in watched file should be run (#31842)
Closes https://github.com/microsoft/playwright/issues/22211

Currently, when the server notifies the UI about changed files, the UI
determines what files to re-run based on an old test list. By listing
tests before that, we make sure that the test list is up-to-date, and
that added tests are included in the next run.

I've also removed the `listChanged` event as discussed in the team sync.
The event isn't used anywhere and fires in exactly the same cases where
`testFilesChanged` fired, so i've folded them into one another. This allowed simplifying `Watcher`.
2024-07-30 14:17:41 +02:00
Dmitry Gozman
f232507afa
feat(ui mode): ui updates (#31894)
- Update copy to clipboard button.
- Reveal test source in the Source tab instead of external editor.
- New button to reveal in the external editor in the Source tab.
- Move the Pick Locator button next to snapshot tabs.
2024-07-29 07:32:13 -07:00
Dmitry Gozman
a41cebc1c9
feat(ui mode): introduce Testing Options and Settings (#31841)
Testing Options control tests, while Settings are UI mode settings.

<img width="298" alt="Screenshot 2024-07-25 at 10 54 22 AM"
src="https://github.com/user-attachments/assets/7b6f5fff-687b-48d1-80b3-d1e6f2a257e8">


These sections are separately expandable, collapsed by default.

<img width="294" alt="Screenshot 2024-07-24 at 2 06 25 PM"
src="https://github.com/user-attachments/assets/5d35ac8c-9289-46ca-aaa2-ebc5419fa0c4">

References #31520.

---------

Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Simon Knott <info@simonknott.de>
2024-07-25 11:23:43 -07:00
Simon Knott
a1a4216b88
chore(ui): add test expectation that doesn't immediately pass on blank tree (#31850)
Followup to #31815
2024-07-25 12:21:24 +02:00
Pavel Feldman
7735affef4
fix(ui): print the web server output in the ui mode (#31824)
Fixes https://github.com/microsoft/playwright/issues/31300
2024-07-23 16:52:32 -07:00
Simon Knott
f23d02a211
feat(test runner): --only-changed option (#31727)
Introduces an `--only-changed [base ref]` option.

`playwright test --only-changed` filters the test run to only run test
suites that have uncommitted changes.
`playwright test --only-changed=foo` runs only tests that were changed
since commit `foo`.

In pull request CI, this can be used to run changed tests first and fail
fast: `--only-changed=$GITHUB_BASE_REF`.
During local development, it can be used to quickly filter down to the
touched set of tests suites.
In some rare usecases, this can also help to cut down on CI usage for
pull requests. Tread with caution though.

File dependencies are taken into account to ensure that if you touched a
utility file, all relevant tests are still executed.

Closes https://github.com/microsoft/playwright/issues/15075
2024-07-23 18:04:17 +02:00