Closes https://github.com/microsoft/playwright/issues/22211
Currently, when the server notifies the UI about changed files, the UI
determines what files to re-run based on an old test list. By listing
tests before that, we make sure that the test list is up-to-date, and
that added tests are included in the next run.
I've also removed the `listChanged` event as discussed in the team sync.
The event isn't used anywhere and fires in exactly the same cases where
`testFilesChanged` fired, so i've folded them into one another. This allowed simplifying `Watcher`.
Introduces an `--only-changed [base ref]` option.
`playwright test --only-changed` filters the test run to only run test
suites that have uncommitted changes.
`playwright test --only-changed=foo` runs only tests that were changed
since commit `foo`.
In pull request CI, this can be used to run changed tests first and fail
fast: `--only-changed=$GITHUB_BASE_REF`.
During local development, it can be used to quickly filter down to the
touched set of tests suites.
In some rare usecases, this can also help to cut down on CI usage for
pull requests. Tread with caution though.
File dependencies are taken into account to ensure that if you touched a
utility file, all relevant tests are still executed.
Closes https://github.com/microsoft/playwright/issues/15075
When merging blob reports test ids are patched to make sure there is no
collision when merging reports that might have overlapping test ids.
However, even if you were merging reports that had no overlapping ids,
all test ids will be modified, which is an undesirable side effect.
This PR only modify test ids when the same test id has already been used
in a previous blob report.
----
This change is also part of
https://github.com/microsoft/playwright/pull/30962
This includes two major changes:
- reuse `SerializedFS` for live test runner tracing;
- merge scheduled `appendFile` operations into a single `fs` call.
In some cases, this improves performance of UI mode by 61% and
performance of `trace: on` mode by 38%. Note that performance
improvement on the average test will not be as noticeable.
References #30875, #30635.
- Documents `PLAYWRIGHT_FORCE_TTY` and `FORCE_COLOR` across terminal
reporters.
- New `PLAYWRIGHT_LIST_PRINT_STEPS`. Removes undocumented test-only
`PW_TEST_DEBUG_REPORTERS_PRINT_STEPS`.
- Replaces `PLAYWRIGHT_HTML_REPORT` with `PLAYWRIGHT_HTML_OUTPUT_DIR`
and `PW_TEST_HTML_REPORT_OPEN` with `PLAYWRIGHT_HTML_OPEN` for
consistency, supports older versions for backwards compatibility.
- New `PLAYWRIGHT_HTML_HOST`, `PLAYWRIGHT_HTML_PORT` and
`PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL`.
- New `PLAYWRIGHT_JUNIT_STRIP_ANSI` and
`PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME`.
- Removes `PW_HTML_REPORT` that was set for unknown reason.
Also:
- remove `blob-report` directory at the start;
- markdown's `report.md` next to package.json;
- use default location in playwright's workflows.
References #24451.
Migrate to version 4 which returns a promise rather than leverages a
callback. -> https://www.npmjs.com/package/rimraf?activeTab=readme
- contains its own types, eliminate "@types/rimraf"
- Parameter `maxBusyTries` changed to `maxRetries`
- `unhandled rejection during beforeAll should be reported and prevent
more tests` had timeout of 100ms that is too short on busy system;
- `runTSC` is always slow, so mark as such.
Running `npx playwright test file:123` will have the following behavior
- if only test files match then only matching subset of tests will run
but all setup files will run as well
- if only setup files match the filter then only those setup tests will
run
- if both setup and test files match an error will be thrown
