This PR moves around some of our CI docs. It moves the GitHub actions
docs from `ci-intro.md` to `ci.md`, reduces `ci-intro.md` to be an
introduction, adds a mention of Sharding to the best practices, and adds
a section on `--only-changed` called "Fail-Fast". Each of those changes
is a separate commit, to make this a little easier to review. If we find
any of those to commits to be contentious, i'll pull them out into
individual PRs.
While rolling this to playwright.dev, we'll also make the following
changes to its sidebar:
- move the `ci.md` document from the "Integrations" section to the
"Playwright Test" section
- make "Best Practices" the last item of the "Getting Started" section
---------
Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Yury Semikhatsky <yurys@chromium.org>
Introduces an `--only-changed [base ref]` option.
`playwright test --only-changed` filters the test run to only run test
suites that have uncommitted changes.
`playwright test --only-changed=foo` runs only tests that were changed
since commit `foo`.
In pull request CI, this can be used to run changed tests first and fail
fast: `--only-changed=$GITHUB_BASE_REF`.
During local development, it can be used to quickly filter down to the
touched set of tests suites.
In some rare usecases, this can also help to cut down on CI usage for
pull requests. Tread with caution though.
File dependencies are taken into account to ensure that if you touched a
utility file, all relevant tests are still executed.
Closes https://github.com/microsoft/playwright/issues/15075
