97 Commits

Author SHA1 Message Date
Yury Semikhatsky
05fb3e6de8
Revert "chore: hide TestProject.setup and TestInfo.storage (#18800)" (#18836)
Now that the patch has been cherry-picked to the release branch
reverting it on main.

This reverts commit 6deba5dc21f8945635663119c1cd5db5c3776444.
2022-11-16 11:23:10 -08:00
Yury Semikhatsky
6deba5dc21
chore: hide TestProject.setup and TestInfo.storage (#18800) 2022-11-14 17:03:01 -08:00
Dmitry Gozman
194abc35cd
Revert #18561 "feat(test runner): test.reset() to reset options to default/config value" (#18704)
This reverts commit 6fef227f43f122d45327768addc19bb96db0f6d5.

Not shipping in 1.28.
2022-11-10 14:45:05 -08:00
Andrey Lushnikov
f52fa4ceba
chore(snapshotPathTemplate): address review comments (#18702) 2022-11-10 11:37:41 -08:00
Andrey Lushnikov
6d491f928d
feat(playwright-test): introduce snapshotPathTemplate configuration (#18568)
This configuration option allows to set a string with template
values for precise control over snapshot path location.

An example of `snapshotPathTemplate` usage:

```ts
// playwright.config.ts
// Notice the `testDir` configuration!
export default {
  testDir: './tests',
  snapshotPathTemplate: './__screenshots__/{platform}/{projectName}/{testFilePath}/{arg}{ext}',
}
```

Currently supported "magic tokens" inside the `snapshotPathTemplate`
are:

- `{testDir}` - project's `testDir`
- `{snapshotDir}` - project's `snapshotDir`
- `{platform}` - `process.platform`
- `{projectName}` - Project's sanitized name
- `{testFileDir}` - Directories in relative path from `testDir` to test
  file path (e.g. `page/` in the example below)
- `{testFileName}` - Test file name (with extension) (e.g.
  `page-click.spec.ts` in the example below)
- `{testFilePath}` - Relative path from `testDir` to test file path
  (e.g. `page/page-click.spec.ts` in the example below)
- `{ext}` - snapshot extension (with dots)
- `{arg}` - joined snapshot name parts, without extension (e.g.
`foo/bar/baz` in the example below)
- `{snapshotSuffix}` - `testInfo.snapshotSuffix` value.

Consider the following file structure:

```
playwright.config.ts
tests/
└── page/
    └── page-click.spec.ts
```

The following `page-click.spec.ts`:

```ts
// page-click.spec.ts
import { test, expect } from '@playwright/test';

test('should work', async ({ page }) => {
  await expect(page).toHaveScreenshot(['foo', 'bar', 'baz.png']);
});
```

Fixes #7792
2022-11-09 15:29:07 -08:00
Yury Semikhatsky
25dc0bfacb
feat(runner): change storage fixture to TestInfo.storage() (#18584) 2022-11-04 14:28:25 -07:00
Dmitry Gozman
6fef227f43
feat(test runner): test.reset() to reset options to default/config value (#18561) 2022-11-04 09:03:29 -07:00
Yury Semikhatsky
1d2fc1e963
feat(runner): project.setup (#18486) 2022-11-01 23:44:30 -07:00
Yury Semikhatsky
7337dd4e28
chore: revert project.{stage,run} (#18462) 2022-10-31 14:04:24 -07:00
Dmitry Gozman
2d3b2a0768
feat(test runner): test.describe.configure({ retries, timeout }) (#18388)
References #10825.
2022-10-27 15:53:27 -07:00
Yury Semikhatsky
11eb719d13
feat(runner): project run: "always" (#18160)
Projects marked with `run: 'always'` are non shard-able and run after
failures.
2022-10-18 17:18:45 -07:00
Yury Semikhatsky
08a3a269cd
feat(runner): project.canShard (#18037) 2022-10-12 14:34:22 -07:00
Yury Semikhatsky
3b8f63d703
feat(runner): project.stopOnFailure (#18009) 2022-10-11 17:04:01 -07:00
Yury Semikhatsky
3592269caf
feat(runner): project.stage (#17971) 2022-10-10 17:56:18 -07:00
Yury Semikhatsky
2d72d0ba03
chore: remove config.groups (#17974) 2022-10-10 16:42:48 -07:00
Dmitry Gozman
6750a02506
fix(api): hide TestConfig.groups (#17853)
Also fix types for `FullConfig.configFile`.
2022-10-05 14:24:14 -07:00
Yury Semikhatsky
51966bc045
fix: --grep and --grep-invert should intersect with config (#17716)
Fixes https://github.com/microsoft/playwright/issues/17405
2022-09-29 16:39:21 -07:00
Pavel Feldman
a97deb1600
chore: hide reporter project id (#17710) 2022-09-29 12:51:06 -07:00
Yury Semikhatsky
43ccd7bc96
chore(runner): remove watch mode (#17682) 2022-09-29 10:20:21 -07:00
Yury Semikhatsky
9f17ee6871
feat: group filter (#17646) 2022-09-28 18:45:01 -07:00
Pavel Feldman
3a13a897b9
Revert "fix(pwt): compatibility in CWD with wrong casing (#16636)" (#17599) 2022-09-26 18:53:08 -07:00
Yury Semikhatsky
10d7c60abf
feat(runner): project execution schedule (#17548) 2022-09-23 20:01:27 -07:00
Jean-François Greffier
a15fe50e7b
feat(test runner): workers as percentage (#17400)
Allows to set workers as a percentage of logical CPUs, for example
"50%".

Examples :
```bash
npx playwright test --workers 3
npx playwright test --workers 50%
```

```js
const config: PlaywrightTestConfig = {
  // ...
  workers: '33%',
}
```
2022-09-21 11:17:36 -07:00
Pavel Feldman
854c783019
chore: project.id, configFile in reporter apis (#17346) 2022-09-14 14:56:28 -07:00
Andrey Lushnikov
4f11a4d5e7
feat(docker): use the __screenshots__ dir for snapshots by default (#17311)
This patch opts into the `__screnshots__` folder snapshot management
for docker.

With this patch, docker-originating snapshots will be stored in the
following folder:

```
{testDir}/__screenshots__/{projectName}/{testFilePath}/{snapshotName}
```

Where `{testFilePath}` is a test file path relative to `testDir`

Drive-by: introduce and document the `PLAYWRIGHT_DOCKER` environment
variable that enables docker integration.
2022-09-13 15:49:04 -07:00
Andrey Lushnikov
fafd9837ba
feat: introduce the --ignore-snapshots option (#17004)
This patch introduces `--ignore-snapshots` Playwright Test CLI option,
and `ignoreSnapshots` configuration argument.
2022-09-01 05:34:36 -07:00
Max Schmitt
e344fe60e7
fix(pwt): compatibility in CWD with wrong casing (#16636) 2022-08-22 21:45:59 +02:00
Max Schmitt
7e2aec7454
chore: align more ESLint rules with VSCode formatting (#16647) 2022-08-18 20:12:33 +02:00
Dmitry Gozman
7a86e140f5
chore: remove TestProject.projectSetup (#16321)
There are better ideas to address this issue in more general manner.
2022-08-05 15:24:30 -07:00
Pavel Feldman
7645ac25a0
chore: watch mode straw man (#16127) 2022-08-04 08:09:54 -07:00
Pavel Feldman
de147fafba
chore: experimental project isolation mode (#16081) 2022-08-02 11:22:22 -07:00
Dmitry Gozman
3112edb4ca
feat(test runner): TestProject.projectSetup (#16063)
`projectSetup` is a project-scoped alternative to `globalSetup`.
It is only executed if at least one test from the project is scheduled to run.
2022-08-01 09:01:23 -07:00
Pavel Feldman
8c190c3e09
feat(testId): expose persistent test id (#16003) 2022-07-27 20:17:19 -07:00
Dmitry Gozman
6009804e0e
Revert "fix(test runner): ignore undefined values in fixtures definit…ions (#15119)" (#15979)
Revert "fix(test runner): ignore undefined values in fixtures definitions (#15119)"

Revert commit d7b63fa0b45ab12e40d0018b0e6f796ecd2df4b7.
Add a test for the broken behavior.
2022-07-27 08:51:45 -07:00
Ross Wollman
799d4703bc
feature(test-runner): multiple web servers (#15388)
Fixes #8206.

Since #8206 is a long-awaited (~ 1 year old), popular (~ 45 reactions, frequently requested in community channels, etc.), this PR aims to unblock folks.

Notably, we do not innovate on the `webServer` API, despite knowing we're not in love with it. We'll save the innovation for either Plugins or a new `LaunchConfigs` option. (We haven't yet arrived at a Plugin API we like, and instead of launching a new option guessing what the "better" launchConfig API would be, let's wait and see how folks use this new Array-variant of `webServer` which—despite its name—can be used for non-Web Server launches!
2022-07-07 15:27:21 -07:00
Dmitry Gozman
89c16cbc8b
feat(runner): anonymous describes (#15424)
Useful to set some options without coming up with an artificial name.

```js
test.describe(() => {
  test.use({ colorScheme: 'dark' });

  test('my test', () => {});
})
```
2022-07-06 13:54:11 -07:00
Dmitry Gozman
d7b63fa0b4
fix(test runner): ignore undefined values in fixtures definitions (#15119)
These mean "I don't want to specify this fixture/option"
instead of "I want the value of undefined", aligned with how TypeScript works.
We already do similar things in the config.
2022-06-27 11:31:41 -07:00
Pavel Feldman
88664c39c9
chore: remove esm error interception, it only confuses users (#14829) 2022-06-13 12:32:42 -07:00
Pavel Feldman
c4b003a29a
chore: don't mask ESM error (#14628) 2022-06-03 15:28:53 -07:00
Yury Semikhatsky
b164d82ba3
fix(runner): ignore .gitignore if testDir is explicitly configured (#14430)
If the tests are in an explicitly configured testDir (either at the global config level or per project) .gitignore filters are not applied.

Fixes #14381
2022-05-26 14:39:51 -07:00
Pavel Feldman
9440f52b88
feat(ct): support cra w/ .js (#14326) 2022-05-24 19:43:28 -07:00
Pavel Feldman
e9378ba5fc
chore: hide plugins again (#14038) 2022-05-09 10:06:13 -07:00
Pavel Feldman
1bbefce9af
Revert "chore: use plugins for component testing again (#13977)" (#13998)
This reverts commit a2f9f15e3e2419d24b8b1deb87d304c363f93435.
2022-05-06 12:02:07 -07:00
Pavel Feldman
a2f9f15e3e
chore: use plugins for component testing again (#13977) 2022-05-05 14:26:56 -07:00
Pavel Feldman
058f32caff
chore: fixtures-via-plugin implementation (#13950) 2022-05-05 10:14:00 -07:00
Pavel Feldman
29fd1d86df
chore: hide plugins for now (#13908) 2022-05-03 14:25:56 -07:00
Pavel Feldman
19ce4e3514
chore: refactor web server plugin to extract legacy (#13904) 2022-05-03 11:47:37 -07:00
Dmitry Gozman
ef32069299
chore: small test runner changes in preparation of global fixtures (#13899)
The main change is splitting up options from the config from other
fixtures to ensure unique location for them.
2022-05-03 15:19:27 +01:00
Dmitry Gozman
4f5fbea26f
chore: get rid of ProjectImpl (#13894) 2022-05-03 13:36:24 +01:00
Ross Wollman
3b3cad7d69
feat: rewrite gitCommitInfo plugin, drop GlobalInfo & attachments (#13837) 2022-05-02 16:28:14 -07:00