Previously, only the "actual" attachment was created, pointing to the
file in `test-results`. Now, the "expected" attachment pointing to the
file in `__screenshots__` is also created. This will help any reporters
that would like to know the "expected" path, for example to do a manual
accept/decline of the baseline.
Fixes#30693.
... unless an array of file-system-friendly parts is provided.
Motivation: attachment name is used as a file system path when
downloading attachments, so we keep them fs-friendly.
References #30693.
When `updateSnapshots === 'missing'` we generate new expectations on the
first attempt and don't retry the test afterwards instead of trying it
retries-1 times and only writing new expectation on the last attempt.
This logic infects all serial mode suites that contain the test with
missing expectations, so they also will not be retried.
Reference https://github.com/microsoft/playwright/issues/29073
This reverts commit 303c5998f8b747e94ac80e2e02c53b47ded11322.
Reason for revert: I tried enabling `ssim-cie94` by default on
ionic-framework test suite, and it proves to be overly strict for their
usecase.
The option defines a comparator to be used to compare images.
Possible values are `"pixelmatch"` and `"ssim-cie94"`.
Note: This reverts commit 8167f8bf548308ad8c6f1188508aadee84f26023.
This patch implements a new image comparison function, codenamed
"ssim-cie94". The goal of the new comparison function is to cancel out
browser non-determenistic rendering.
To use the new comparison function:
```ts
await expect(page).toHaveScreenshot({
comparator: 'ssim-cie94',
});
```
As of Nov 30, 2022, we identified the following sources of
non-determenistic rendering for Chromium:
- Anti-aliasing for certain shapes might be different due to the
way skia rasterizes certain shapes.
- Color blending might be different on `x86` and `aarch64`
architectures.
The new function employs a few heuristics to fight these
differences.
Consider two non-equal image pixels `(r1, g1, b1)` and `(r2, g2, b2)`:
1. If the [CIE94] metric is less then 1.0, then we consider these pixels
**EQUAL**. (The value `1.0` is the [just-noticeable difference] for
[CIE94].). Otherwise, proceed to next step.
1. If all the 8 neighbors of the first pixel match its color, or
if the 8 neighbors of the second pixel match its color, then these
pixels are **DIFFERENT**. (In case of anti-aliasing, some of the
direct neighbors have to be blended up or down.) Otherwise, proceed
to next step.
1. If SSIM in some locality around the different pixels is more than
0.99, then consider this pixels to be **EQUAL**. Otherwise, mark them
as **DIFFERENT**. (Local SSIM for anti-aliased pixels turns out to be
very close to 1.0).
[CIE94]: https://en.wikipedia.org/wiki/Color_difference#CIE94
[just-noticeable difference]:
https://en.wikipedia.org/wiki/Just-noticeable_difference
This configuration option allows to set a string with template
values for precise control over snapshot path location.
An example of `snapshotPathTemplate` usage:
```ts
// playwright.config.ts
// Notice the `testDir` configuration!
export default {
testDir: './tests',
snapshotPathTemplate: './__screenshots__/{platform}/{projectName}/{testFilePath}/{arg}{ext}',
}
```
Currently supported "magic tokens" inside the `snapshotPathTemplate`
are:
- `{testDir}` - project's `testDir`
- `{snapshotDir}` - project's `snapshotDir`
- `{platform}` - `process.platform`
- `{projectName}` - Project's sanitized name
- `{testFileDir}` - Directories in relative path from `testDir` to test
file path (e.g. `page/` in the example below)
- `{testFileName}` - Test file name (with extension) (e.g.
`page-click.spec.ts` in the example below)
- `{testFilePath}` - Relative path from `testDir` to test file path
(e.g. `page/page-click.spec.ts` in the example below)
- `{ext}` - snapshot extension (with dots)
- `{arg}` - joined snapshot name parts, without extension (e.g.
`foo/bar/baz` in the example below)
- `{snapshotSuffix}` - `testInfo.snapshotSuffix` value.
Consider the following file structure:
```
playwright.config.ts
tests/
└── page/
└── page-click.spec.ts
```
The following `page-click.spec.ts`:
```ts
// page-click.spec.ts
import { test, expect } from '@playwright/test';
test('should work', async ({ page }) => {
await expect(page).toHaveScreenshot(['foo', 'bar', 'baz.png']);
});
```
Fixes#7792
This patch opts into the `__screnshots__` folder snapshot management
for docker.
With this patch, docker-originating snapshots will be stored in the
following folder:
```
{testDir}/__screenshots__/{projectName}/{testFilePath}/{snapshotName}
```
Where `{testFilePath}` is a test file path relative to `testDir`
Drive-by: introduce and document the `PLAYWRIGHT_DOCKER` environment
variable that enables docker integration.
Note: all toHaveScreenshot tests still use `__screenshots__` directory
for their expectations. One more test was added to make sure that
by default, `toHaveScreenshot` uses old snapshots.
feat: support experimental doc entries
- Params/options/members are marked as experimental in the docs.
- `experimental.d.ts` is generated that contains all types and
includes experimental features.
- `experimental.d.ts` is references in our tests so that we
can test experimental features.
- `fonts` option is restored as experimental.
