268 Commits

Author SHA1 Message Date
Pavel Feldman
81bd637d94
chore: merge traces into a single zip file (#21242) 2023-02-27 22:31:47 -08:00
Pavel Feldman
09f77c41dd
chore: migrate to protocol's StackFrame type (#21198) 2023-02-24 18:36:15 -08:00
Pavel Feldman
3fa19e80ad
chore: wrap expect call in a zone (#21191) 2023-02-24 12:17:03 -08:00
Pavel Feldman
b61036d22a
chore(trace): dedupe filenames, not stack frames (#21160) 2023-02-23 09:10:09 -08:00
Pavel Feldman
55c95a4463
chore: do not send stacks as a part of the call metainfo (#21089) 2023-02-22 21:08:47 -08:00
Max Schmitt
ae369144b6
chore: consolidate http/https fetching (#21104)
Fixes https://github.com/microsoft/playwright/issues/20784
Supersedes https://github.com/microsoft/playwright/pull/21076
2023-02-22 17:09:56 +01:00
Pavel Feldman
3f8f2a0fdd
chore: do not use library stack capturer in test runner (#21075) 2023-02-21 19:24:17 -08:00
Pavel Feldman
06fc72b6ed
revert(#12706): also fix related bugs it introduced (#21070) 2023-02-21 14:15:11 -08:00
Dmitry Gozman
24c8d45c70
chore: enable tsc for ttest sources (#21018) 2023-02-19 11:18:07 -08:00
Andrey Lushnikov
4feb7c9e3b
fix: best-effort support for KDE Neon (#20920)
This is a yet-another linux distribution based on Ubuntu.

Fixes #20838
2023-02-15 09:13:14 -08:00
Andrey Lushnikov
fdcd7b549d
chore: mark comparator option back as experimental (#20816)
This reverts commit 303c5998f8b747e94ac80e2e02c53b47ded11322.

Reason for revert: I tried enabling `ssim-cie94` by default on
ionic-framework test suite, and it proves to be overly strict for their
usecase.
2023-02-13 11:11:44 -08:00
Pavel Feldman
0678b6575f
chore(watch): allow toggling browser (#20738) 2023-02-08 08:36:02 -08:00
Andrey Lushnikov
303c5998f8
feat: release "comparator" option from experiment (#20720)
The option defines a comparator to be used to compare images.
Possible values are `"pixelmatch"` and `"ssim-cie94"`.

Note: This reverts commit 8167f8bf548308ad8c6f1188508aadee84f26023.
2023-02-07 08:51:48 -08:00
Dmitry Gozman
c27ad35299
fix(locators): properly escape slash inside attributes (#20510)
Fixes #20471.
2023-01-30 13:27:41 -08:00
Dmitry Gozman
38f260cc51
test: unflake "should respect interval" (#20298) 2023-01-23 17:57:37 -08:00
Max Schmitt
05843bdfd9
chore: support Debian 11 arm64 (#19676)
https://github.com/microsoft/playwright/issues/18319
2023-01-23 15:54:25 +01:00
Dmitry Gozman
b700c08dc5
feat(screenshots): when actual and expected have different sizes, pad and produce the diff image (#20208)
Also show sizes in the html report to easier spot the size mismatch
issue.

<img width="1030" alt="diff"
src="https://user-images.githubusercontent.com/9881434/213327632-b8fcd69c-8d08-460c-9de1-b5f4f8c56359.png">

Fixes #15802.
2023-01-20 19:41:43 -08:00
Dmitry Gozman
eafa6fda13
fix(expect): poll/toPass should not wait over specified timeout (#20266)
Drive-by: unflake some timeout-dependent tests.
2023-01-20 15:47:24 -08:00
Max Schmitt
df05c00da3
fix: filter out pwt stacks correctly (#20158)
Fixes https://github.com/microsoft/playwright/issues/20155
2023-01-17 22:38:30 +01:00
Pavel Feldman
d1c161ce99
chore: merge the util exports (#20110) 2023-01-13 13:50:38 -08:00
Andrey Lushnikov
1afa38d5a7
chore(expect): extract polling from expect.poll and expect().toPass (#19882)
This extracts & unifies polling machinery from `expect.poll` and
`expect.toPass` methods.
2023-01-05 11:14:37 -08:00
Max Schmitt
7508c574e1
chore: bring ubuntu bionic back (#19625) 2022-12-21 18:37:42 +01:00
Andrey Lushnikov
8167f8bf54
chore: hide "comparator" option from documentation and types (#19441) 2022-12-14 09:58:19 -08:00
Pavel Feldman
6cadc56ea3
feat(api): allow getByTestId(regex) (#19419) 2022-12-13 08:43:13 -08:00
Andrey Lushnikov
cac67fb94f
chore: do not use "threshold" option for the ssim-cie94 comparator (#19314) 2022-12-06 17:03:13 -08:00
Dmitry Gozman
d294c5da33
chore: roll stable-test-runner to 1.28.1 (#19310)
This required `allowJs: false` in our `tsconfig.json` due to the
following error:

```

Error: Cannot find module './utilsBundleImpl'
Require stack:
- <playwright>/packages/playwright-core/src/utilsBundle.ts
- <playwright>/packages/playwright-test/lib/reporters/html.js
- <playwright>/tests/playwright-test/reporter-html.spec.ts
- <playwright>/tests/playwright-test/stable-test-runner/node_modules/@playwright/test/lib/loader.js
- <playwright>/tests/playwright-test/stable-test-runner/node_modules/@playwright/test/lib/runner.js
- <playwright>/tests/playwright-test/stable-test-runner/node_modules/@playwright/test/lib/cli.js
- <playwright>/tests/playwright-test/stable-test-runner/node_modules/playwright-core/lib/cli/cli.js
- <playwright>/tests/playwright-test/stable-test-runner/node_modules/playwright-core/cli.js
- <playwright>/tests/playwright-test/stable-test-runner/node_modules/@playwright/test/cli.js

   at ../../packages/playwright-core/src/utilsBundle.ts:20

  18 | import path from 'path';
  19 |
> 20 | export const colors: typeof import('../bundles/utils/node_modules/colors/safe') = require('./utilsBundleImpl').colors;
     |                                                                                   ^
  21 | export const debug: typeof import('../bundles/utils/node_modules/@types/debug') = require('./utilsBundleImpl').debug;
  22 | export const getProxyForUrl: typeof import('../bundles/utils/node_modules/@types/proxy-from-env').getProxyForUrl = require('./utilsBundleImpl').getProxyForUrl;
  23 | export const HttpsProxyAgent: typeof import('../bundles/utils/node_modules/https-proxy-agent').HttpsProxyAgent = require('./utilsBundleImpl').HttpsProxyAgent;

    at Object.<anonymous> (<playwright>/packages/playwright-core/src/utilsBundle.ts:20:83)
    at Object.<anonymous> (<playwright>/packages/playwright-test/src/reporters/html.ts:17:1)

```
2022-12-06 15:46:19 -08:00
Dmitry Gozman
48182a4eb2
chore: refactor code around text selectors (#19278) 2022-12-05 14:08:54 -08:00
Andrey Lushnikov
6d82460a02
feat: implement a new image comparison function (#19166)
This patch implements a new image comparison function, codenamed
"ssim-cie94". The goal of the new comparison function is to cancel out
browser non-determenistic rendering.

To use the new comparison function:

```ts
await expect(page).toHaveScreenshot({
  comparator: 'ssim-cie94',
});
```

As of Nov 30, 2022, we identified the following sources of
non-determenistic rendering for Chromium:
- Anti-aliasing for certain shapes might be different due to the
  way skia rasterizes certain shapes.
- Color blending might be different on `x86` and `aarch64`
architectures.

The new function employs a few heuristics to fight these
differences.

Consider two non-equal image pixels `(r1, g1, b1)` and `(r2, g2, b2)`:
1. If the [CIE94] metric is less then 1.0, then we consider these pixels
   **EQUAL**. (The value `1.0` is the [just-noticeable difference] for
   [CIE94].). Otherwise, proceed to next step.
1. If all the 8 neighbors of the first pixel match its color, or
   if the 8 neighbors of the second pixel match its color, then these
   pixels are **DIFFERENT**. (In case of anti-aliasing, some of the
   direct neighbors have to be blended up or down.) Otherwise, proceed
   to next step.
1. If SSIM in some locality around the different pixels is more than
   0.99, then consider this pixels to be **EQUAL**. Otherwise, mark them
   as **DIFFERENT**. (Local SSIM for anti-aliased pixels turns out to be
   very close to 1.0).

[CIE94]: https://en.wikipedia.org/wiki/Color_difference#CIE94
[just-noticeable difference]:
https://en.wikipedia.org/wiki/Just-noticeable_difference
2022-12-02 15:22:05 -08:00
Max Schmitt
6a5552ba24
chore: drop Ubuntu 18.04 support (#18924) 2022-11-21 11:23:45 -10:00
Pavel Feldman
4bbfcef792 Revert "chore: support narrow terminal windows for messages (#18714)"
This reverts commit 23171c5037ed487e7be0b561d05c458331f5ca7c.
2022-11-14 08:47:17 -08:00
Pavel Feldman
8e882fdd58
chore: make generated report writable (#18750)
Fixes: https://github.com/microsoft/playwright/issues/18747
2022-11-13 12:46:35 -08:00
Dmitry Gozman
a7b2b04588
fix(getByRole): name and exact (#18719)
Following the `getByText()` and other methods:

- By default, matching is substring and case-insensitive. Before, it was
only case-insensitive, but not substring.
- With new option `exact: true`, matching is full string and
case-sensitive.
- Matching always normalizes whitespace.
- Codegen generates `exact: false` by default.
- `internal:role` treats `[name="foo"i]` as non-exact match.

Various fixes:
- Updated `getByRole` docs to match the reality.
- Locator generator edge cases.
2022-11-11 15:58:36 -08:00
Andrey Lushnikov
dfb4ad388a
feat: support custom png comparator (#18689)
This way we might experiment with different custom PNG comparators
for VRT.
2022-11-10 16:16:05 -08:00
Andrey Lushnikov
23171c5037
chore: support narrow terminal windows for messages (#18714)
This patch starts using a message box that's not really a box and thus
is better behaving on a narrow-width terminals.

Before:
<img width="744" alt="image"
src="https://user-images.githubusercontent.com/746130/201216551-abbac0f8-71b4-413f-9f4e-159c7123ef3d.png">

After:
<img width="745" alt="image"
src="https://user-images.githubusercontent.com/746130/201216504-25257727-06c8-4ae9-8557-a2d937b7ca0b.png">

Signed-off-by: Andrey Lushnikov <aslushnikov@gmail.com>
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2022-11-10 15:50:52 -08:00
Yury Semikhatsky
0765182a4d
fix: conversion to snake case (#18707)
Fixes https://github.com/microsoft/playwright/issues/18695
2022-11-10 14:54:07 -08:00
Andrey Lushnikov
9bcb28f25a
Revert "feat: send SIGTERM to webserver before SIGKILL'ing it. (#18220)" (#18661)
This reverts commit c63a0b536d1f0119794a909e4f9c420c8506b4d5.

Reason: https://github.com/microsoft/playwright/pull/18564
2022-11-09 09:18:33 -08:00
Pavel Feldman
0355d8618f
chore: use provided value for the generated test id (#18631) 2022-11-08 12:04:43 -08:00
Pavel Feldman
8432d1592f
chore: expose new locator apis on window.playwright (#18595) 2022-11-07 09:06:13 -08:00
Pavel Feldman
2c3fa1b1ff
chore: add explore locator parser (#18429) 2022-10-31 12:55:35 -07:00
Andrey Lushnikov
c63a0b536d
feat: send SIGTERM to webserver before SIGKILL'ing it. (#18220)
We now will send `SIGTERM` to the webserver and wait for the `timeout`
before sending `SIGKILL` to it.

Fixes #18209
2022-10-21 08:55:06 -07:00
Pavel Feldman
304a4ee8ec
chore: migrate to the internal:text selector (#18135) 2022-10-18 13:09:54 -07:00
Dmitry Gozman
b64457d8ce
fix(locators): make regex escape work when multiple spaces are present (#17893) 2022-10-06 16:23:40 -07:00
Pavel Feldman
8b018f6b41
chore: make role name case-insensitive (#17888) 2022-10-06 14:35:10 -07:00
Pavel Feldman
f2685cab95
chore: use api selectors in codegen hover (#17855) 2022-10-05 17:59:34 -07:00
Pavel Feldman
3ecaa36e25
chore: make locators generator isomorphic (#17850) 2022-10-05 13:13:22 -07:00
Dmitry Gozman
2bcd9ce9ae
chore: internal selectors (#17827)
- Rename internal selectors `has`, `control` and `attr` to
`internal:has`, `internal:control` and `internal:attr`.
- Fix `getByLabel()` to respect strictness, by introducing
`internal:label` selector.
- Move tests essential for ports to `selectors-by.spec`.
2022-10-05 08:45:10 -07:00
Dmitry Gozman
a9c02b9753
fix(locators): properly escape various locator methods (#17798)
References #17604.
2022-10-04 13:21:38 -07:00
Pavel Feldman
1311767f87
chore: generate api calls (#17794) 2022-10-03 17:14:02 -07:00
Dmitry Gozman
3409a37f77
feat: allow opening multiple html reporters and trace viewers (#17636)
This makes `HttpServer` accept `preferredPort` option that will first
try to listen on that port, and if that port is already in use, listen
on some available port instead.

Fixes #17201.
2022-09-27 12:45:42 -07:00
Andrey Lushnikov
ec40e49978
feat: expose configuration for HTML report serving. (#17560)
Fixes #16667
2022-09-25 20:36:38 -07:00