102 Commits

Author SHA1 Message Date
Max Schmitt
3853014fa7
chore: add ubuntu 22 support (#14588) 2022-06-09 13:20:18 +02:00
Dmitry Gozman
dbcf039717
chore: remove experimental types (#14560) 2022-06-01 15:22:43 -07:00
Yury Semikhatsky
ed344a882b
fix: delete headers with undefined value (#13880) 2022-05-02 12:38:57 -07:00
Pavel Feldman
18bff137ce
chore: remove default config from cli (#13832) 2022-04-28 16:49:36 -07:00
Dmitry Gozman
dab2384b0e
fix(process launcher): use spawnSync to cleanup synchronously (#13769)
This allows us to use the full retry logic of rimraf in the `onexit` handler.
Note this is already covered by failing on Windows test
`should remove temp dir on process.exit`.
2022-04-27 15:01:30 +01:00
Pavel Feldman
3c896f6bc5
fix(tests): installation tests (#13623) 2022-04-18 22:17:56 -07:00
Pavel Feldman
7989427ea6
chore: bundle core deps (#13621) 2022-04-18 20:20:49 -07:00
Pavel Feldman
63ea81ec54
chore: make bundles typed (#13620) 2022-04-18 17:50:25 -07:00
Pavel Feldman
58d79e5e4d
chore: add core utils bundle (#13615) 2022-04-18 13:47:23 -07:00
Pavel Feldman
5f843c347d
chore: bundle pixelmatch, it has not changed for 2 years (#13585) 2022-04-15 16:58:31 -07:00
Ross Wollman
db7bd8ebd2
chore: don't auto-install browsers if global install (#13299) 2022-04-08 10:46:24 -07:00
Dmitry Gozman
dfb1584703
chore: remove MEMUSAGE workaround on Windows (#13408)
This was introduced in #7500 to fight `ERROR: The process "4436" not found.`
messages when killing a process that did already exit.

Since then, we no longer inherit stdout/stderr, so the error message
should not appear anymore.
2022-04-08 09:37:51 -07:00
Pavel Feldman
845e32cbb8
chore: use utils via index export (6) (#13417) 2022-04-07 21:48:41 -07:00
Pavel Feldman
e79b90f454
chore: use utils via index export (5) (#13413) 2022-04-07 20:18:22 -07:00
Dmitry Gozman
155bb7fcae
fix: wait for cleanup on double SIGINT (#13411)
This is a speculative fix to leftover tmp directories.
When users issues SIGINT twice, we enter `gracefullyClose()`
twice, and shortcut the second time. It turns out, we do
not wait for directories removal.

Note: it is unknown how often we reach this codepath in practice.
2022-04-07 19:20:54 -07:00
Yury Semikhatsky
f803a929e4
feat: pass server address as argument (#13412) 2022-04-07 17:22:52 -07:00
Pavel Feldman
a3c02222bf
chore: use utils via index export (4) (#13404) 2022-04-07 14:36:13 -07:00
Pavel Feldman
40d5e3a3c9
chore: use utils via index export (3) (#13403) 2022-04-07 13:55:44 -07:00
Pavel Feldman
6c1fa62000
chore: use utils via index export (2) (#13399) 2022-04-07 13:28:30 -07:00
Pavel Feldman
dbafe3bb4e
chore: use utils via index export (1) (#13398) 2022-04-07 13:26:50 -07:00
Pavel Feldman
bde7bf4ea9
chore: move registry & dispatchers to under server (#13370) 2022-04-06 22:21:27 -07:00
Pavel Feldman
c80365dd43
chore: distill deps after import types update (#13367) 2022-04-06 15:41:08 -07:00
Pavel Feldman
5ae2017a5b
chore: always import type (#13365) 2022-04-06 14:57:14 -07:00
Pavel Feldman
1c3db811f5
chore: move to folder-based deps lists (#13361) 2022-04-06 14:40:19 -07:00
Andrey Lushnikov
a87794dae6
fix: demand --force to re-install stable browser channels (#13200)
This patch will check if browser channel is already installed.
If it is, it'll abort installation with the following error:

```
aslushnikov:~/prog/playwright$ npx playwright install msedge
Failed to install browsers
Error:
╔═════════════════════════════════════════════════════════════════╗
║ ATTENTION: "msedge" is already installed on the system!         ║
║                                                                 ║
║ "msedge" installation is not hermetic; installing newer version ║
║ requires *removal* of a current installation first.             ║
║                                                                 ║
║ To *uninstall* current version and re-install latest "msedge":  ║
║                                                                 ║
║ - Close all running instances of "msedge", if any               ║
║ - Use "--force" to install browser:                             ║
║                                                                 ║
║     npx playwright install --force msedge                       ║
║                                                                 ║
║ <3 Playwright Team                                              ║
╚═════════════════════════════════════════════════════════════════╝
```

To re-install browser channel, use `--force`.

Fixes https://github.com/microsoft/playwright/issues/13061
2022-04-01 10:05:53 -07:00
Philip Sanetra
be41c4a35d
fix(registry): Fix support for generic-linux (#13129)
Fixes https://github.com/microsoft/playwright/issues/13128
2022-03-28 14:50:56 -07:00
Dmitry Gozman
8c19f71c36
feat(selectors): role selector engine (#12999)
This introduces `role=button[name="Click me"][pressed]` attribute-style
role selector. It is only available under `env.PLAYWRIGHT_EXPERIMENTAL_FEATURES`.

Supported attributes:
- `role` is required, for example `role=button`;
- `name` is accessible name, supports matching operators and regular expressions:
  `role=button[name=/Click(me)?/]`;
- `checked` boolean/mixed, for example `role=checkbox[checked=false]`;
- `selected` boolean, for example `role=option[selected]`;
- `expanded` boolean, for example `role=button[expanded=true]`;
- `disabled` boolean, for example `role=button[disabled]`;
- `level` number, for example `role=heading[level=3]`;
- `pressed` boolean/mixed, for example `role=button[pressed="mixed"]`;
- `includeHidden` - by default, only non-hidden elements are considered.
   Passing `role=button[includeHidden]` matches hidden elements as well.
2022-03-28 09:24:58 -07:00
Andrey Lushnikov
4ab4c0bda1
feat: detect docker version and Playwright version mismatch (#12806)
This patch prints a friendly instructions in case Docker image version
mismatches Playwright version and there are missing browser
dependencies.

With this patch, Playwright will yield the following error:

```
root@f0774d2b2097:~# node a.mjs
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

browserType.launch:
╔════════════════════════════════════════════════════════════════════════════════════════════╗
║ Host system is missing dependencies to run browsers.                                       ║
║ This is most likely due to docker image version not matching Playwright version:           ║
║ - Playwright: 1.22.0                                                                       ║
║ -     Docker: 1.21.0                                                                       ║
║                                                                                            ║
║ Either:                                                                                    ║
║ - (recommended) use docker image "mcr.microsoft.com/playwright:v1.22.0-focal"              ║
║ - (alternative 1) run the following command inside docker to install missing dependencies: ║
║                                                                                            ║
║     npx playwright install-deps                                                            ║
║                                                                                            ║
║ - (alternative 2) use Aptitude inside docker:                                              ║
║                                                                                            ║
║     apt-get install libgbm1                                                                ║
║                                                                                            ║
║ <3 Playwright Team                                                                         ║
╚════════════════════════════════════════════════════════════════════════════════════════════╝
    at file:///root/a.mjs:3:10 {
  name: 'Error'
}```

Fixes #12796

Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2022-03-25 14:45:53 -07:00
Andrey Lushnikov
bbc1a4fea0
fix: print all missing libraries when used on unsupported linux distro (#12966)
This patch:
- Adds 3 new host platform types:
  * `generic-linux` and `generic-linux-arm64` for the unsupported
    linux distributions
  * `<unknown>` for non-supported OS versions
- Prints a warning when downloading Ubuntu browser builds on
  unsupported Linux distribution
- Makes sure launch doctor prints all missing shared libraries
  on unknown Linux distributions
- Also prints an `apt` command as an alternative to Playwright CLI
  dependency installation.
2022-03-23 14:06:14 -07:00
Andrey Lushnikov
7f51336068
fix: allow toMatchSnapshot to use text comparator for text data (#12934)
This was regressed awhile ago.

In v1.17 we shipped the following code: 30e15ad36f/packages/playwright-test/src/matchers/golden.ts (L122-L131)

`toMatchSnapshot` should fallback to text comparator in case of
unknown extension and string data.

Fixes #12862
2022-03-21 16:42:21 -07:00
Andrey Lushnikov
c18077c0de
feat(toHaveScreenshot): align screenshot generation & comparison (#12812)
This patch aligns the strategies that are used to generate new
screnshot expectations and to compare screenshot expectations against
baseline.

With this patch, `toHaveScreenshot` will:
- when generating a new expectation: will wait for 2 consecutive
  screenshots to match and accept the last one as expectation.
- when given an expectation:
  * will compare first screenshot against expectation. If matches,
    resolve successfully
  * if first screenshot doesn't match, then wait for 2 consecutive
    screenshots to match and then compare last screenshot with the
    expectation.

An example of a new detailed call log:

```
  1) a.spec.ts:3:1 › should work ===================================================================

    Error: Screenshot comparison failed:

      20000 pixels (ratio 0.03 of all image pixels) are different

    Call log:
      - expect.toHaveScreenshot with timeout 5000ms
      -   verifying given screenshot expectation
      - fast-path: checking first screenshot to match expectation
      - taking page screenshot
      -   disabled all CSS animations
      -   waiting for fonts to load...
      -   fonts in all frames are loaded
      - fast-path failed: first screenshot did not match expectation - 20000 pixels (ratio 0.03 of all image pixels) are different
      - waiting for 2 consecutive screenshots to match
      - waiting 100ms before taking screenshot
      - taking page screenshot
      -   disabled all CSS animations
      -   waiting for fonts to load...
      -   fonts in all frames are loaded
      - 2 consecutive screenshots matched
      - final screenshot did not match expectation - 20000 pixels (ratio 0.03 of all image pixels) are different
      - 20000 pixels (ratio 0.03 of all image pixels) are different

    Expected: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-expected.png
    Received: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-actual.png
        Diff: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-diff.png

      3 | test('should work', async ({ page }) => {
      4 |   await page.goto('file:///Users/andreylushnikov/prog/playwright/tests/assets/rotate-z.html');
    > 5 |   await expect(page).toHaveScreenshot();
        |                      ^
      6 | });
      7 |
```
2022-03-21 15:10:33 -07:00
Max Schmitt
d795f5dd64
chore: print error if install-deps is used != ubuntu (#12877) 2022-03-18 19:13:11 +01:00
Yury Semikhatsky
1284ab8101
feat(webkit): bump to 1617, freeze macOS 10.15 (#12790) 2022-03-15 16:09:56 -07:00
Andrey Lushnikov
88610c8b4c
fix: properly define apiName for web-first assertions (#12706)
Turns out relying on PWTRAP in stack is not reliable: depending on the
call structure, the stack might be cut unpredictably by Node.js.

This patch removes PWTRAP and instead plumbs explicit stack and
pre-set `apiName` all the way down to `wrapApiCall`.
2022-03-14 18:01:13 -07:00
Yury Semikhatsky
d49843edaa
fix(firefox): add missing dependency on Ubuntu 18.04 (#12625) 2022-03-09 11:21:15 -08:00
Dmitry Gozman
682eeb4ae0
reland(hooks): separate test timeout from beforeAll/afterAll timeouts (#12609)
This makes it possible to have longer `beforeAll`/`afterAll` and not
affect first/last test timeout.
2022-03-08 19:05:23 -08:00
Andrey Lushnikov
d7648d390c
fix(toHaveScreenshot): round ratio up (#12540)
This way the ratio value could be used right away in configuation.
2022-03-04 18:55:48 -08:00
Yury Semikhatsky
50cc1641d1
fix(webkit): add libopengl0 to the native deps (#12526) 2022-03-04 11:13:34 -08:00
Andrey Lushnikov
1f5cfcaeec
fix(to-have-screenshot): show number of bad pixels in mismatched error (#12502)
Drive-by: do not show empty call log.

References #12441
2022-03-04 11:01:05 -08:00
Andrey Lushnikov
279e579097
chore: address pixelCount and pixelRatio review comments (#12501)
- documentation fixes
- rename pixelCount to maxDiffPixels
- rename pixelRatio to maxDiffPixelRatio
2022-03-03 23:17:31 -08:00
Pavel Feldman
b79bb32c82
chore: do not stall on actions when test timeout is 0 (#12418) 2022-03-01 13:43:38 -08:00
Pavel Feldman
6a663ef54f
chore(test-runner): revert recent changes to fix tests (#12439)
* Revert "fix(hooks): separate test timeout from beforeAll/afterAll timeouts (#12413)"

This reverts commit 73dee69558110b00916708dafaa99e121c6e7c94.

* Revert "fix(test-runner): rely on test title paths instead of ordinal (#12414)"

This reverts commit d744a87aeefe9c131ed10cedfff201efa1b5c156.

* Revert "chore(test runner): run hooks/modifiers as a part of the test  (#12329)"

This reverts commit 47045ba48dd84d38bde17541c8a0a05cdfed7ffc.
2022-03-01 09:11:17 -08:00
Dmitry Gozman
73dee69558
fix(hooks): separate test timeout from beforeAll/afterAll timeouts (#12413)
This makes it possible to have longer `beforeAll`/`afterAll` and not
affect first/last test timeout.
2022-02-28 15:09:04 -08:00
Andrey Lushnikov
396d920145
feat(test-runner): implement expect(pageOrLocator).toHaveScreenshot (#12242)
Fixes #9938
2022-02-28 12:25:59 -08:00
Max Schmitt
41e9b36f63
fix: install-deps did not throw on non 0 exit code (#12375) 2022-02-26 00:45:27 +01:00
Andrey Lushnikov
df7fbe7857
Revert "chore: best-effort cleanup for output folders that are mounted (#12300)" (#12321)
This reverts commit e5c9d1e39fd0e4bb30e25d69adba2cc7ae3f0071.

Reason for revert: turns out this fix results in a 5-second delay
when starting tests in docker, with `test-results` folder being
a non-removable mount.

The reason for the delay is the `maxBusyTries` option that we
supply by default to rimraf when trying to remove the folder.

While this option might come handy when removing temporary
browser profile folder, it doesn't serve us well in this particular
usecase.

References #12106
2022-02-23 14:10:11 -08:00
Andrey Lushnikov
e5c9d1e39f
chore: best-effort cleanup for output folders that are mounted (#12300)
Fixes #12106
2022-02-23 12:55:41 -08:00
Andrey Lushnikov
0682672242
chore: move comparator logic to playwright-core (#12232)
This will enable implementation of `toHaveScreenshot` on the
server-side.

Drive-by: drop blink-diff

References #9938
2022-02-18 14:39:17 -08:00
Andrey Lushnikov
e6d79a4f10
fix(docker): add missing dependency to the docker 1.19 (#12124)
The `libxtst6` is required in both amd64 and arm64.

Fixes #12075
2022-02-15 10:25:59 -08:00
Dmitry Gozman
fb00991a78
chore: intercept socks proxy in the driver (#12021) 2022-02-13 14:03:47 -08:00