1626 Commits

Author SHA1 Message Date
Dmitry Gozman
776b04e5ea
feat: APIRequestContext dispose reason (#30765)
Similarly to page.close, we pass test-runner specific reason to
facilitate better error messages.

```
  1) a.test.ts:10:11 › test

    Error: apiRequestContext.fetch: Fixture { request } from beforeAll cannot be reused in a test.
      - Recommended fix: use a separate { request } in the test.
      - Alternatively, manually create APIRequestContext in beforeAll and dispose it in afterAll.
    See https://playwright.dev/docs/api-testing#sending-api-requests-from-ui-tests for more details.

       9 |
      10 |       test('test', async () => {
    > 11 |         await context.fetch('http://example.com');
         |                       ^
      12 |       });
      13 |
```

Closes #29260.
2024-05-13 18:51:30 -07:00
Viktor Szépe
150cbcbdf3
chore: fix typos (#30645) 2024-05-08 19:40:03 +01:00
jonghoonpark
a6488c4a28
fix(html-reporter): add filter for anonymous describe (#30621)
related issue: https://github.com/microsoft/playwright/issues/30475

## Motivation:
On https://github.com/microsoft/playwright/issues/30475, we found that
anonymous describe is rendered in html report

## Modification:
Make filter for anonymous describe

## Result:
anonymous describe will be filtered out.
Not render empty describe
Close https://github.com/microsoft/playwright/issues/30475 issue
2024-05-02 09:54:44 -07:00
georg.dev
8173cdc485
fix(reporters): improve detection of output folder clashes (#30607)
When comparing `outputDir` and html-reporter `outputFolder`, we now make
sure that both paths end with a forward-slash.

Fixes #28677

---------

Co-authored-by: Georg Unterholzner <georg.unterholzner@dynatrace.com>
2024-05-02 08:32:57 -07:00
Yury Semikhatsky
3b7c4fac22
chore: add common env vars for junit and json reporters (#30611) 2024-05-01 10:16:49 -07:00
Yury Semikhatsky
6c827121bf
chore: roll stable test runner to 1.44 beta (#30600) 2024-04-29 18:35:21 -07:00
Pavel Feldman
fb90797d73
fix(last-run): remove globalOutputDir (#30571) 2024-04-29 09:02:54 -07:00
Pavel Feldman
b5dee9ecb7
feat(html): allow ctrl+clicking status (#30556) 2024-04-26 10:50:20 -07:00
Tan Li Hau
194479d90e
fix: pass key attribute from jsx to component test (#30426)
When using the `key` attribute in jsx inside the test modules, it is not
serialised and passed to the browser in component test
2024-04-26 09:49:44 -07:00
Yury Semikhatsky
3643fd456b
feat: outputFile option and PLAYWRIGHT_BLOB_OUTPUT_FILE env for blob (#30559)
Reference https://github.com/microsoft/playwright/issues/30091
2024-04-26 09:33:53 -07:00
Yury Semikhatsky
b5aca9fca8
test: relax "should print if globalTimeout is reached" expectation (#30558) 2024-04-25 18:40:58 -07:00
Yury Semikhatsky
9a1b34a4b0
Revert "fix: shut down workers before reporter.onEnd (#30329)" (#30551)
This reverts commit 3cea17abb6c8d7371a6d4dd913f714c427ea0fa0.

Reference https://github.com/microsoft/playwright/issues/30550

The test is preserved but marked as fixme.
2024-04-25 14:21:54 -07:00
Dmitry Gozman
dc0665210f
fix(reporters): properly determine flaky status for serial mode (#30529)
There are plenty of edge cases in this area:
- interrupted test run;
- did not run because of serial mode failure;
- failed before `test.skip()` call (e.g. in `beforeEach`) in one of the
retries;
- and more...

Related issues: #28322, #28321, #27455, #17652.
Prior changes: #27762, #26385, #28360, probably more.

There is still some duplication between `outcome()` and similar logic in
`base.ts`, which might be deduped in a follow-up.

Fixes #28322.
2024-04-25 13:39:14 -07:00
Yury Semikhatsky
5502a16e1d
fix(junit): merged report should preserve total duration (#30525)
Fixes https://github.com/microsoft/playwright/issues/30518
2024-04-25 13:34:17 -07:00
Yury Semikhatsky
714235d6c8
fix(merge): include command hash into the report name (#30528)
Reference https://github.com/microsoft/playwright/issues/30091
2024-04-25 13:34:00 -07:00
Pavel Feldman
4fa1030af6
chore: hide skipped tests by default (#30546)
Fixes https://github.com/microsoft/playwright/issues/30540
2024-04-25 12:48:41 -07:00
Yury Semikhatsky
b074932a44
feat(blob): PLAYWRIGHT_BLOB_FILE_NAME env variable (#30530)
Reference https://github.com/microsoft/playwright/issues/30091
2024-04-25 09:45:38 -07:00
Pavel Feldman
a2eb43b335
feat(runner): allow running last failed tests (#30533)
Fixes: https://github.com/microsoft/playwright/issues/30506
2024-04-25 08:15:27 -07:00
Dmitry Gozman
a6f0a89169
test: unflake "should work with custom reporter" (#30526) 2024-04-24 20:03:39 -07:00
Pavel Feldman
ff3d3ae8f2
fix(ui): print config and global setup errors (#30531)
Fixes: https://github.com/microsoft/playwright/issues/30513
2024-04-24 18:54:48 -07:00
Yury Semikhatsky
f1f3929a67
chore: get rid of ConfigInWorker, use FullConfig instead (#30517)
Addressing API review feedback.
2024-04-24 14:14:05 -07:00
Pavel Feldman
e91d372544
fix(ct): allow importing components from node_modules (#30493)
Fixes https://github.com/microsoft/playwright/issues/29854
2024-04-24 12:32:28 -07:00
Yury Semikhatsky
73c12f1f77
fix: increase global timeout for a test (#30491)
1s is not enought on CI to start running the tests sometimes.
2024-04-23 16:10:30 -07:00
Pavel Feldman
4514b7e3ed
chore: use original test ids in html report (#30486)
Fixes https://github.com/microsoft/playwright/issues/30430
2024-04-23 14:28:27 -07:00
Dmitry Gozman
b52cd7e4dc
test: make sure skipped tests in serial mode are reported (#30484)
This is a regression test for #28321, in preparation for changing some
skipped/flaky test status logic.
2024-04-23 10:51:27 -07:00
Pavel Feldman
f5ca5241f3
feat(esm): support import attributes (#30482)
Fixes https://github.com/microsoft/playwright/issues/30473
2024-04-23 10:13:01 -07:00
Dmitry Gozman
18dcd6adff
test: make expectations for some reporter tests readable (#30470) 2024-04-22 18:29:26 -07:00
Dmitry Gozman
b0fbe058ae
feat: TestProject.ignoreSnapshots (#30466) 2024-04-22 16:16:38 -07:00
Dmitry Gozman
b9e5a934ee
test: make expectations readable for more step reporting tests (#30468) 2024-04-22 15:30:51 -07:00
Yury Semikhatsky
1d786c804d
chore: generate globalTimeout and maxFailures errors in runner (#30467)
Generate 'global timeout' and 'max failures' errors in the runner. Avoid
reading `config.globalTimeout` and `config.maxFailures` in the base
reporters.

Reference https://github.com/microsoft/playwright/issues/29768
2024-04-22 14:00:16 -07:00
Playwright Service
dd1a9f5e96
feat(chromium): roll to r1115 (#30449) 2024-04-20 18:42:22 +02:00
Yury Semikhatsky
8c181f7e2b
fix: rebuild project tree from scratch when listing tests (#30407)
Instead of filtering tests assuming there are no two projects with same
name we always rebuild test tree from scratch and restore previos test
results in the list mode.

Fixes https://github.com/microsoft/playwright/issues/30396
2024-04-18 16:49:07 -07:00
Yury Semikhatsky
cda1c945af
test: web assertion inside expect.poll (#30391)
Reference https://github.com/microsoft/playwright/issues/30322
2024-04-16 17:37:25 -07:00
Yury Semikhatsky
3bdbe4284e
fix(steps): make expect.toPass and expect.poll step containers (#30389)
Fixes https://github.com/microsoft/playwright/issues/30322
2024-04-16 16:18:37 -07:00
Yury Semikhatsky
73fce8fb98
chore: replace Zones with AsyncLocalStorage (#30381)
Reference https://github.com/microsoft/playwright/issues/30322
2024-04-16 12:51:20 -07:00
Yury Semikhatsky
3cea17abb6
fix: shut down workers before reporter.onEnd (#30329) 2024-04-16 09:49:11 -07:00
Yury Semikhatsky
6ad48fb594
chore: make step test expectations readable (#30362)
Reference https://github.com/microsoft/playwright/issues/30322
2024-04-13 20:07:18 -07:00
Yury Semikhatsky
c4418875d3
chore: do skip failing test (#30361) 2024-04-12 15:47:13 -07:00
Yury Semikhatsky
dc1077c5a2
test: steps inside expect.toPass (#30359)
Reference https://github.com/microsoft/playwright/issues/30322
2024-04-12 14:51:10 -07:00
Yury Semikhatsky
1ef85015f3
fix(merge): preserve static annotations on the tests that did not run (#30348)
Fixes https://github.com/microsoft/playwright/issues/30260
2024-04-11 16:17:57 -07:00
Max Schmitt
15b423106f
fix(ui-mode): do not loose run information after writing into testDir (#30312)
Partially reverts https://github.com/microsoft/playwright/pull/30008
that started to reset all test results upon listing tests, including the
test that did just run and triggered re-listing.

https://github.com/microsoft/playwright/issues/30300.
2024-04-10 19:01:51 +02:00
Max Schmitt
b2ded9fed1
fix(expect): throw better received error when no element was found (#29890)
Fixes https://github.com/microsoft/playwright/issues/29873
2024-04-10 10:01:19 +02:00
Dmitry Gozman
a6827772a5
fix(esmLoader): unref MessagePort in the main thread (#30271)
This prevents the process from not exiting forever due to an open port.
2024-04-06 14:25:57 -07:00
Dmitry Gozman
e18a358bc6
test: check that ESM loader works without config with type:module (#30258) 2024-04-05 07:03:27 -07:00
Dmitry Gozman
5d810f1e07
chore: roll stable test runner to 1.43.0-beta-1712217576000 (#30250)
Co-authored-by: Yury Semikhatsky <yurys@chromium.org>
2024-04-04 11:26:03 -07:00
Dmitry Gozman
d5907f4b13
feat(junit): includeProjectInTestName option (#30233)
Fixes #30246.
2024-04-04 11:04:51 -07:00
Yury Semikhatsky
81bcf2a53b
docs: document FullReporter API (#30186)
* FullConfig split into FullConfig and ConfigInWorker
* FullProject split into FullProject and ProjectInWorker
* Tests can only access {Config, Project}InWorker, while reporters can
only access Full{Config,Project}. This will allow in the future to
change the reporter API independently from the tests.
* Added documentation for the 4 classes.
2024-04-02 16:15:02 -07:00
Hansanto
a950a7b86b
feat: Support intervals arg for toPass (#30037)
Fixes #30040

# Context

In the `playwright.config` file, we can custom the default `timeout`
property for `expect.toPass` by the property:

```ts title='playwright.config.ts'
export default defineConfig({
  // ...
  expect: {
    toPass: {
      timeout: 60_000,
    },
  },
});
```

However, we can't customize the `intervals` option. So the default value
is always `[100, 250, 500, 1000]`.

# Goal

Add the possibility to customize the intervals option from the
`playwright.config` file.

```ts title='playwright.config.ts'
export default defineConfig({
  // ...
  expect: {
    toPass: {
      timeout: 60_000,
      intervals: [1000, 2000, 3000]
    },
  },
});
```
2024-03-29 10:17:29 -07:00
Yury Semikhatsky
3001c9ac73
fix: preserve test declaration order in html and merged report (#30159)
* Add `Suite.entries` that returns tests and suites in their declaration
order
* Exposed `Suite.type` and `TestCase.type` for discriminating between
different entry types.
* Blob report format is updated to store entries instead of separate
lists for suites and tests.
* Bumped blob format version to 2, added modernizer.

Fixes https://github.com/microsoft/playwright/issues/29984
2024-03-29 10:12:33 -07:00
Yury Semikhatsky
16318ea715
chore: remove unused botName from types (#30172) 2024-03-28 16:16:15 -07:00