Dmitry Gozman dc0665210f

fix(reporters): properly determine flaky status for serial mode (#30529) ...

There are plenty of edge cases in this area:
- interrupted test run;
- did not run because of serial mode failure;
- failed before `test.skip()` call (e.g. in `beforeEach`) in one of the
retries;
- and more...

Related issues: #28322, #28321, #27455, #17652.
Prior changes: #27762, #26385, #28360, probably more.

There is still some duplication between `outcome()` and similar logic in
`base.ts`, which might be deduped in a follow-up.

Fixes #28322.

2024-04-25 13:39:14 -07:00

..

__screenshots__/playwright.artifacts.spec.ts

feat: allow options in automatic screenshots (#19143)

2022-12-07 08:45:33 -08:00

assets

chore: allow calling spawnSync on Node.js file inside test (#26429)

2023-08-11 18:37:14 +02:00

stable-test-runner

chore: roll stable test runner to 1.43.0-beta-1712217576000 (#30250)

2024-04-04 11:26:03 -07:00

access-data.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

babel.spec.ts

chore: allow marking scripts as external for transform (#23449)

2023-06-01 20:28:49 -07:00

basic.spec.ts

chore: rewrite worker main through TestInfoImpl._runAsStage (#29644)

2024-03-04 19:36:58 -08:00

clear-cache.spec.ts

fix(esmLoader): unref MessagePort in the main thread (#30271)

2024-04-06 14:25:57 -07:00

command-line-filter.spec.ts

chore: remove unnecessary args in ttest (#21012)

2023-02-18 13:08:17 -08:00

config.spec.ts

feat: TestProject.ignoreSnapshots (#30466)

2024-04-22 16:16:38 -07:00

decorators.spec.ts

chore: allow ts decorators (#22080)

2023-03-29 20:43:08 -07:00

deps.spec.ts

chore(test runner): do not produce some of the fake skipped test results (#27730)

2023-10-20 17:01:46 -07:00

esm.spec.ts

feat(esm): support import attributes (#30482)

2024-04-23 10:13:01 -07:00

exit-code.spec.ts

fix: collect stdio from forked processes (#27074)

2023-09-14 18:34:26 +02:00

expect-configure.spec.ts

chore: populate matcherResult in web assertions (#27133)

2023-09-16 14:24:10 -07:00

expect-poll.spec.ts

chore: show last poller error on test timeout (#24292)

2023-07-18 17:03:26 -07:00

expect-soft.spec.ts

docs: do not call custom expect message an "error message" (#29390)

2024-02-06 12:12:45 -08:00

expect-to-pass.spec.ts

feat: Support intervals arg for toPass (#30037)

2024-03-29 10:17:29 -07:00

expect.spec.ts

fix(expect): throw better received error when no element was found (#29890)

2024-04-10 10:01:19 +02:00

find-related-tests.spec.ts

chore: rename find-related-tests to find-related-test-files (#29483)

2024-02-13 15:40:49 -08:00

fixture-errors.spec.ts

fix(test runner): update fixture teardown error message (#30109)

2024-03-25 17:04:03 -07:00

fixtures.spec.ts

fix(test runner): make sure auto worker fixtures run before auto test fixtures (#27131)

2023-09-15 14:51:31 -07:00

gitignore.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

global-setup.spec.ts

chore: get rid of ConfigInWorker, use FullConfig instead (#30517)

2024-04-24 14:14:05 -07:00

golden.spec.ts

chore: roll stable test runner to 1.43.0-beta-1712217576000 (#30250)

2024-04-04 11:26:03 -07:00

hooks.spec.ts

fix(reporters): properly determine flaky status for serial mode (#30529)

2024-04-25 13:39:14 -07:00

list-files.spec.ts

chore: implement server-based list files (#29633)

2024-02-23 08:29:44 -08:00

list-mode.spec.ts

Revert "fix: respect .only in --list mode" (#28770)

2023-12-22 11:03:30 -08:00

loader.spec.ts

chore: unconflict ct test ports (#29332)

2024-02-05 16:47:39 -08:00

match-grep.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

max-failures.spec.ts

chore(test runner): do not produce some of the fake skipped test results (#27730)

2023-10-20 17:01:46 -07:00

override-timeout.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

playwright-test-fixtures.ts

feat: exposes tags in testInfo (#29794)

2024-03-06 08:33:06 -08:00

playwright.artifacts.spec.ts

feat(trace mode): add on-first-failure mode for traces (#29647)

2024-02-28 14:39:18 -08:00

playwright.config.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

playwright.config.ts

devops: custom blob report name for test runner tests (#29280)

2024-02-01 09:17:48 -08:00

playwright.connect.spec.ts

Fix various typos (Fixes #27396) (#27391)

2023-10-04 19:56:42 -07:00

playwright.ct-build.spec.ts

fix(ct): allow importing components from node_modules (#30493)

2024-04-24 12:32:28 -07:00

playwright.ct-react.spec.ts

fix(ct): stop-gap for shared file import (#29744)

2024-02-29 19:13:32 -08:00

playwright.fetch.spec.ts

test: unflake "should stop tracing on requestContext.dispose()" (#23489)

2023-06-02 16:37:50 -07:00

playwright.reuse.browser.spec.ts

fix: encode launch options correctly when reusing browser (#24174)

2023-07-12 16:40:55 +02:00

playwright.reuse.spec.ts

fix(reuse): reset Origin Private File System API (#29921)

2024-03-15 17:40:46 +01:00

playwright.spec.ts

chore: delete store.ts (#29423)

2024-02-08 12:55:18 -08:00

playwright.trace.spec.ts

chore(test runner): move timeout handling to the top, stop inheriting runnable (#29857)

2024-03-08 15:19:36 -08:00

playwright.unhandled.spec.ts

fix(test runner): properly handle uncaught errors in test.fail() (#27734)

2023-10-23 09:14:31 -07:00

repeat-each.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

reporter-attachment.spec.ts

fix: quote trace file path when printing error message (#29052)

2024-01-18 14:23:12 -08:00

reporter-base.spec.ts

fix: increase global timeout for a test (#30491)

2024-04-23 16:10:30 -07:00

reporter-blob.spec.ts

fix(merge): include command hash into the report name (#30528)

2024-04-25 13:34:00 -07:00

reporter-dot.spec.ts

chore: run more reporter tests through blob report, some fixes (#23765)

2023-06-16 21:30:55 -07:00

reporter-github.spec.ts

chore: run more reporter tests through blob report, some fixes (#23765)

2023-06-16 21:30:55 -07:00

reporter-html.spec.ts

chore: hide skipped tests by default (#30546)

2024-04-25 12:48:41 -07:00

reporter-json.spec.ts

chore: remove --tag and tagFilter (#29468)

2024-02-15 11:37:16 -08:00

reporter-junit.spec.ts

fix(junit): merged report should preserve total duration (#30525)

2024-04-25 13:34:17 -07:00

reporter-line.spec.ts

chore: run more reporter tests through blob report, some fixes (#23765)

2023-06-16 21:30:55 -07:00

reporter-list.spec.ts

chore: more network panel polish (#26780)

2023-08-29 22:20:28 -07:00

reporter-markdown.spec.ts

fix(merge): normalize path separators when merging across platforms (#28227)

2023-11-27 12:43:56 -08:00

reporter-onend.spec.ts

feat(onEnd): allow overriding the exit code (#27010)

2023-09-12 13:37:30 -07:00

reporter.spec.ts

test: unflake "should work with custom reporter" (#30526)

2024-04-24 20:03:39 -07:00

resolver.spec.ts

fix(tsconfig): when extending, retain pathsBasePath from the original config (#29822)

2024-03-05 16:34:39 -08:00

retry.spec.ts

fix(reporters): properly determine flaky status for serial mode (#30529)

2024-04-25 13:39:14 -07:00

runner.spec.ts

fix(reporters): properly determine flaky status for serial mode (#30529)

2024-04-25 13:39:14 -07:00

shard.spec.ts

Fix various typos (Fixes #27396) (#27391)

2023-10-04 19:56:42 -07:00

snapshot-path-template.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

stdio.spec.ts

feat(test runner): do not mock tty in the worker process (#30107)

2024-03-25 15:31:58 -07:00

test-extend.spec.ts

chore: composed->merge (#27555)

2023-10-11 13:56:27 -07:00

test-grep.spec.ts

Fix various typos (Fixes #27396) (#27391)

2023-10-04 19:56:42 -07:00

test-ignore.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

test-info.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

test-modifiers.spec.ts

fix(reporters): properly determine flaky status for serial mode (#30529)

2024-04-25 13:39:14 -07:00

test-output-dir.spec.ts

chore: trim file names longer than 60 chars (was 100) (#29725)

2024-03-18 12:53:15 -07:00

test-parallel.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

test-serial.spec.ts

fix(reporters): properly determine flaky status for serial mode (#30529)

2024-04-25 13:39:14 -07:00

test-step.spec.ts

test: make expectations readable for more step reporting tests (#30468)

2024-04-22 15:30:51 -07:00

test-tag.spec.ts

feat: exposes tags in testInfo (#29794)

2024-03-06 08:33:06 -08:00

test-use.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00

timeout.spec.ts

chore(test runner): move timeout handling to the top, stop inheriting runnable (#29857)

2024-03-08 15:19:36 -08:00

to-have-screenshot.spec.ts

chore: roll stable test runner to 1.43.0-beta-1712217576000 (#30250)

2024-04-04 11:26:03 -07:00

types-2.spec.ts

feat(test runner): tags/annotations (#29248)

2024-02-07 16:31:25 -08:00

types.spec.ts

fix(components): make sure defineConfig(c1, c2) works (#28608)

2023-12-12 14:01:01 -08:00

ui-mode-fixtures.ts

feat(ui-mode): show native tags in test tree (#30092)

2024-03-26 01:06:22 +01:00

ui-mode-test-attachments.spec.ts

fix(trace-viewer): encode attachment filenames as UTF-8 (#29993)

2024-03-18 21:26:45 +01:00

ui-mode-test-ct.spec.ts

chore: automatically detect the dev server (#29176)

2024-01-25 11:46:47 -08:00

ui-mode-test-filters.spec.ts

feat(ui-mode): show native tags in test tree (#30092)

2024-03-26 01:06:22 +01:00

ui-mode-test-output.spec.ts

chore: implement faster ansi2html format for console (#26826)

2023-09-01 09:09:47 -07:00

ui-mode-test-progress.spec.ts

fix(test runner): make TestTracing responsible for trace management (#29181)

2024-01-26 15:52:25 -08:00

ui-mode-test-run.spec.ts

chore: fix total progress reporting w/ deps (#27716)

2023-10-19 20:07:47 -07:00

ui-mode-test-screencast.spec.ts

chore: do not select after hooks automatically (#27805)

2023-10-25 17:05:06 -07:00

ui-mode-test-setup.spec.ts

fix(ui): print config and global setup errors (#30531)

2024-04-24 18:54:48 -07:00

ui-mode-test-shortcut.spec.ts

chore: address UI Mode keyboard shortcut feedback (#30088)

2024-03-25 19:48:20 +01:00

ui-mode-test-source.spec.ts

chore: do not lose error name for js errors (#28177)

2023-11-15 18:27:32 -08:00

ui-mode-test-tree.spec.ts

fix: rebuild project tree from scratch when listing tests (#30407)

2024-04-18 16:49:07 -07:00

ui-mode-test-update.spec.ts

fix(ui-mode): do not loose run information after writing into testDir (#30312)

2024-04-10 19:01:51 +02:00

ui-mode-test-watch.spec.ts

chore: prepare to reuse test server from ui mode (2) (#29966)

2024-03-18 09:59:02 -07:00

ui-mode-trace.spec.ts

chore(test runner): move timeout handling to the top, stop inheriting runnable (#29857)

2024-03-08 15:19:36 -08:00

watch.spec.ts

fix(test runner): avoid dependency tracking colliding between esm and cjs (#29994)

2024-03-18 17:17:58 -07:00

web-server.spec.ts

fix(HEAD): revert GET->HEAD migration, net-effect was negative (#29738)

2024-02-29 09:02:05 -08:00

worker-index.spec.ts

test: remove magic headers in ttest (#20867)

2023-02-14 19:20:56 -08:00