105 Commits

Author SHA1 Message Date
Pavel Feldman
608e336dba
fix(error): create a step for raw runtime error (#24057)
Fix https://github.com/microsoft/playwright/issues/23850
2023-07-06 10:48:12 -07:00
Yury Semikhatsky
11770156eb
feat(html): show number of filtered tests, update total time (#23743) 2023-06-16 09:22:57 -07:00
Yury Semikhatsky
abdfe264fa
chore: run html reporter tests with merged report (#23626) 2023-06-09 15:41:15 -07:00
Pavel Feldman
d1666d2dde
chore: include test-end-screenshot in a trace (#23457)
Fixes https://github.com/microsoft/playwright/issues/23222
2023-06-01 20:29:32 -07:00
Dmitry Gozman
871032b411
test: unflake some reporter tests (#23264) 2023-05-24 13:28:40 -07:00
Pavel Feldman
b9e7a91368
fix(html): fix the filter to respect status (#23208) 2023-05-22 15:35:19 -07:00
Pavel Feldman
8fee175896
test(filter): add a filter by location test (#23170) 2023-05-19 13:35:11 -07:00
Pavel Feldman
49370cb6fd
chore: fix file view padding (#23166) 2023-05-19 11:07:49 -07:00
Yury Semikhatsky
3ccec7eae5
feat(merge): generate html report with links to remote resources (#22968) 2023-05-12 09:26:04 -07:00
Alex Neo
5bd5cea705
feat(html): describe labels (#22870) 2023-05-08 18:59:01 -07:00
Alex Neo
26cad0b31f
feat(html): matching labels with dash&spec symbol (#22709)
Fixes #22700
2023-05-01 09:13:30 -07:00
Alex Neo
7937699b28
fix(html): label not matched if first with describe (#22680) 2023-04-27 20:04:54 -07:00
Dmitry Gozman
223baa3393
fix(html): keep tests in the declaration order (#22690)
Fixes #22143.
2023-04-27 13:54:15 -07:00
Max Schmitt
49c9284bc7
Revert "feat(html reporter): render multiple annotations of the same type together (#21580)" (#22466)
Fixes https://github.com/microsoft/playwright/issues/22323
2023-04-18 20:43:07 +02:00
mindaugasm
be79ee0450
feat(html-report): add attachmentsBaseURL option (#22212)
Fixes https://github.com/microsoft/playwright/issues/21636
2023-04-18 11:25:11 -07:00
Alex Neo
6947f47f05
feat(html): added labels to test cases and test files (#21468) 2023-03-22 14:35:58 -07:00
Pavel Feldman
b85d670491
chore(ui): show output on demand (#21592) 2023-03-11 11:43:33 -08:00
Dmitry Gozman
21950bc8ce
feat(html reporter): render multiple annotations of the same type together (#21580)
Multiple annotations are rendered as comma-separated lists. Fixes
#21253.

![IMAGE 2023-03-10
14:24:10](https://user-images.githubusercontent.com/9881434/224439435-3bf8e4f4-d874-4595-931d-ea5ff33ea374.jpg)
2023-03-10 15:26:02 -08:00
Pavel Feldman
81bd637d94
chore: merge traces into a single zip file (#21242) 2023-02-27 22:31:47 -08:00
Pavel Feldman
ed41fd0643
chore: use listview to render stack trace (#21197) 2023-02-24 15:31:10 -08:00
Max Schmitt
f3a46f7405
feat(html): render annotations as links if needed (#21165)
Fixes https://github.com/microsoft/playwright/issues/20584
2023-02-23 21:57:02 +01:00
Dmitry Gozman
24c8d45c70
chore: enable tsc for ttest sources (#21018) 2023-02-19 11:18:07 -08:00
Dmitry Gozman
595502ac46
chore: remove unnecessary args in ttest (#21012) 2023-02-18 13:08:17 -08:00
Dmitry Gozman
56dbd1fcb0
chore: remove usesCustomOutputDir from tests (#21011) 2023-02-18 11:41:41 -08:00
Yury Semikhatsky
90c4e6f9b2
chore: revert sharded html report (#20923)
We've decided not to ship it in the current form.

#10437
2023-02-15 12:38:03 -08:00
Dmitry Gozman
91da67fab1
test: remove magic headers in ttest (#20867)
Instead, explicitly import from '@playwright/test'.
2023-02-14 19:20:56 -08:00
Dmitry Gozman
de69b766d9
test: code health in ttest (#20837)
- Remove duplicate tests.
- Remove unused test helpers.
- Print full watch output on failures.
- Unflake some tests.
2023-02-10 20:26:19 -08:00
Pavel Feldman
e1f287f255
chore: more watch tests (#20797) 2023-02-09 16:03:54 -08:00
Yury Semikhatsky
a93cf767a1
feat: html reporter sharded option (#20737)
Make sharded report feature an opt-in:

```ts
{
   reporter: [['html', { sharded: true }]]
};
```

#10437
2023-02-07 22:21:50 -08:00
Pavel Feldman
1b941bcf2e
chore: simplify ttests (#20733) 2023-02-07 15:11:44 -08:00
Yury Semikhatsky
f10b29fd5e
feat: show warning when some report shards are missing (#20731)
When some of the report shards are missing still show the report but
display an error in the status line:

![image](https://user-images.githubusercontent.com/9798949/217371112-2317a307-204e-4c11-8d01-42e91fe6ca7a.png)

#10437
2023-02-07 14:22:10 -08:00
Yury Semikhatsky
8f53bf7b41
feat: sharded html report (#20700)
This implementation is based on the [original
PR](https://github.com/microsoft/playwright/pull/19691) by @kevin940726.
It makes the reporter produce single file when there is no sharding and
multiple out-of-line report-x-of-y.zip reports which are automatically
merged together when put in one folder.

References https://github.com/microsoft/playwright/issues/10437

Co-authored-by: Kai Hao <kevin830726@gmail.com>
2023-02-07 10:21:26 -08:00
Sam Chen
340cacf4fd
feat: show diff tab by default (#19820)
Close https://github.com/microsoft/playwright/issues/19803

I had to manually inject `window.playwrightReportBase64` data into
`playwright/packages/html-reporter/index.html` when developing,
wondering if there's any method I don't know.
2023-02-03 10:32:23 -08:00
Pavel Feldman
2c27bd3b07
chore: remove plugin registration sink (#20460) 2023-01-27 14:36:41 -08:00
Dmitry Gozman
b700c08dc5
feat(screenshots): when actual and expected have different sizes, pad and produce the diff image (#20208)
Also show sizes in the html report to easier spot the size mismatch
issue.

<img width="1030" alt="diff"
src="https://user-images.githubusercontent.com/9881434/213327632-b8fcd69c-8d08-460c-9de1-b5f4f8c56359.png">

Fixes #15802.
2023-01-20 19:41:43 -08:00
Pavel Feldman
7ff27600b4
chore: experimental oop loader (#20269) 2023-01-20 18:24:15 -08:00
Pavel Feldman
d1c161ce99
chore: merge the util exports (#20110) 2023-01-13 13:50:38 -08:00
Max Schmitt
e2a2196e6a
chore(html-reporter): show project name always for the test file (#19905)
Fixes https://github.com/microsoft/playwright/issues/19874
2023-01-10 17:11:38 +01:00
Dmitry Gozman
28577afde4
feat(soft expect): mark steps with failed soft expects as failed (#19973)
Fixes #19673.
2023-01-09 16:17:06 -08:00
Ross Wollman
06c7f1fb6c
fix(html): test and fix reporter timing display (#19670)
#19576 introduced a regression where the CLI reporters displayed some
times with way too many decimals. e.g. 7.123456789ms.

Prior to #19576, there were two monotonicTime implementations; that PR
updated the reporters to use the common definition that had existed in
utils.ts. However, that introduced a regression in the base.ts reporters
which used the ms duration humanizing package which did not account for
the more precise decimals used by the shared monotonicTime function.

This fix removes the dependency on the third-party ms package and now
consistently uses Pavel's humanize function which the HTML reporter had
been using and proved to have better defaults for decimals.
Additionally, we add more test coverage to limit future regressions
since this was caught in passing.

Closes #19556.
Relates #19576.
2022-12-22 15:57:55 -08:00
Ross Wollman
0844394270
feat(html): display overall duration (#19576)
<img width="1390" alt="Screenshot 2022-12-19 at 4 15 33 PM"
src="https://user-images.githubusercontent.com/11915034/208552484-c0127615-d2c6-414f-ae3b-e7836553d890.png">

* Adds duration (time ellapsed from `onBegin` to `onEnd`); roughly
equivalent to `time npx playwright test …`.
* Removes cumulative per-file time

Resolves #19566.
2022-12-20 14:13:10 -08:00
Andrey Lushnikov
6d491f928d
feat(playwright-test): introduce snapshotPathTemplate configuration (#18568)
This configuration option allows to set a string with template
values for precise control over snapshot path location.

An example of `snapshotPathTemplate` usage:

```ts
// playwright.config.ts
// Notice the `testDir` configuration!
export default {
  testDir: './tests',
  snapshotPathTemplate: './__screenshots__/{platform}/{projectName}/{testFilePath}/{arg}{ext}',
}
```

Currently supported "magic tokens" inside the `snapshotPathTemplate`
are:

- `{testDir}` - project's `testDir`
- `{snapshotDir}` - project's `snapshotDir`
- `{platform}` - `process.platform`
- `{projectName}` - Project's sanitized name
- `{testFileDir}` - Directories in relative path from `testDir` to test
  file path (e.g. `page/` in the example below)
- `{testFileName}` - Test file name (with extension) (e.g.
  `page-click.spec.ts` in the example below)
- `{testFilePath}` - Relative path from `testDir` to test file path
  (e.g. `page/page-click.spec.ts` in the example below)
- `{ext}` - snapshot extension (with dots)
- `{arg}` - joined snapshot name parts, without extension (e.g.
`foo/bar/baz` in the example below)
- `{snapshotSuffix}` - `testInfo.snapshotSuffix` value.

Consider the following file structure:

```
playwright.config.ts
tests/
└── page/
    └── page-click.spec.ts
```

The following `page-click.spec.ts`:

```ts
// page-click.spec.ts
import { test, expect } from '@playwright/test';

test('should work', async ({ page }) => {
  await expect(page).toHaveScreenshot(['foo', 'bar', 'baz.png']);
});
```

Fixes #7792
2022-11-09 15:29:07 -08:00
Pavel Feldman
2183d9e9a2
chore: use codemirror for editor (#18482) 2022-11-01 15:04:30 -07:00
Ross Wollman
e73676d094
fix(reporters): output relative to config (#17518)
Fixes #17412.
Supercedes #17413.

- if configured via playwright.config.ts, relative paths should be
relative to the config.
- if configured via env var, should be relative to `cwd`
2022-09-26 11:01:43 -07:00
Andrey Lushnikov
4f11a4d5e7
feat(docker): use the __screenshots__ dir for snapshots by default (#17311)
This patch opts into the `__screnshots__` folder snapshot management
for docker.

With this patch, docker-originating snapshots will be stored in the
following folder:

```
{testDir}/__screenshots__/{projectName}/{testFilePath}/{snapshotName}
```

Where `{testFilePath}` is a test file path relative to `testDir`

Drive-by: introduce and document the `PLAYWRIGHT_DOCKER` environment
variable that enables docker integration.
2022-09-13 15:49:04 -07:00
Max Schmitt
7e2aec7454
chore: align more ESLint rules with VSCode formatting (#16647) 2022-08-18 20:12:33 +02:00
Ross Wollman
a3d99f1b4a
chore: trace viewer fallback error (#16365)
Fixes #16349.
2022-08-09 15:53:11 -07:00
Pavel Feldman
e8070ee1b2
chore: warn users on clashing test output and html reporter folders (#14964) 2022-06-18 15:47:26 -07:00
Ross Wollman
b9114f9cbc
fix(html-reporter): file-browser friendly extensions (#14943)
Fixes #14904.

This is done to make looking at the raw contents of the report
friendlier when using a file browser. However, it should be noted, the
public API of the HTML Reporter makes no guarantees of its contents
structure/layout/naming-conventions.
2022-06-17 08:09:49 -07:00
Dmitry Gozman
dbcf039717
chore: remove experimental types (#14560) 2022-06-01 15:22:43 -07:00