441 Commits

Author SHA1 Message Date
Andrey Lushnikov
2336692e8a
feat: support clarification message for expect (#11735)
The clarification message is displayed in the HTML report as the name of the step:

![image](https://user-images.githubusercontent.com/746130/151852652-48194140-5ea4-439d-afee-12583a8caf71.png)

It is also shown in terminal output:

![image](https://user-images.githubusercontent.com/746130/151852666-5c956ef1-6e94-4bc2-8e55-b58688dfc7e0.png)

Fixes #7816
2022-01-31 17:14:59 -08:00
Pavel Feldman
2b55adaafa
feat(breaking): always report onBegin/onEnd, report file errors (#11758) 2022-01-31 17:09:04 -08:00
Andrey Lushnikov
3a4e506479
chore: extract SigIntWatcher (#11749)
This is to reduce the size of the long `_run` method in the `runner.ts`.
It also might come handy around the codebase.
2022-01-31 08:51:22 -08:00
Andrey Lushnikov
b58b004f0f
fix: run reporter.onEnd after webserver teardown (#11712)
Fixes #11647
2022-01-31 06:19:33 -08:00
Dmitry Gozman
eeebcd53ae
chore(test runner): extract TestInfoImpl (#11725) 2022-01-28 17:39:42 -08:00
Dmitry Gozman
30f3cbd6da
fix(fixtures): make sure defaultLaunchOptions are always there (#11713) 2022-01-28 10:51:55 -08:00
Pavel Feldman
29a84b2df8
chore: allow pre-processing scripts (#11702) 2022-01-27 14:32:23 -08:00
Feng Yu
0698254253
docs: fix broken link (#11687)
fix #11680
2022-01-27 13:00:09 -08:00
Dmitry Gozman
b1fbc4fdbe
fix(test runner): resolve tsconfig for each file (#11662)
This allows us to properly handle path mappings
that are not too ambiguous.
2022-01-26 18:28:42 -08:00
divdavem
512a245f13
feat(test-runner): wait for a url before starting tests (#10138)
The webServer configuration in @playwright/test now accepts a url as an
alternative to a port number to wait for a url to return a 2xx status code.
2022-01-26 16:32:58 -08:00
Dmitry Gozman
19820de7a9
fix(test runner): correctly save videos when running remotely (#11633) 2022-01-26 07:43:07 -08:00
Dmitry Gozman
4f42484c64
chore(test runner): simplify runTestWithBeforeHooks (#11629) 2022-01-25 16:46:50 -08:00
Dmitry Gozman
800b813d4b
chore(test runner): prepare to per-fixture timeout (#11605)
This reworks DeadlineRunner to use exception to signal timeout. This way,
we'll be able to run fixtures against a shared deadline vs their own
deadline and still get an easy control-flow timeout handling.
2022-01-25 11:22:28 -08:00
Andrey Lushnikov
42876a0528
fix: support PWDEBUG=0 to disable debug (#11611)
Fixes #11606
2022-01-25 08:40:24 -08:00
Ross Wollman
64e7557fb9
fix: falsey behavior in route.continue, page.post, testInfo.attach (#11421)
In several of the Playwright APIs, falsey values were not handled correctly. This changeset adds tests (and some fixes):

- route.continue: If options.postData was the empty string, the continue failed to override the post data.
- page.post (application/json with options.data: false|''|0|null): Raw falsey values were getting dropped (i.e. you can't do the equivalent of curl --header application/json … -d 'false'). This has been fixed with most values across all browsers, but an additional fix is needed for 'null' which the channel serializer treats extra specially.
- testInfo.attach: This didn't get reported as an error when options.path was the empty string, but should have been.
#11413 (and its fix #11414) inspired this search as they are the same
class of bug.
2022-01-24 15:06:36 -08:00
Pavel Feldman
9542b007cf
fix(esm): don't emit module shorthand warning (#11596) 2022-01-24 11:22:56 -08:00
Pavel Feldman
0b99d83911
chore: include config dir in quick list report (#11563) 2022-01-22 17:02:46 -08:00
Pavel Feldman
baf0e5b47a
chore: implement lightweight test listing (#11551) 2022-01-21 19:11:22 -08:00
Pavel Feldman
6b3e596fd8
fix(baseurl): support path-less baseurl (#11527) 2022-01-20 18:11:56 -08:00
Pavel Feldman
50fa839e64
chore: include transpilation mode in the compute cache hash (#11521) 2022-01-20 13:33:40 -08:00
Ross Wollman
33f32368e5
fix(test runner): dispatcher transformation of empty attachments (#11414) 2022-01-17 18:47:24 -08:00
Pavel Feldman
8ecf581b45
chore: focus by line should override only (#11427) 2022-01-16 08:47:09 -08:00
Pavel Feldman
8b0ef4ad6b
chore: introduce oop testing mode for debugging (#11428) 2022-01-15 19:25:32 -08:00
Max Schmitt
badb5b4d13
chore(pw-test): forward web-server stdout to debug target (#11384) 2022-01-13 23:55:46 +01:00
Dmitry Gozman
9d5bf0e90d
fix(test runner): fixture teardown double error, testInfo.attach() (#11365)
- Use file path, not content to calculate the attachment hash.
- Always cleanup fixture from the list on teardown, to avoid reporting
  teardown error multiple times: from the test, and from the cleanup.
2022-01-13 10:38:47 -08:00
Pavel Feldman
feb7148b3f
chore: don't write into stdout from the test runner, use reporters instead (#11367) 2022-01-12 19:52:40 -08:00
Andrey Lushnikov
9285596806
chore: cut v1.18.0 (#11358)
Drive-by: fix the `//utils/bump_package_versions.js` script.
2022-01-12 11:50:44 -08:00
Andrey Lushnikov
6f932fcb4a
fix: default list reporter should dump stderr to stderr (#11351)
This is consistent with all other reporters we have in place.
2022-01-12 07:43:25 -08:00
Dmitry Gozman
4efb30999f
feat(request): show request context methods as steps (#11337) 2022-01-11 17:33:41 -08:00
Andrey Lushnikov
2a0930c7a4
docs: clarification about trailing slash in docs (#11326)
Fixes #10557
2022-01-11 07:50:58 -08:00
Dmitry Gozman
da1f39fb29
chore(test runner): refactor worker runner parts (#11316)
This will make it easier to change in the future.
2022-01-10 20:25:56 -08:00
Dmitry Gozman
d36ff8a96c
fix(test runner): remove test output for failures (#11308) 2022-01-10 13:22:09 -08:00
Dmitry Gozman
14fd837e94
fix(test runner): hide beforeAll/afterAll hooks from the reporter api (#11306)
This api is not ready yet.
2022-01-10 12:09:51 -08:00
Andrey Lushnikov
7b1fecc009
fix: default to en-US locale in all browsers in Playwright Test (#11133)
It makes sense to default to en-US by default across all our browsers in Playwright Test.

Fixes #11127
2022-01-10 08:37:55 -08:00
Michaël De Boey
ff5f75dc10
fix: pin colors to 1.4.0 (#11274) 2022-01-09 16:18:21 -08:00
Dmitry Gozman
058f98d3dd
fix(test runner): revert error location from top frame (#11250)
This does not play nicely with some internal Playwright errors, so
it needs more tweaks.
2022-01-07 11:06:47 -08:00
Dmitry Gozman
19460d50e4
chore(test runner): process-wide cache of required test files (#11230)
This is a preparation to TestRunner api, where we should be
able to run tests multiple times in the same runner process.
2022-01-06 15:38:10 -08:00
Pavel Feldman
373042ed89
chore: don't allow importing @playwright/test twice (#11218) 2022-01-06 09:29:05 -08:00
Dmitry Gozman
3ecac56cc0
fix(test runner): testInfo.attach api review changes (#11211)
Remove overload, require name, merge options.
2022-01-05 16:39:33 -08:00
Dmitry Gozman
f77c874e8a
feat(test runner): make _extendTest experimental (#11210)
Hidden from types and docs.
2022-01-05 15:54:00 -08:00
Dmitry Gozman
ccc61e31ea
chore(test runner): preparation for TestRunner api (#11209) 2022-01-05 15:49:01 -08:00
Dmitry Gozman
3839917eb6
chore(test runner): move config loading to runner (#11186)
In preparation for the TestRunner api.
2022-01-05 13:44:29 -08:00
Dmitry Gozman
16a779a5ff
fix(test runner): show codeframe and location from the error top stack frame (#11179)
Previously, reporter would look for a stack frame directly in the test file.
Often times, that is not a top stack frame, especially when the test uses
some helper functions.

This changes error snippets and locations to use the top frame. When top
frame does not match the test file, we additionally show the location
to avoid confusion:

```
  1) a.spec.ts:7:7 › foobar ========================================================================

    Error: oh my

       at helper.ts:5

      3 |
      4 |       export function ohMy() {
    > 5 |         throw new Error('oh my');
        |               ^
      6 |       }
      7 |

        at ohMy (.../reporter-base-should-print-codeframe-from-a-helper/helper.ts:5:15)
        at .../reporter-base-should-print-codeframe-from-a-helper/a.spec.ts:8:9
        at FixtureRunner.resolveParametersAndRunHookOrTest (.../src/fixtures.ts:281:12)
```
2022-01-04 16:00:55 -08:00
Pavel Feldman
7bfa6f9b5f
feat(html): group similar items in the report (#11160) 2022-01-03 21:17:17 -08:00
Pavel Feldman
a0aeaeb929
test: expose repeatEachIndex (#11158) 2022-01-03 17:29:54 -08:00
Ross Wollman
dadb5cbc30
fix(html reporter): fix too much strikethrough in diffs (#11115)
Textual snapshot diffs were previously broken in the HTML Report. The strikethrough'd text extended beyond the intended region.

HTML Report Before: 
<img width="693" alt="Screen Shot 2021-12-27 at 4 43 35 PM" src="https://user-images.githubusercontent.com/11915034/147518750-a60f9002-6eed-48a1-a412-20fabd076fa6.png">

HTML Report After:
<img width="206" alt="Screen Shot 2021-12-27 at 4 48 37 PM" src="https://user-images.githubusercontent.com/11915034/147518762-19a4c8f9-ccc3-4a3c-a962-5a42edc6fc5d.png">

This now matches what's expected and shown in the terminal (which has always been correct):

<img width="1384" alt="Screen Shot 2021-12-27 at 4 36 29 PM" src="https://user-images.githubusercontent.com/11915034/147518799-f538259e-5a45-4d6f-916c-a12ccb620c5b.png">

NB: This MR is a workaround, but not a root cause fix. It works, but I never fully got to the root cause so a bug upstream may be required. It's unclear whether it's (1) in [`colors`](https://www.npmjs.com/package/colors), (2) in [`ansi-to-html`](https://www.npmjs.com/package/ansi-to-html), or (3) Playwright's use of the two. Since the terminal output is correct, I suspect it is in `ansi-to-html`. For example:

```js
const colors = require("colors");
const Convert = require('ansi-to-html');
const convert = new Convert();

// original (strike incorrectly wraps everything in the HTML)
console.log(convert.toHtml(colors.strikethrough("crossed out") + ' ' + colors.red("red")))
// prints: <strike>crossed out <span style="color:#A00">red<span style="color:#FFF"></span></span></strike>

// workaround
console.log(convert.toHtml(colors.reset(colors.strikethrough("crossed out")) + ' ' + colors.red("red")))
// prints: <strike>crossed out</strike> <span style="color:#A00">red<span style="color:#FFF"></span></span>
```

Fixes #11116
2021-12-28 09:56:34 -08:00
Dmitry Gozman
f933759ad1
chore(test runner): minor improvements (#11067)
- Types for fixture options and more.
- Refined type for deadline runner.
2021-12-22 09:59:58 -08:00
Dmitry Gozman
f5304e3bda
fix(fixtures): await fixture teardown when shutting down the worker (#11033) 2021-12-20 16:19:21 -08:00
Dmitry Gozman
2d00836f0e
fix(test runner): show the location of afterAll timeout (#11007) 2021-12-18 09:32:41 -08:00
Dmitry Gozman
c9ba49936f
feat(reporters): show retry #x when running a test (#11003) 2021-12-17 21:07:04 -08:00