29 Commits

Author SHA1 Message Date
Dmitry Gozman
738d71ebc2
chore: unify two copies of monotonicTime (#15473) 2022-07-07 15:59:49 -07:00
Dmitry Gozman
d7b63fa0b4
fix(test runner): ignore undefined values in fixtures definitions (#15119)
These mean "I don't want to specify this fixture/option"
instead of "I want the value of undefined", aligned with how TypeScript works.
We already do similar things in the config.
2022-06-27 11:31:41 -07:00
Pavel Feldman
007e908cd9
chore: bundle pwt deps (#13622) 2022-04-18 21:47:18 -07:00
Pavel Feldman
7989427ea6
chore: bundle core deps (#13621) 2022-04-18 20:20:49 -07:00
Pavel Feldman
63ea81ec54
chore: make bundles typed (#13620) 2022-04-18 17:50:25 -07:00
Pavel Feldman
4f0103fef0
chore: babel, expect, zip bundles (#13588) 2022-04-18 11:31:58 -07:00
Dmitry Gozman
189a5fa0f8
chore: freez FullProject public type (#13570)
This is similar to FullConfig.
Properties `expect` and `fullyParallel` are not exposed.
2022-04-14 16:58:01 -07:00
Ross Wollman
1af32e400f
feat(test-runner): introduce GlobalInfo (#13083) 2022-04-08 13:22:14 -07:00
Pavel Feldman
40d5e3a3c9
chore: use utils via index export (3) (#13403) 2022-04-07 13:55:44 -07:00
Max Schmitt
5536e64538
feat(test-runner): support filtering by columns (#13401) 2022-04-07 22:45:45 +02:00
Pavel Feldman
5ae2017a5b
chore: always import type (#13365) 2022-04-06 14:57:14 -07:00
Ross Wollman
12abae7f31
feat(test-runner): friendly expect errors for typos (#13229)
If you typo'd an `expect` property, you got a cryptic error message:

```
Uncaught TypeError: Cannot create proxy with a non-object as target or handler
```

Now we get this nice friendly message:

```
  1) a.spec.ts:6:9 › explodes ======================================================================

    Error: expect: Property 'toBeLessThen' not found.

    Did you mean 'toBeLessThan'?

    See https://playwright.dev/docs/test-assertions for available options and documentation.

      5 |         const { test } = pwt;
      6 |         test('explodes', () => {
    > 7 |           expect.soft(1).toBeLessThen();
        |           ^
      8 |         });
      9 |

```

Fixes #13218
2022-04-01 13:38:22 -07:00
Andrey Lushnikov
aa1daeba85
fix(html): put HTML report next to package.json by default (#13141)
Fixes #12970
2022-03-29 14:19:31 -07:00
Pavel Feldman
8758cf8cbf
chore: migrate html reporter to vite (#13116) 2022-03-28 18:21:19 -07:00
Andrey Lushnikov
88610c8b4c
fix: properly define apiName for web-first assertions (#12706)
Turns out relying on PWTRAP in stack is not reliable: depending on the
call structure, the stack might be cut unpredictably by Node.js.

This patch removes PWTRAP and instead plumbs explicit stack and
pre-set `apiName` all the way down to `wrapApiCall`.
2022-03-14 18:01:13 -07:00
Andrey Lushnikov
ef4b34fb7a
feat(toHaveScreenshot): enhance messaging in case of failures (#12677)
This patch:
- adds call logs to track screenshot timeouts, e.g. due to
  waiting for web fonts
- makes sure all snapshot expectations have `.png` extension
- throws a polite error when given a buffer or a string instead of a
  page or a locator
- removes stray NL between error description and call log
- makes sure `apiName` is always correct (and adds a test for it)
2022-03-11 22:40:28 -08:00
Andrey Lushnikov
5879c7f362
chore: refactor toMatchSnapshot once again (#12313)
Keep massaging code in preparation for `toHaveScreenshot`.

References #9938
2022-02-23 13:17:37 -08:00
Dmitry Gozman
4399623f9f
chore(test runner): make timeout a separate error in TestInfo.errors (#12315)
This way we control the timeout error message from the runner,
so that later on we can differentiate between test timeout, fixture
timeout and hook timeout.
2022-02-23 12:32:12 -08:00
Andrey Lushnikov
ef21ce3f56
feat(test-runner): filter out syntax error stack traces (#12095)
Filter out long stack traces from babel when it fails compilation
due to syntax error in test.
2022-02-14 14:33:14 -08:00
Andrey Lushnikov
ba0c7e679b
feat(test-runner): support expect.soft (#11800)
Soft expects will still fail the test, but will not abort it's execution. As a consequence of this:
-  `TestResult` now might have multiple errors, which is reflected with a new `testResult.erros: TestError[]` field.
- `TestInfo` now might have multiple errors as well, which is reflected with a new `testInfo.errors: TestError[]` field.

Fixes #7819
2022-02-02 18:33:51 -08:00
Andrey Lushnikov
b0daa7754f
feat: filter stack traces to exclude test runner frames (#11795)
Before:

```bash
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at Proxy.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/expect.ts:151:30)
        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
        at /Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:335:13
        at runNextTicks (node:internal/process/task_queues:61:5)
        at processImmediate (node:internal/timers:437:9)
        at TestInfoImpl._runFn (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:164:7)
        at WorkerRunner._runTestWithBeforeHooks (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:317:24)
        at TimeoutRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-core/src/utils/async.ts:48:14)
        at TestInfoImpl._runWithTimeout (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/testInfo.ts:151:7)
        at WorkerRunner._runTestOrAllHook (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:276:5)
        at WorkerRunner._runSuite (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:190:11)
        at WorkerRunner.run (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/workerRunner.ts:137:9)
        at process.<anonymous> (/Users/andreylushnikov/prog/playwright/packages/playwright-test/src/worker.ts:87:5)
```

after:

```
Running 1 test using 1 worker
  1) [chromium] › tests/example.spec.ts:3:1 › should work ==========================================

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 2
    Received: 1

      2 |
      3 | test('should work', async({page}) => {
    > 4 |   expect(1).toBe(2);
        |             ^
      5 | });
      6 |

        at /Users/andreylushnikov/tmp/tests/example.spec.ts:4:13
```
2022-02-01 18:40:44 -08:00
Caio Agiani
b6b60decdf
fix: typos (#11789) 2022-02-01 11:09:41 -08:00
Pavel Feldman
2b55adaafa
feat(breaking): always report onBegin/onEnd, report file errors (#11758) 2022-01-31 17:09:04 -08:00
Dmitry Gozman
2d00836f0e
fix(test runner): show the location of afterAll timeout (#11007) 2021-12-18 09:32:41 -08:00
Joel Einbinder
c27491cd4d
feat(test-runner): shorten long output paths (#10523) 2021-12-13 10:56:03 -08:00
Joel Einbinder
20c0facfb9
fix(test runner): dont mangle test names with multiple dashes (#10447) 2021-11-24 19:36:38 -05:00
Dmitry Gozman
8f43f4c98f
feat(serial): better errors from beforeAll (#10419)
When beforeAll hook times out or fails with an exception, we now
close the context and show a nice error.
2021-11-18 14:36:55 -08:00
Joel Einbinder
f2888395a6
fix: support even older node 12 (#9718) 2021-10-22 15:59:52 -04:00
Joel Einbinder
c89d5a50dd
chore: migrate to monorepo (#9421) 2021-10-11 16:52:17 +02:00