Previously, reporter would look for a stack frame directly in the test file.
Often times, that is not a top stack frame, especially when the test uses
some helper functions.
This changes error snippets and locations to use the top frame. When top
frame does not match the test file, we additionally show the location
to avoid confusion:
```
1) a.spec.ts:7:7 › foobar ========================================================================
Error: oh my
at helper.ts:5
3 |
4 | export function ohMy() {
> 5 | throw new Error('oh my');
| ^
6 | }
7 |
at ohMy (.../reporter-base-should-print-codeframe-from-a-helper/helper.ts:5:15)
at .../reporter-base-should-print-codeframe-from-a-helper/a.spec.ts:8:9
at FixtureRunner.resolveParametersAndRunHookOrTest (.../src/fixtures.ts:281:12)
```
This changes previous layout shift attempt (see #9546)
to account for more valid usecases:
- On the first event that is intercepted we enforce the hit target. This
is similar to the current mode that checks hit target before the action,
but is better timed.
- On subsequent events we assume that everything is fine. This covers more
scenarios like react rerender, glass pane on mousedown, detach on mouseup.
This check is enabled by default, with `process.env.PLAYWRIGHT_NO_LAYOUT_SHIFT_CHECK`
to opt out.
This patch introduces 109 "#smoke" tests - a subset of tests that makes
sure that basic Playwright functionality works. This set is loosely
defined; feel free to add/remove tests to the set. The only goal is to
keep this set minimal & fast to run.
I tried to pick tests so that various parts of Playwright functionality
are exercised.
Textual snapshot diffs were previously broken in the HTML Report. The strikethrough'd text extended beyond the intended region.
HTML Report Before:
<img width="693" alt="Screen Shot 2021-12-27 at 4 43 35 PM" src="https://user-images.githubusercontent.com/11915034/147518750-a60f9002-6eed-48a1-a412-20fabd076fa6.png">
HTML Report After:
<img width="206" alt="Screen Shot 2021-12-27 at 4 48 37 PM" src="https://user-images.githubusercontent.com/11915034/147518762-19a4c8f9-ccc3-4a3c-a962-5a42edc6fc5d.png">
This now matches what's expected and shown in the terminal (which has always been correct):
<img width="1384" alt="Screen Shot 2021-12-27 at 4 36 29 PM" src="https://user-images.githubusercontent.com/11915034/147518799-f538259e-5a45-4d6f-916c-a12ccb620c5b.png">
NB: This MR is a workaround, but not a root cause fix. It works, but I never fully got to the root cause so a bug upstream may be required. It's unclear whether it's (1) in [`colors`](https://www.npmjs.com/package/colors), (2) in [`ansi-to-html`](https://www.npmjs.com/package/ansi-to-html), or (3) Playwright's use of the two. Since the terminal output is correct, I suspect it is in `ansi-to-html`. For example:
```js
const colors = require("colors");
const Convert = require('ansi-to-html');
const convert = new Convert();
// original (strike incorrectly wraps everything in the HTML)
console.log(convert.toHtml(colors.strikethrough("crossed out") + ' ' + colors.red("red")))
// prints: <strike>crossed out <span style="color:#A00">red<span style="color:#FFF"></span></span></strike>
// workaround
console.log(convert.toHtml(colors.reset(colors.strikethrough("crossed out")) + ' ' + colors.red("red")))
// prints: <strike>crossed out</strike> <span style="color:#A00">red<span style="color:#FFF"></span></span>
```
Fixes#11116
This prepares for beforeAll/afterAll hooks to be handled in the same way.
Since we do not know in advance whether a hook will run, we must create
TestResults lazily.
