88 Commits

Author SHA1 Message Date
Yury Semikhatsky
51b8f609fb
Revert "devops: zip blob report artifact before uploading (#23667)" (#23687)
This reverts commit a1cdae6bffcea5fd16c5bc56633e0949bfcc6315.

The problem with this approach is that each job overwrites the zip
artifact whereas previously it was merging all reports in the same
directory. We are going to zip .jsonl files instead.
2023-06-13 12:36:04 -07:00
Yury Semikhatsky
a1cdae6bff
devops: zip blob report artifact before uploading (#23667)
Compressed size for `tests 1` blob report is 19Mb whil uncompressed one
is 211Mb. Also according to [GitHub
policy](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts
) it is uncompressed size that is used for billing:

"Artifacts are uploaded during a workflow run, and you can view an
artifact's name and size in the UI. When an artifact is downloaded using
the GitHub UI, all files that were individually uploaded as part of the
artifact get zipped together into a single file. This means that billing
is calculated based on the size of the uploaded artifact and not the
size of the zip file."
2023-06-12 14:03:22 -07:00
Max Schmitt
b21c81d344
devops: add bot for Chromium --headless=new (#23440)
https://github.com/microsoft/playwright/issues/23389
2023-06-02 20:40:12 +02:00
Yury Semikhatsky
2719f408b2
fix(devops): pass pr number in a file (#23456)
For some reason pull_requests field on workflow_run is empty for pull
requests created from branches in forked repositories, see
https://github.com/orgs/community/discussions/25220. As a workaround we
store triggering pull request number in a file.
2023-06-01 17:12:58 -07:00
Yury Semikhatsky
6cbcf18c48
fix(devops): path to blob report (#23444) 2023-06-01 10:17:34 -07:00
Yury Semikhatsky
972ad89af2
feat(devops): publish merged reports to pull requests (#23419)
* Moved report merging and publishing logic into create_test_report.yml
shared between all workflows
* Merged reports are now published for try jobs on pull requests too. In
order to achieve that the logic had to be extracted into a separate
workflow triggered by
[workflow_run](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_run),
this way it can access secrets even if the original workflow was not
able to.
* The blob report data flow is different depending on whether the
workflow is triggered by a pull request or a push:
- For `pull_request` the workflow doesn't have access to the secrets it
uploads the blob report to the GitHub
artifact storage. Later on the merge workflow uploads that blob report
to Azure blob storage.
- Workflows triggered by `push` event can read secrets. They upload blob
report directly to Azure blob storage
and the merge workflow downloads the report from there rather than from
GitHub artifacts.
2023-06-01 08:40:43 -07:00
Yury Semikhatsky
ce9892c694
fix(devops): azure conection string var name (#23416) 2023-05-31 16:54:44 -07:00
Max Schmitt
92d650c317
devops: add missing workflow permission (#23413) 2023-06-01 01:19:23 +02:00
Yury Semikhatsky
90aa378e8f
chore: extract batch download action (#23410) 2023-05-31 16:07:29 -07:00
Max Schmitt
40581ed906
devops: post merged test results comment as GitHub Status Check (#23409) 2023-06-01 00:29:57 +02:00
Yury Semikhatsky
38c89df330
chore: replace "blob download-batch" with a bash script (#23381)
`az storage blob download-batch` has been timing out over the last few
days, see upstream issue
https://github.com/Azure/azure-cli/issues/26567. Replacing it with a
simple bash script that discovers blobs with a given prefix and then
downloads one-by-one.
2023-05-30 16:46:37 -07:00
Max Schmitt
1f7223eb21
chore: test on Node.js 20 (#22651)
Fixes https://github.com/microsoft/playwright/issues/22582
2023-05-30 18:16:34 +02:00
Yury Semikhatsky
1e64fa4f7c
chore: no empty dash suffix in blob report (#23285) 2023-05-25 13:18:57 -07:00
Yury Semikhatsky
0a65a0b8e8
devops: post merged report URL to the PR (#23266)
Posting a comment to the PR using GITHUB_TOKEN requires elevated
permissions, see
https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
2023-05-24 14:43:36 -07:00
Yury Semikhatsky
10b7cb3979
chore: blob report project suffix (#23212) 2023-05-22 17:54:37 -07:00
Yury Semikhatsky
c8a42af11f
devops: merged report for tracing tests (#23167) 2023-05-19 13:31:37 -07:00
Yury Semikhatsky
5004f908d9
devops: do not upload test-results artifacts (#23168) 2023-05-19 13:06:42 -07:00
Yury Semikhatsky
27403dcbb4
devops: remove explicit npm@8 installation (#23096)
Node 16+ comes with npm 8+ by default.
2023-05-17 14:32:38 -07:00
Max Schmitt
89aa02228f
devops: add macOS 13 coverage (#23065)
Coverage https://github.com/microsoft/playwright/issues/22226
2023-05-17 01:13:09 +02:00
Max Schmitt
44b63b042e
devops: do not download electron on non-electron workflows (#22232) 2023-04-06 19:49:29 +02:00
Max Schmitt
6682fb6075
devops: bump Node.js version 14 -> 16 (#20800)
Fixes https://github.com/microsoft/playwright/issues/20788
2023-02-10 00:53:32 +01:00
Dmitry Gozman
edfff8cd8c
test: run more tests in 'service' mode (#20734) 2023-02-07 18:25:33 -08:00
Andrey Lushnikov
0a752f3fd6
chore(docker): remove experimental docker integration (#20676)
This removes everything related to docker integration experiments that
we conducted over the last 6 months.

I'll send a follow-up with an alternative suggestion that was demo'ed on
a team meeting in the end of December.
2023-02-06 10:50:45 -08:00
Max Schmitt
3acf3f5cae
devops: fix service tests 2
Signed-off-by: Max Schmitt <max@schmitt.mx>
2022-12-21 21:10:44 +01:00
Max Schmitt
2c801f9309
test: fix service tests (#19592) 2022-12-20 15:37:30 +01:00
Max Schmitt
6a5552ba24
chore: drop Ubuntu 18.04 support (#18924) 2022-11-21 11:23:45 -10:00
Andrey Lushnikov
db826c9c8c
chore: drive-by fixes for the snapshotPathTemplate (#18715)
- Use `snapshotPathTemplate` for docker screenshots in html-reporter
- Mark the snapshot path template test as slow since it re-spawns
  worker for each project.
- Fix docker smoke tests
2022-11-10 15:50:13 -08:00
Max Schmitt
d1d24bb490
Revert "devops: use debug Node.js build on CI for browser downloads (#17504)" (#17711)
This reverts commit 56df171d17a31b86b68111a4f3c90a70403a1638.
2022-09-29 21:08:28 +02:00
Max Schmitt
56df171d17
devops: use debug Node.js build on CI for browser downloads (#17504)
https://github.com/microsoft/playwright/issues/17394
2022-09-28 10:52:30 +02:00
Andrey Lushnikov
a12112c24d
devops(docker): add docker integration smoke tests (#17267) 2022-09-14 15:05:18 -07:00
Max Schmitt
e8b9d4e758
devops: bump github actions (#16952) 2022-08-31 17:23:42 +02:00
Andrey Lushnikov
f3269c6891
tests: do not test on MacOS 10.15 any more (#16465)
Playwright v1.25 was the last release to support macOS 10.15, so we
no longer need to test on macOS 10.15.
2022-08-11 09:50:44 -07:00
Ross Wollman
a3836de182
chore: relax npm version constraint in installation tests (#16427)
Resolves #16281.
2022-08-10 15:33:47 -07:00
Ross Wollman
7c4099a011
tests(installation): pin npm version as workaround (#16323)
Relates https://github.com/microsoft/playwright/issues/16281.
2022-08-05 15:24:47 -07:00
Max Schmitt
93a96347c2
devops: use bash always for upload_flakiness_dashboard (#16298) 2022-08-05 10:47:44 +02:00
Max Schmitt
57637d4a4a
devops: upload edge stable,beta,dev to flakiness dashboard (#16297) 2022-08-05 09:51:45 +02:00
Yury Semikhatsky
37f9500efa
devops: add macos-12 bots, rearrange others (#16218)
* Added macos-12 bots to the secondary workflow
* Moved ubuntu 20.04 from primary to secondary workflow
* For the bots where we don't care about macos version (Chrome Stable, Edge Dev etc.) switched to macos-latest

References #16180
2022-08-03 12:34:41 -07:00
Max Schmitt
deaef17e70
devops: run tests on ubuntu 22 (#14741) 2022-06-09 14:30:55 +02:00
Dmitry Gozman
a5506a80ee
test: normalize node versions for installation tests (#14610)
We should cover node 14 on all platforms (similarly to
other types of tests), and node 16/18 on linux.
2022-06-07 18:11:58 -07:00
Dmitry Gozman
c5face5ecc
test: add more chromium-tip-of-tree bots (#14120) 2022-05-12 15:44:13 +01:00
Andrey Lushnikov
3efd0e15e5
devops: upload all headed tests to flakiness (#14056) 2022-05-10 06:49:28 -07:00
Pavel Feldman
1658172b2c
chore: run headed on all OSs (#14042) 2022-05-09 12:55:59 -07:00
Andrey Lushnikov
bc9ef1c76e
devops: follow-up with installation tests on flakienss dashbaord (#14015)
- upload more installation tests to the dashboard
- use json reporter on CI
2022-05-07 09:17:07 -07:00
Pavel Feldman
f4933e6d8b
chore(electron): add Win+Mac bots (#13943) 2022-05-04 12:29:22 -07:00
Ross Wollman
c4af7f2fc8
fix: unbreak installation tests, include in primary CI runs (#13903) 2022-05-03 12:48:35 -07:00
Ross Wollman
3e84ab4701
test: run installation tests on all major platforms (#13742) 2022-04-26 11:09:49 -07:00
Max Schmitt
f7af79137b
devops: unpin NPM on CI and use again version 8 (#13765) 2022-04-26 17:52:10 +02:00
Max Schmitt
6931f6dbeb
devops: bump Node.js versions on CI by 2 (#13754) 2022-04-26 12:20:26 +02:00
Ross Wollman
e69e836c40
chore: port installation tests to @playwright/test (#13644) 2022-04-25 09:30:14 -07:00
Andrey Lushnikov
317b649f78
feat: support tip-of-tree chromium channel (#13701) 2022-04-22 09:43:57 -07:00