2021-01-01 15:17:27 -08:00
---
id: ci
title: "Continuous Integration"
---
2020-12-30 18:04:51 -08:00
Playwright tests can be executed in CI environments. We have created sample
configurations for common CI providers.
## Introduction
3 steps to get your tests running on CI:
2021-01-03 08:47:29 -08:00
1. **Ensure CI agent can run browsers** : Use [our Docker image ](./docker.md )
2023-03-31 10:48:05 +02:00
in Linux agents or install your dependencies using the [CLI ](./browsers#install-system-dependencies ).
2021-01-21 16:41:42 -08:00
1. **Install Playwright** :
2021-06-02 09:23:06 -07:00
```bash js
2021-07-09 16:13:33 -07:00
# Install NPM packages
2021-05-11 20:47:48 +02:00
npm ci
2021-07-09 16:13:33 -07:00
2022-03-24 15:08:32 -06:00
# Install Playwright browsers and dependencies
npx playwright install --with-deps
2021-01-21 16:41:42 -08:00
```
2021-06-02 09:23:06 -07:00
```bash python
2021-05-11 20:47:48 +02:00
pip install playwright
2022-03-24 15:08:32 -06:00
playwright install --with-deps
```
```bash java
2023-01-05 19:55:07 +01:00
mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps"
2022-03-24 15:08:32 -06:00
```
```bash csharp
2023-05-11 17:52:51 +02:00
dotnet build
2022-08-14 20:01:00 +02:00
pwsh bin/Debug/netX/playwright.ps1 install --with-deps
2021-01-21 16:41:42 -08:00
```
1. **Run your tests** :
2021-06-02 09:23:06 -07:00
```bash js
2022-09-12 17:24:23 +02:00
npx playwright test
2021-01-21 16:41:42 -08:00
```
2021-06-02 09:23:06 -07:00
```bash python
2021-05-11 20:47:48 +02:00
pytest
2021-01-21 16:41:42 -08:00
```
2022-07-11 23:33:15 +02:00
```bash java
mvn test
```
```bash csharp
dotnet test
```
2020-12-30 18:04:51 -08:00
2023-05-17 17:38:03 +02:00
## Workers
* langs: js
We recommend setting [workers ](./api/class-testconfig.md#test-config-workers ) to "1" in CI environments to prioritize stability and reproducibility. Running tests sequentially ensures each test gets the full system resources, avoiding potential conflicts. However, if you have a powerful self-hosted CI system, you may enable [parallel ](./test-parallel.md ) tests. For wider parallelization, consider [sharding ](./test-parallel.md#shard-tests-between-multiple-machines ) - distributing tests across multiple CI jobs.
```js title="playwright.config.ts"
import { defineConfig, devices } from '@playwright/test ';
export default defineConfig({
// Opt out of parallel tests on CI.
workers: process.env.CI ? 1 : undefined,
});
```
2020-12-30 18:04:51 -08:00
## CI configurations
2023-03-31 10:48:05 +02:00
The [Command line tools ](./browsers#install-system-dependencies ) can be used to install all operating system dependencies on GitHub Actions.
2020-12-30 18:04:51 -08:00
2021-06-23 23:15:00 +02:00
### GitHub Actions
2020-12-30 18:04:51 -08:00
2023-05-11 17:52:51 +02:00
#### On push/pull_request
2021-01-21 16:41:42 -08:00
```yml js
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npx playwright test
- uses: actions/upload-artifact@v3
if: always()
with:
name: playwright-report
path: playwright-report/
retention-days: 30
2020-12-30 18:04:51 -08:00
```
2021-01-21 16:41:42 -08:00
```yml python
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
2023-05-31 14:29:54 +02:00
pip install -r requirements.txt
2023-05-11 17:52:51 +02:00
- name: Ensure browsers are installed
run: python -m playwright install --with-deps
- name: Run your tests
run: pytest
2021-01-21 16:41:42 -08:00
```
2022-07-11 23:33:15 +02:00
```yml java
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '17'
- name: Build & Install
run: mvn -B install -D skipTests --no-transfer-progress
- name: Install Playwright
run: mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps"
- name: Run tests
run: mvn test
2022-07-11 23:33:15 +02:00
```
```yml csharp
2021-09-17 11:22:14 +02:00
name: Playwright Tests
on:
2023-05-11 17:52:51 +02:00
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
2021-09-17 11:22:14 +02:00
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
2022-07-11 23:33:15 +02:00
- uses: actions/checkout@v3
2023-05-11 17:52:51 +02:00
- name: Setup dotnet
uses: actions/setup-dotnet@v3
2021-09-17 11:22:14 +02:00
with:
2023-05-11 17:52:51 +02:00
dotnet-version: 6.0.x
- run: dotnet build
- name: Ensure browsers are installed
2023-05-31 14:29:54 +02:00
run: pwsh bin/Debug/net6.0/playwright.ps1 install --with-deps
2023-05-11 17:52:51 +02:00
- name: Run your tests
run: dotnet test
2021-09-17 11:22:14 +02:00
```
2023-05-11 17:52:51 +02:00
#### On push/pull_request (sharded)
* langs: js
2020-12-30 18:04:51 -08:00
2023-05-11 17:52:51 +02:00
GitHub Actions supports [sharding tests between multiple jobs ](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs ) using the [`jobs.<job_id>.strategy.matrix` ](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix ) option. The `matrix` option will run a separate job for every possible combination of the provided options. In the example below, we have 2 `project` values, 10 `shardIndex` values and 1 `shardTotal` value, resulting in a total of 20 jobs to be run. So it will split up the tests between 20 jobs, each running a different browser and a different subset of tests, see [here ](./test-parallel.md#shard-tests-between-multiple-machines ) for more details.
2020-12-30 18:04:51 -08:00
2023-05-11 17:52:51 +02:00
```yml js
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
playwright:
name: 'Playwright Tests - ${{ matrix.project }} - Shard ${{ matrix.shardIndex }} of ${{ matrix.shardTotal }}'
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
project: [chromium, webkit]
shardIndex: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
shardTotal: [10]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Install browsers
run: npx playwright install --with-deps
- name: Run your tests
run: npx playwright test --project=${{ matrix.project }} --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
```
> Note: The `${{ <expression> }}` is the [expression](https://docs.github.com/en/actions/learn-github-actions/expressions) syntax that allows accessing the current [context](https://docs.github.com/en/actions/learn-github-actions/contexts). In this example, we are using the [`matrix`](https://docs.github.com/en/actions/learn-github-actions/contexts#matrix-context) context to set the job variants.
2020-12-30 18:04:51 -08:00
2023-05-11 17:52:51 +02:00
#### Via Containers
2022-07-11 15:34:24 -05:00
2023-05-11 17:52:51 +02:00
GitHub Actions support [running jobs in a container ](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container ) by using the [`jobs.<job_id>.container` ](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontainer ) option. This is useful to not pollute the host environment with dependencies and to have a consistent environment for e.g. screenshots/visual regression testing across different operating systems.
2022-07-11 15:34:24 -05:00
```yml js
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
2023-01-11 15:14:27 +05:30
jobs:
2022-07-11 15:34:24 -05:00
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2022-07-11 15:34:24 -05:00
steps:
2022-07-11 23:33:15 +02:00
- uses: actions/checkout@v3
2022-11-16 22:42:04 +01:00
- uses: actions/setup-node@v3
2022-07-11 15:34:24 -05:00
with:
2023-05-11 17:52:51 +02:00
node-version: 18
2022-07-11 15:34:24 -05:00
- name: Install dependencies
run: npm ci
- name: Run your tests
run: npx playwright test
```
2022-07-11 23:33:15 +02:00
```yml python
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
2023-01-11 15:14:27 +05:30
jobs:
2022-07-11 23:33:15 +02:00
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/python:v%%VERSION%%-jammy
2022-07-11 23:33:15 +02:00
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
2022-11-16 22:42:04 +01:00
python-version: '3.11'
2022-07-11 23:33:15 +02:00
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r local-requirements.txt
pip install -e .
- name: Run your tests
run: pytest
```
```yml java
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
2023-01-11 15:14:27 +05:30
jobs:
2022-07-11 23:33:15 +02:00
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/java:v%%VERSION%%-jammy
2022-07-11 23:33:15 +02:00
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '17'
- name: Build & Install
run: mvn -B install -D skipTests --no-transfer-progress
- name: Run tests
run: mvn test
```
```yml csharp
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
2023-01-11 15:14:27 +05:30
jobs:
2022-07-11 23:33:15 +02:00
playwright:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-jammy
2022-07-11 23:33:15 +02:00
steps:
- uses: actions/checkout@v3
- name: Setup dotnet
2022-11-16 22:42:04 +01:00
uses: actions/setup-dotnet@v3
2022-07-11 23:33:15 +02:00
with:
dotnet-version: 6.0.x
- run: dotnet build
- name: Run your tests
run: dotnet test
```
2023-05-11 17:52:51 +02:00
#### Via Containers (sharded)
2022-07-11 15:34:24 -05:00
* langs: js
2023-05-11 17:52:51 +02:00
GitHub Actions supports [sharding tests between multiple jobs ](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs ) using the [`jobs.<job_id>.strategy.matrix` ](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix ) option. The `matrix` option will run a separate job for every possible combination of the provided options. In the example below, we have 2 `project` values, 10 `shardIndex` values and 1 `shardTotal` value, resulting in a total of 20 jobs to be run. So it will split up the tests between 20 jobs, each running a different browser and a different subset of tests, see [here ](./test-parallel.md#shard-tests-between-multiple-machines ) for more details.
2022-07-11 15:34:24 -05:00
```yml js
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
2023-01-11 15:14:27 +05:30
jobs:
2022-07-11 15:34:24 -05:00
playwright:
name: 'Playwright Tests - ${{ matrix.project }} - Shard ${{ matrix.shardIndex }} of ${{ matrix.shardTotal }}'
runs-on: ubuntu-latest
container:
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2022-07-11 15:34:24 -05:00
strategy:
fail-fast: false
matrix:
2022-08-31 10:53:54 +02:00
project: [chromium, webkit]
2022-07-11 15:34:24 -05:00
shardIndex: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
shardTotal: [10]
steps:
2022-07-11 23:33:15 +02:00
- uses: actions/checkout@v3
2022-11-16 22:42:04 +01:00
- uses: actions/setup-node@v3
2022-07-11 15:34:24 -05:00
with:
2023-05-11 17:52:51 +02:00
node-version: 18
2022-07-11 15:34:24 -05:00
- name: Install dependencies
run: npm ci
- name: Run your tests
run: npx playwright test --project=${{ matrix.project }} --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
```
2023-05-11 17:52:51 +02:00
#### On deployment
This will start the tests after a [GitHub Deployment ](https://developer.github.com/v3/repos/deployments/ ) went into the `success` state.
Services like Vercel use this pattern so you can run your end-to-end tests on their deployed environment.
2023-05-31 14:29:54 +02:00
```yml js
2023-05-11 17:52:51 +02:00
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npx playwright test
env:
2023-05-31 14:29:54 +02:00
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
```yml python
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v3
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Ensure browsers are installed
run: python -m playwright install --with-deps
- name: Run tests
run: pytest
env:
# This might depend on your test-runner
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
```yml java
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '17'
- name: Build & Install
run: mvn -B install -D skipTests --no-transfer-progress
- name: Install Playwright
run: mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps"
- name: Run tests
run: mvn test
env:
# This might depend on your test-runner
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
```yml csharp
name: Playwright Tests
on:
deployment_status:
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v3
- name: Setup dotnet
uses: actions/setup-dotnet@v3
with:
dotnet-version: 6.0.x
- run: dotnet build
- name: Ensure browsers are installed
run: pwsh bin/Debug/net6.0/playwright.ps1 install --with-deps
- name: Run tests
run: dotnet test
env:
# This might depend on your test-runner
2023-05-11 17:52:51 +02:00
PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.target_url }}
```
### Docker
We have a [pre-built Docker image ](./docker.md ) which can either be used directly, or as a reference to update your existing Docker definitions.
Suggested configuration
1. Using `--ipc=host` is also recommended when using Chromium. Without it Chromium can run out of memory
and crash. Learn more about this option in [Docker docs ](https://docs.docker.com/engine/reference/run/#ipc-settings---ipc ).
1. Seeing other weird errors when launching Chromium? Try running your container
with `docker run --cap-add=SYS_ADMIN` when developing locally.
1. Using `--init` Docker flag or [dumb-init ](https://github.com/Yelp/dumb-init ) is recommended to avoid special
treatment for processes with PID=1. This is a common reason for zombie processes.
2022-07-11 15:34:24 -05:00
2020-12-30 18:04:51 -08:00
### Azure Pipelines
For Windows or macOS agents, no additional configuration required, just install Playwright and run your tests.
2022-03-24 15:08:32 -06:00
For Linux agents, you can use [our Docker container ](./docker.md ) with Azure
Pipelines support [running containerized
jobs](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops).
2023-03-31 10:48:05 +02:00
Alternatively, you can use [Command line tools ](./browsers#install-system-dependencies ) to install all necessary dependencies.
2020-12-30 18:04:51 -08:00
2022-09-08 08:22:42 +02:00
For running the Playwright tests use this pipeline task:
2023-05-31 14:29:54 +02:00
```yml js
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: NodeTool@0
inputs:
versionSpec: '18'
displayName: 'Install Node.js'
- script: npm ci
displayName: 'npm ci'
- script: npx playwright install --with-deps
displayName: 'Install Playwright browsers'
- script: npx playwright test
displayName: 'Run Playwright tests'
```
```yml python
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.11'
displayName: 'Use Python'
- script: |
python -m pip install --upgrade pip
pip install -r requirements.txt
displayName: 'Install dependencies'
- script: npx playwright install --with-deps
displayName: 'Install Playwright browsers'
- script: npx playwright test
displayName: 'Run Playwright tests'
```
```yml java
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: JavaToolInstaller@0
inputs:
versionSpec: '17'
jdkArchitectureOption: 'x64'
jdkSourceOption: AzureStorage
- script: mvn -B install -D skipTests --no-transfer-progress
displayName: 'Build and install'
- script: mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps"
displayName: 'Install Playwright browsers'
- script: mvn test
displayName: 'Run tests'
2022-09-08 08:22:42 +02:00
```
2023-05-31 14:29:54 +02:00
```yml csharp
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: UseDotNet@2
inputs:
packageType: sdk
version: '6.0.x'
displayName: 'Use .NET SDK'
- script: dotnet build --configuration Release
displayName: 'Build'
- script: pwsh bin/Debug/net6.0/playwright.ps1 install --with-deps
displayName: 'Install Playwright browsers'
- script: dotnet test --configuration Release
displayName: 'Run tests'
```
#### Uploading playwright-report folder with Azure Pipelines
* langs: js
2022-09-08 08:22:42 +02:00
This will make the pipeline run fail if any of the playwright tests fails.
2022-11-29 21:02:30 +00:00
If you also want to integrate the test results with Azure DevOps, use the task `PublishTestResults` task like so:
2023-05-31 14:29:54 +02:00
2022-09-08 08:22:42 +02:00
```yml
2023-05-31 14:29:54 +02:00
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: NodeTool@0
inputs:
versionSpec: '18'
displayName: 'Install Node.js'
- script: npm ci
displayName: 'npm ci'
- script: npx playwright install --with-deps
displayName: 'Install Playwright browsers'
- script: npx playwright test
displayName: 'Run Playwright tests'
- task: PublishTestResults@2
displayName: 'Publish test results'
inputs:
searchFolder: 'test-results'
testResultsFormat: 'JUnit'
testResultsFiles: 'e2e-junit-results.xml'
mergeTestResults: true
failTaskOnFailedTests: true
testRunTitle: 'My End-To-End Tests'
condition: succeededOrFailed()
- task: PublishPipelineArtifact@1
inputs:
targetPath: playwright-report
artifact: playwright-report
publishLocation: 'pipeline'
condition: succeededOrFailed()
2022-11-29 21:02:30 +00:00
2022-09-08 08:22:42 +02:00
```
Note: The JUnit reporter needs to be configured accordingly via
2023-03-16 21:16:18 +01:00
```js
2023-06-27 11:53:53 +02:00
import { defineConfig } from '@playwright/test ';
export default defineConfig({
reporter: ['junit', { outputFile: 'test-results/e2e-junit-results.xml' }],
});
2020-12-30 18:04:51 -08:00
```
2022-09-08 08:22:42 +02:00
in `playwright.config.ts` .
2020-12-30 18:04:51 -08:00
2023-05-31 14:29:54 +02:00
#### Azure Pipelines (sharded)
* langs: js
```yaml
trigger:
- main
pool:
vmImage: ubuntu-latest
strategy:
matrix:
chromium-1:
project: chromium
shardIndex: 1
shardTotal: 3
chromium-2:
project: chromium
shardIndex: 2
shardTotal: 3
chromium-3:
project: chromium
shardIndex: 3
shardTotal: 3
firefox-1:
project: firefox
shardIndex: 1
shardTotal: 3
firefox-2:
project: firefox
shardIndex: 2
shardTotal: 3
firefox-3:
project: firefox
shardIndex: 3
shardTotal: 3
webkit-1:
project: webkit
shardIndex: 1
shardTotal: 3
webkit-2:
project: webkit
shardIndex: 2
shardTotal: 3
webkit-3:
project: webkit
shardIndex: 3
shardTotal: 3
steps:
- task: NodeTool@0
inputs:
versionSpec: '18'
displayName: 'Install Node.js'
- script: npm ci
displayName: 'npm ci'
- script: npx playwright install --with-deps
displayName: 'Install Playwright browsers'
- script: npx playwright test --project=$(project) --shard=$(shardIndex)/$(shardTotal)
displayName: 'Run Playwright tests'
```
#### Azure Pipelines (containerized)
```yml js
trigger:
- main
pool:
vmImage: ubuntu-latest
2023-06-01 01:14:00 +02:00
container: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
steps:
- task: NodeTool@0
inputs:
versionSpec: '18'
displayName: 'Install Node.js'
- script: npm ci
displayName: 'npm ci'
- script: npx playwright test
displayName: 'Run Playwright tests'
```
```yml python
trigger:
- main
pool:
vmImage: ubuntu-latest
2023-06-01 01:14:00 +02:00
container: mcr.microsoft.com/playwright/python:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.11'
displayName: 'Use Python'
- script: |
python -m pip install --upgrade pip
pip install -r requirements.txt
displayName: 'Install dependencies'
- script: pytest
displayName: 'Run tests'
```
```yml java
trigger:
- main
pool:
vmImage: ubuntu-latest
2023-06-01 01:14:00 +02:00
container: mcr.microsoft.com/playwright/java:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
steps:
- task: JavaToolInstaller@0
inputs:
versionSpec: '17'
jdkArchitectureOption: 'x64'
jdkSourceOption: AzureStorage
- script: mvn -B install -D skipTests --no-transfer-progress
displayName: 'Build and install'
- script: mvn test
displayName: 'Run tests'
```
```yml csharp
trigger:
- main
pool:
vmImage: ubuntu-latest
2023-06-01 01:14:00 +02:00
container: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
steps:
- task: UseDotNet@2
inputs:
packageType: sdk
version: '6.0.x'
displayName: 'Use .NET SDK'
- script: dotnet build --configuration Release
displayName: 'Build'
- script: dotnet test --configuration Release
displayName: 'Run tests'
```
2020-12-30 18:04:51 -08:00
### CircleCI
2023-05-11 17:52:51 +02:00
Running Playwright on CircleCI is very similar to running on GitHub Actions. In order to specify the pre-built Playwright [Docker image ](./docker.md ), simply modify the agent definition with `docker:` in your config like so:
2020-12-30 18:04:51 -08:00
2023-05-31 14:29:54 +02:00
```yml js
executors:
pw-jammy-development:
docker:
2023-06-01 01:14:00 +02:00
- image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
```
```yml python
executors:
pw-jammy-development:
docker:
2023-06-01 01:14:00 +02:00
- image: mcr.microsoft.com/playwright/python:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
```
```yml java
executors:
pw-jammy-development:
docker:
2023-06-01 01:14:00 +02:00
- image: mcr.microsoft.com/playwright/java:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
```
```yml csharp
executors:
pw-jammy-development:
docker:
2023-06-01 01:14:00 +02:00
- image: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
```
2020-12-30 18:04:51 -08:00
2022-09-06 14:08:14 -04:00
Note: When using the docker agent definition, you are specifying the resource class of where playwright runs to the 'medium' tier [here ](https://circleci.com/docs/configuration-reference?#docker-execution-environment ). The default behavior of Playwright is to set the number of workers to the detected core count (2 in the case of the medium tier). Overriding the number of workers to greater than this number will cause unnecessary timeouts and failures.
2022-11-29 15:52:07 -05:00
#### Sharding in CircleCI
2023-06-21 14:04:15 +02:00
* langs: js
2022-09-06 14:08:14 -04:00
2022-11-29 15:52:07 -05:00
Sharding in CircleCI is indexed with 0 which means that you will need to override the default parallelism ENV VARS. The following example demonstrates how to run Playwright with a CircleCI Parallelism of 4 by adding 1 to the `CIRCLE_NODE_INDEX` to pass into the `--shard` cli arg.
2022-09-06 14:08:14 -04:00
```yml
playwright-job-name:
2023-04-12 19:40:23 +02:00
executor: pw-jammy-development
2022-09-06 14:08:14 -04:00
parallelism: 4
steps:
2023-05-08 11:58:38 +01:00
- run: SHARD="$((${CIRCLE_NODE_INDEX}+1))"; npx playwright test -- --shard=${SHARD}/${CIRCLE_NODE_TOTAL}
2022-09-06 14:08:14 -04:00
```
2020-12-30 18:04:51 -08:00
### Jenkins
2021-01-03 08:47:29 -08:00
Jenkins supports Docker agents for pipelines. Use the [Playwright Docker image ](./docker.md )
2020-12-30 18:04:51 -08:00
to run tests on Jenkins.
2023-05-31 14:29:54 +02:00
```groovy js
2020-12-30 18:04:51 -08:00
pipeline {
2023-06-01 01:14:00 +02:00
agent { docker { image 'mcr.microsoft.com/playwright:v%%VERSION%%-jammy' } }
2020-12-30 18:04:51 -08:00
stages {
stage('e2e-tests') {
steps {
2023-05-31 14:29:54 +02:00
sh 'ci'
2022-09-12 17:24:23 +02:00
sh 'npx playwright test'
2020-12-30 18:04:51 -08:00
}
}
}
}
```
2023-05-31 14:29:54 +02:00
```groovy python
pipeline {
2023-06-01 01:14:00 +02:00
agent { docker { image 'mcr.microsoft.com/playwright/python:v%%VERSION%%-jammy' } }
2023-05-31 14:29:54 +02:00
stages {
stage('e2e-tests') {
steps {
sh 'pip install -r requirements.txt'
sh 'pytest'
}
}
}
}
```
```groovy java
pipeline {
2023-06-01 01:14:00 +02:00
agent { docker { image 'mcr.microsoft.com/playwright/java:v%%VERSION%%-jammy' } }
2023-05-31 14:29:54 +02:00
stages {
stage('e2e-tests') {
steps {
sh 'mvn -B install -D skipTests --no-transfer-progress'
sh 'mvn test'
}
}
}
}
```
```groovy csharp
pipeline {
2023-06-01 01:14:00 +02:00
agent { docker { image 'mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-jammy' } }
2023-05-31 14:29:54 +02:00
stages {
stage('e2e-tests') {
steps {
sh 'dotnet build'
sh 'dotnet test'
}
}
}
}
```
2020-12-30 18:04:51 -08:00
### Bitbucket Pipelines
2021-01-03 08:47:29 -08:00
Bitbucket Pipelines can use public [Docker images as build environments ](https://confluence.atlassian.com/bitbucket/use-docker-images-as-build-environments-792298897.html ). To run Playwright tests on Bitbucket, use our public Docker image ([see Dockerfile ](./docker.md )).
2020-12-30 18:04:51 -08:00
2023-05-31 14:29:54 +02:00
```yml js
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2020-12-30 18:04:51 -08:00
```
2023-05-31 14:29:54 +02:00
```yml python
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/python:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
```
```yml java
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/java:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
```
```yml csharp
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
```
2020-12-30 18:04:51 -08:00
### GitLab CI
2021-01-03 08:47:29 -08:00
To run Playwright tests on GitLab, use our public Docker image ([see Dockerfile ](./docker.md )).
2020-12-30 18:04:51 -08:00
2023-05-31 14:29:54 +02:00
```yml js
2020-12-30 18:04:51 -08:00
stages:
- test
tests:
stage: test
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2020-12-30 18:04:51 -08:00
script:
2021-01-21 16:41:42 -08:00
...
2020-12-30 18:04:51 -08:00
```
2023-05-31 14:29:54 +02:00
```yml python
stages:
- test
tests:
stage: test
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/python:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
script:
...
```
```yml java
stages:
- test
tests:
stage: test
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/java:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
script:
...
```
```yml dotnet
stages:
- test
tests:
stage: test
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-jammy
2023-05-31 14:29:54 +02:00
script:
...
```
2022-08-31 10:53:54 +02:00
#### Sharding
* langs: js
GitLab CI supports [sharding tests between multiple jobs ](https://docs.gitlab.com/ee/ci/jobs/job_control.html#parallelize-large-jobs ) using the [parallel ](https://docs.gitlab.com/ee/ci/yaml/index.html#parallel ) keyword. The test job will be split into multiple smaller jobs that run in parallel. Parallel jobs are named sequentially from `job_name 1/N` to `job_name N/N` .
```yml
stages:
- test
tests:
stage: test
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2022-08-31 10:53:54 +02:00
parallel: 7
script:
- npm ci
- npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
```
GitLab CI also supports sharding tests between multiple jobs using the [parallel:matrix ](https://docs.gitlab.com/ee/ci/yaml/index.html#parallelmatrix ) option. The test job will run multiple times in parallel in a single pipeline, but with different variable values for each instance of the job. In the example below, we have 2 `PROJECT` values, 10 `SHARD_INDEX` values and 1 `SHARD_TOTAL` value, resulting in a total of 20 jobs to be run.
```yml
stages:
- test
tests:
stage: test
2023-06-01 01:14:00 +02:00
image: mcr.microsoft.com/playwright:v%%VERSION%%-jammy
2022-08-31 10:53:54 +02:00
parallel:
matrix:
- PROJECT: ['chromium', 'webkit']
SHARD_INDEX: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
SHARD_TOTAL: 10
script:
- npm ci
- npx playwright test --project=$PROJECT --shard=$SHARD_INDEX/$SHARD_TOTAL
```
2020-12-30 18:04:51 -08:00
## Caching browsers
2023-06-21 14:04:15 +02:00
Caching browser binaries is not recommended, since the amount of time it takes to restore the cache is comparable to the time it takes to download the binaries. Especially under Linux, [operating system dependencies ](./browsers.md#install-system-dependencies ) need to be installed, which are not cacheable.
2020-12-30 18:04:51 -08:00
2023-06-21 14:04:15 +02:00
If you still want to cache the browser binaries between CI runs, cache [these directories ](./browsers.md#managing-browser-binaries ) in your CI configuration, against a hash of the Playwright version.
2020-12-30 18:04:51 -08:00
## Debugging browser launches
2023-05-31 14:29:54 +02:00
Playwright supports the `DEBUG` environment variable to output debug logs during execution. Setting it to `pw:browser` is helpful while debugging `Error: Failed to launch browser` errors.
2020-12-30 18:04:51 -08:00
2021-06-02 09:23:06 -07:00
```bash js
2023-05-31 14:29:54 +02:00
DEBUG=pw:browser npx playwright test
2020-12-30 18:04:51 -08:00
```
2021-06-02 09:23:06 -07:00
```bash python
2023-05-31 14:29:54 +02:00
DEBUG=pw:browser pytest
2021-01-21 16:41:42 -08:00
```
2020-12-30 18:04:51 -08:00
2023-05-11 17:52:51 +02:00
```bash java
2023-05-31 14:29:54 +02:00
DEBUG=pw:browser mvn test
2023-05-11 17:52:51 +02:00
```
```bash csharp
2023-05-31 14:29:54 +02:00
DEBUG=pw:browser dotnet test
2023-05-11 17:52:51 +02:00
```
2021-04-01 20:13:50 +02:00
## Running headed
2020-12-30 18:04:51 -08:00
By default, Playwright launches browsers in headless mode. This can be changed by passing a flag when the browser is launched.
```js
// Works across chromium, firefox and webkit
const { chromium } = require('playwright');
const browser = await chromium.launch({ headless: false });
```
2021-03-01 09:18:44 -08:00
```java
// Works across chromium, firefox and webkit
import com.microsoft.playwright.*;
public class Example {
public static void main(String[] args) {
try (Playwright playwright = Playwright.create()) {
BrowserType chromium = playwright.chromium();
2021-03-05 13:50:34 -08:00
Browser browser = chromium.launch(new BrowserType.LaunchOptions().setHeadless(false));
2021-03-01 09:18:44 -08:00
}
}
}
```
2021-01-11 09:34:49 -08:00
```python async
2021-01-08 17:39:33 +01:00
import asyncio
2021-01-11 17:04:24 -08:00
from playwright.async_api import async_playwright
2021-01-08 17:39:33 +01:00
async def main():
async with async_playwright() as p:
# Works across chromium, firefox and webkit
browser = await p.chromium.launch(headless=False)
2021-01-15 09:12:47 -08:00
asyncio.run(main())
2021-01-08 17:39:33 +01:00
```
2021-01-11 09:34:49 -08:00
```python sync
2021-01-11 17:04:24 -08:00
from playwright.sync_api import sync_playwright
2021-01-08 17:39:33 +01:00
with sync_playwright() as p:
# Works across chromium, firefox and webkit
browser = p.chromium.launch(headless=False)
```
2021-05-20 04:49:48 +02:00
```csharp
using Microsoft.Playwright;
2022-04-19 21:23:26 +03:00
using var playwright = await Playwright.CreateAsync();
2022-07-13 11:50:18 +02:00
await playwright.Chromium.LaunchAsync(new()
2021-05-20 04:49:48 +02:00
{
2022-04-19 21:23:26 +03:00
Headless = false
});
2021-05-20 04:49:48 +02:00
```
2021-04-01 20:13:50 +02:00
On Linux agents, headed execution requires [Xvfb ](https://en.wikipedia.org/wiki/Xvfb ) to be installed. Our [Docker image ](./docker.md ) and GitHub Action have Xvfb pre-installed. To run browsers in headed mode with Xvfb, add `xvfb-run` before the Node.js command.
2020-12-30 18:04:51 -08:00
2021-06-02 09:23:06 -07:00
```bash js
2020-12-30 18:04:51 -08:00
xvfb-run node index.js
```
2021-06-02 09:23:06 -07:00
```bash python
2021-01-21 16:41:42 -08:00
xvfb-run python test.py
```
2023-05-11 17:52:51 +02:00
```bash java
xvfb-run mvn test
```
```bash csharp
xvfb-run dotnet test
```