yujunjun/playwright
1
0
Fork 0
mirror of https://github.com/microsoft/playwright.git synced 2025-06-26 21:40:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: running tests improvements (#26956)

Browse Source
This commit is contained in:
Debbie O'Brien 2023-09-12 16:51:19 +02:00 committed by GitHub
parent 466088215e
commit 8bd26fe074
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 163 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

180
docs/src/running-tests-js.md
View File

@ -2,8 +2,9 @@
id: running-tests
title: "Running tests"
---
## Introduction
You can run a single test, a set of tests or all tests. Tests can be run on one browser or multiple browsers. By default tests are run in a headless manner meaning no browser window will be opened while running the tests and results will be seen in the terminal.
With Playwright you can run a single test, a set of tests or all tests. Tests can be run on one browser or multiple browsers by using the `--project` flag. Tests are run in parallel by default and are run in a headless manner meaning no browser window will be opened while running the tests and results will be seen in the terminal. However you can run tests in headed mode by using the `--headed` CLI argument or you can run your tests in UI mode, by using the `--ui` flag, and see a full trace of your tests complete with watch mode, time travel debugging and more.
**You will learn**
@ -11,103 +12,152 @@ You can run a single test, a set of tests or all tests. Tests can be run on one
- [How to debug tests](/running-tests.md#debugging-tests)
- [How to open the HTML test reporter](/running-tests.md#test-reports)
## VS Code extension
## Run tests in UI Mode
Tests can be run right from VS Code using the [VS Code extension](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright). Once installed you can simply click the green triangle next to the test you want to run or run all tests from the testing sidebar. Check out our [Getting Started with VS Code](./getting-started-vscode.md#running-tests) guide for more details.
Run your tests with [UI Mode](./test-ui-mode.md) for a better developer experience with time travel debugging, watch mode and more.
![Playwright VS Code extension](https://github.com/microsoft/playwright/assets/13063165/47726e70-683b-4bd5-94de-7d03dd45c30f)
```bash
npx playwright test --ui
```
## Command line
## Command Line
You can run your tests with the `playwright test` command. This will run your tests on all browsers as configured in the `playwright.config` file. Tests run in headless mode by default meaning no browser window will be opened while running the tests and results will be seen in the terminal.
- Running all tests
```bash
npx playwright test
```
```bash
npx playwright test
```
![tests running in command line](https://github.com/microsoft/playwright/assets/13063165/981c1b2b-dc7e-4b85-b241-272b44da6628)
- Running a single test file
### Run tests in UI mode
```bash
npx playwright test landing-page.spec.ts
```
We highly recommend running your tests with [UI Mode](./test-ui-mode.md) for a better developer experience where you can easily walk through each step of the test and visually see what was happening before, during, and after each step. UI mode also comes with many other features such as the locator picker, watch mode and more.
- Run a set of test files
```bash
npx playwright test --ui
```
```bash
npx playwright test tests/todo-page/ tests/landing-page/
```
![UI Mode](https://github.com/microsoft/playwright/assets/13063165/c5b501cc-4f5d-485a-87cc-66044c651786)
- Run files that have `landing` or `login` in the file name
Check out or [detailed guide on UI Mode](./test-ui-mode.md) to learn more about it's features.
```bash
npx playwright test landing login
```
### Run tests in headed mode
- Run the test with the title
To run your tests in headed mode use the `--headed` flag. This will give you the ability to visually see, how Playwright interacts with the website.
```bash
npx playwright test -g "add a todo item"
```
```bash
npx playwright test --headed
```
- Running tests in headed mode
### Run tests on different browsers
```bash
npx playwright test landing-page.spec.ts --headed
```
To specify which browser you would like to run your tests on use the `--project` flag followed by the name of the browser.
- Running tests on a specific project
```bash
npx playwright test --project webkit
```
```bash
npx playwright test landing-page.ts --project=chromium
```
To specify multiple browsers to run your tests on use the `--project` flag multiple times followed by the name of each browser.
## Debugging Tests
```bash
npx playwright test --project webkit --project firefox
```
Since Playwright runs in Node.js, you can debug it with your debugger of choice e.g. using `console.log` or inside your IDE or directly in VS Code with the [VS Code Extension](./getting-started-vscode.md). Playwright comes with the [Playwright Inspector](./debug.md#playwright-inspector) which allows you to step through Playwright API calls, see their debug logs and explore [locators](./locators.md).
### Run specific tests
To run a single test file pass in the name of the test file that you want to run.
```bash
npx playwright test landing-page.spec.ts
```
To run a set of test files from different directories pass in the names of the directories that you want to run the tests in.
```bash
npx playwright test tests/todo-page/ tests/landing-page/
```
To run files that have `landing` or `login` in the file name simply pass in these keywords to the CLI.
```bash
npx playwright test landing login
```
To run a test with a specific title use the `-g` flag followed by the title of the test.
```bash
npx playwright test -g "add a todo item"
```
## Debugging tests
Since Playwright runs in Node.js, you can debug it with your debugger of choice e.g. using `console.log` or inside your IDE or directly in VS Code with the [VS Code Extension](./getting-started-vscode.md). Playwright comes with [UI Mode](./test-ui-mode.md), where you can easily walk through each step of the test, see logs, errors, network requests, inspect the DOM snapshot and more. You can also use the [Playwright Inspector](./debug.md#playwright-inspector) which allows you to step through Playwright API calls, see their debug logs and explore [locators](./locators.md).
### Debug tests in UI mode
We highly recommend debugging your tests with [UI Mode](./test-ui-mode.md) for a better developer experience where you can easily walk through each step of the test and visually see what was happening before, during, and after each step. UI mode also comes with many other features such as the locator picker, watch mode and more.
```bash
npx playwright test --ui
```
![showing errors in ui mode](https://github.com/microsoft/playwright/assets/13063165/ffca2fd1-5349-41fb-ade9-ace143bb2c58)
While debugging you can use the Pick Locator button to select an element on the page and see the locator that Playwright would use to find that element. You an also edit the locator in the locator playground and see it highlighting live on the Browser window. Use the Copy Locator button to copy the locator to your clipboard and then paste it into you test.
![pick locator in ui mode](https://github.com/microsoft/playwright/assets/13063165/9e7eeb84-bd26-4010-8614-75e24b56c716)
Check out or [detailed guide on UI Mode](./test-ui-mode.md) to learn more about it's features.
### Debug tests with the Playwright Inspector
To debug all tests run the Playwright test command followed by the `--debug` flag.
```bash
npx playwright test --debug
```
![Debugging Tests with the Playwright inspector](https://github.com/microsoft/playwright/assets/13063165/6b3b3caa-d258-4cb8-aa05-cd407f501626)
This command will open up a Browser window as well as the Playwright Inspector. You can use the step over button at the top of the inspector to step through your test. Or press the play button to run your test from start to finish. Once the test has finished the browser window will close.
To debug one test file run the Playwright test command with the name of the test file that you want to debug followed by the `--debug` flag.
```bash
npx playwright test example.spec.ts --debug
```
To debug a specific test from the line number where the `test(..` is defined add a colon followed by the line number at the end of the test file name, followed by the `--debug` flag.
```bash
npx playwright test example.spec.ts:10 --debug
```
While debugging you can use the Pick Locator button to select an element on the page and see the locator that Playwright would use to find that element. You an also edit the locator and see it highlighting live on the Browser window. Use the Copy Locator button to copy the locator to your clipboard and then paste it into you test.
![Locator picker in the Playwright Inspector](https://github.com/microsoft/playwright/assets/13063165/013d5edb-583e-423a-bb53-9f57bab7f3e1)
- Debugging all tests:
```bash
npx playwright test --debug
```
- Debugging one test file:
```bash
npx playwright test example.spec.ts --debug
```
- Debugging a test from the line number where the `test(..` is defined:
```bash
npx playwright test example.spec.ts:10 --debug
```
<img width="1340" alt="Debugging Tests with the Playwright inspector" src="https://user-images.githubusercontent.com/13063165/212936618-84b87acc-bc2e-46ed-994b-32b2ef742e60.png" />
Check out our [debugging guide](./debug.md) to learn more about debugging with the [VS Code debugger](./debug.md#vs-code-debugger), UI Mode and the [Playwright Inspector](./debug.md#playwright-inspector) as well as debugging with [Browser Developer tools](./debug.md#browser-developer-tools).
Check out our [debugging guide](./debug.md) to learn more about the [Playwright Inspector](./debug.md#playwright-inspector) as well as debugging with [Browser Developer tools](./debug.md#browser-developer-tools).
## Test reports
## Test Reports
The [HTML Reporter](././test-reporters.md#html-reporter) shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed.
The [HTML Reporter](./test-reporters.md#html-reporter) shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed, otherwise you can open it with the following command.
```bash
npx playwright show-report
```
<img width="1424" alt="HTML Report > Test Reports view" src="https://user-images.githubusercontent.com/13063165/221930419-49543647-9130-4429-a857-6851c2005e48.png" />
![HTML Report](https://github.com/microsoft/playwright/assets/13063165/c5f60e56-fb75-4a2d-a4b6-054b8c5d69c1)
You can click on each test and explore the tests errors as well as each step of the test.
You can filter and search for tests as well as click on each test to see the tests errors and explore each step of the test.
<img width="1440" alt="HTML Reporter > Test Reports detailed view" src="https://user-images.githubusercontent.com/13063165/221930640-c1ccda28-7906-44c7-a198-acd9acb40bbe.png" />
![HTML Reporter detail view](https://github.com/microsoft/playwright/assets/13063165/f36ada14-4701-4534-a3be-ed22c2b16cf5)
## What's Next
## What's next
- [Generate tests with Codegen](./codegen-intro.md)
- [See a trace of your tests](./trace-viewer-intro.md)
- [Explore all the features of UI Mode](./test-ui-mode.md)
- [Run your tests on CI with GitHub Actions](./ci-intro.md)

56
docs/src/running-tests-python.md
View File

@ -2,6 +2,7 @@
id: running-tests
title: "Running tests"
---
## Introduction
You can run a single test, a set of tests or all tests. Tests can be run on one browser or multiple browsers by using the `--browser` flag. By default tests are run in a headless manner meaning no browser window will be opened while running the tests and results will be seen in the terminal. If you prefer you can run your tests in headed mode by using the `--headed` CLI argument.
@ -18,14 +19,14 @@ To run your tests use the `pytest` command. This will run your tests on the Chro
pytest
```
### Running tests headed
### Run tests in headed mode
To run your tests in headed mode use the `--headed` flag. This will open up a browser window while running your tests.
To run your tests in headed mode use the `--headed` flag. This will open up a browser window while running your tests and once finished the browser window will close.
```bash
pytest --headed
```
### Running tests on different browsers
### Run tests on different browsers
To specify which browser you would like to run your tests on use the `--browser` flag followed by the name of the browser.
@ -40,7 +41,7 @@ To specify multiple browsers to run your tests on use the `--browser` flag multi
pytest --browser webkit --browser firefox
```
### Running specific tests
### Run specific tests
To run a single test file pass in the name of the test file that you want to run.
@ -60,7 +61,7 @@ To run a specific test pass in the function name of the test you want to run.
pytest -k test_add_a_todo_item
```
### Run tests in Parallel
### Run tests in parallel
To run your tests in parallel use the `--numprocesses` flag followed by the number of processes you would like to run your tests on. We recommend half of logical CPU cores.
@ -72,10 +73,11 @@ To run your tests in parallel use the `--numprocesses` flag followed by the numb
For more information see [Playwright Pytest usage](./test-runners.md) or the Pytest documentation for [general CLI usage](https://docs.pytest.org/en/stable/usage.html).
## Debugging Tests
## Debugging tests
Since Playwright runs in Python, you can debug it with your debugger of choice with e.g. the [Python extension](https://code.visualstudio.com/docs/python/python-tutorial) in Visual Studio Code. Playwright comes with the Playwright Inspector which allows you to step through Playwright API calls, see their debug logs and explore [locators](./locators.md).
To debug all tests run the following command.
```bash tab=bash-bash lang=python
PWDEBUG=1 pytest -s
@ -90,12 +92,50 @@ pytest -s
$env:PWDEBUG=1
pytest -s
```
<img width="712" alt="Playwright Inspector" src="https://user-images.githubusercontent.com/883973/108614092-8c478a80-73ac-11eb-9597-67dfce110e00.png"></img>
To debug one test file run the command followed by the name of the test file that you want to debug.
```bash tab=bash-bash lang=python
PWDEBUG=1 pytest -s test_example.py
```
```batch tab=bash-batch lang=python
set PWDEBUG=1
pytest -s test_example.py
```
```powershell tab=bash-powershell lang=python
$env:PWDEBUG=1
pytest -s test_example.py
```
To debug a specific test add `-k` followed by the name of the test that you want to debug.
```bash tab=bash-bash lang=python
PWDEBUG=1 pytest -s -k test_get_started_link
```
```batch tab=bash-batch lang=python
set PWDEBUG=1
pytest -s -k test_get_started_link
```
```powershell tab=bash-powershell lang=python
$env:PWDEBUG=1
pytest -s -k test_get_started_link
```
This command will open up a Browser window as well as the Playwright Inspector. You can use the step over button at the top of the inspector to step through your test. Or press the play button to run your test from start to finish. Once the test has finished the browser window will close.
While debugging you can use the Pick Locator button to select an element on the page and see the locator that Playwright would use to find that element. You an also edit the locator and see it highlighting live on the Browser window. Use the Copy Locator button to copy the locator to your clipboard and then paste it into you test.
![Playwright Inspector](https://github.com/microsoft/playwright/assets/13063165/c94c89c8-f945-460c-a653-7809c6ca50ee)
Check out our [debugging guide](./debug.md) to learn more about the [Playwright Inspector](./debug.md#playwright-inspector) as well as debugging with [Browser Developer tools](./debug.md#browser-developer-tools).
## What's Next
## What's next
- [Generate tests with Codegen](./codegen.md)
- [See a trace of your tests](./trace-viewer-intro.md)
- [Run your tests on CI with GitHub Actions](./ci-intro.md)