Skip to content

Commit 626bcab

Browse files
nohwndBillWagner
andauthored
Document AzureDevOps plugin (#49949)
* Document AzureDevOps plugin * Fix linter * Remove license notice * Add to table and re-format * Update unit-testing-mstest-sdk.md * Update docs/core/testing/microsoft-testing-platform-extensions-test-reports.md * Apply suggestion from @nohwnd * Apply suggestions from code review * Update docs/core/testing/microsoft-testing-platform-extensions-test-reports.md --------- Co-authored-by: Bill Wagner <wiwagn@microsoft.com>
1 parent afdcd0c commit 626bcab

File tree

3 files changed

+44
-11
lines changed

3 files changed

+44
-11
lines changed
129 KB
Loading

docs/core/testing/microsoft-testing-platform-extensions-test-reports.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,33 @@ The available options as follows:
2424
| `--report-trx-filename` | The name of the generated TRX report. The default name matches the following format `<UserName>_<MachineName>_<yyyy-MM-dd HH:mm:ss>.trx`. |
2525

2626
The report is saved inside the default _TestResults_ folder that can be specified through the `--results-directory` command line argument.
27+
28+
## Azure DevOps reports
29+
30+
Azure DevOps report plugin enhances test running for developers that host their code on GitHub, but build on Azure DevOps build agents. It adds additional information to failures to show failure directly in GitHub PR.
31+
32+
![Error annotation in GitHub PR files view](./media/test-azdoreport-failure.png)
33+
34+
The extension is shipped in [Microsoft.Testing.Extensions.AzureDevOpsReport](https://nuget.org/packages/Microsoft.Testing.Extensions.AzureDevOpsReport) package.
35+
36+
The available options as follows:
37+
38+
| Option | Description |
39+
|--|--|
40+
| `--report-azdo` | Enable outputting errors / warnings in CI builds. |
41+
| `--report-azdo-severity` | Severity to use for the reported event. Options are: `error` (default) and `warning`. |
42+
43+
The extension automatically detects that it is running in continuous integration (CI) environment by checking the `TF_BUILD` environment variable.
44+
45+
### Determining the line to report
46+
47+
To highlight the correct line in code where failure occurred, AzureDevOps report plugin searches the error stacktrace for a file that exists in the current repository.
48+
49+
To determine this it:
50+
51+
- finds the repository root, this is done by searching the `.git` directory closest to the location from where the test application is started (as determined by `AppContext.BaseDirectory`).
52+
- finds the first line in stack trace that has file location and line (the library needs to have debug symbols).
53+
- excludes all files that end with `Assert.cs` to avoid showing details of your assertion implementations or wrappers.
54+
- excludes all files that don't exist on disk (typically those are lines from external libraries that ship debug symbols e.g. MSTest).
55+
56+
(These are internal details that serve to aid debugging the behavior and might change in the future.)

docs/core/testing/unit-testing-mstest-sdk.md

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,7 @@ You can set the profile using the property `TestingExtensionsProfile` with one o
104104
* [Hot Reload](./microsoft-testing-platform-extensions-hosting.md#hot-reload)
105105
* [Retry](./microsoft-testing-platform-extensions-policy.md#retry)
106106
* [Trx Report](./microsoft-testing-platform-extensions-test-reports.md#visual-studio-test-reports)
107+
* [AzureDevOpsReport](./microsoft-testing-platform-extensions-test-reports.md#azure-devops-reports)
107108

108109
Here's a full example, using the `None` profile:
109110

@@ -118,17 +119,19 @@ Here's a full example, using the `None` profile:
118119
</Project>
119120
```
120121

121-
| Extension/Profile | None | Default | AllMicrosoft |
122-
|-------------------------------------------------------------------------------------------|:----:|:------------------:|:--------------------------------------:|
123-
| [Code Coverage](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CodeCoverage) | | :heavy_check_mark: | :heavy_check_mark: |
124-
| [Crash Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CrashDump) | | | :heavy_check_mark: |
125-
| [Fakes](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Fakes) | | | :heavy_check_mark:|
126-
| [Hang Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HangDump) | | | :heavy_check_mark: |
127-
| [Hot Reload](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HotReload) | | | :heavy_check_mark: |
128-
| [Retry](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Retry) | | | :heavy_check_mark: |
129-
| [Trx](https://www.nuget.org/packages/Microsoft.Testing.Extensions.TrxReport) | | :heavy_check_mark: | :heavy_check_mark: |
130-
131-
† MSTest.Sdk 3.7.0+
122+
| Extension/Profile | None | Default | AllMicrosoft |
123+
| ------------------------------------------------------------------------------------------------- | :---: | :----------------: | :-----------------: |
124+
| [Code Coverage](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CodeCoverage) | | :heavy_check_mark: | :heavy_check_mark: |
125+
| [Crash Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CrashDump) | | | :heavy_check_mark: |
126+
| [Fakes](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Fakes) | | | :heavy_check_mark:¹ |
127+
| [Hang Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HangDump) | | | :heavy_check_mark: |
128+
| [Hot Reload](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HotReload) | | | :heavy_check_mark: |
129+
| [Retry](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Retry) | | | :heavy_check_mark: |
130+
| [Trx](https://www.nuget.org/packages/Microsoft.Testing.Extensions.TrxReport) | | :heavy_check_mark: | :heavy_check_mark: |
131+
| [AzureDevOpsReport](./microsoft-testing-platform-extensions-test-reports.md#azure-devops-reports) | | | :heavy_check_mark:² |
132+
133+
¹ MSTest.Sdk 3.7.0+
134+
² MSTest.Sdk 3.11.0+
132135

133136
### Enable or disable extensions
134137

0 commit comments

Comments
 (0)