Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,33 @@ The available options as follows:
| `--report-trx-filename` | The name of the generated TRX report. The default name matches the following format `<UserName>_<MachineName>_<yyyy-MM-dd HH:mm:ss>.trx`. |

The report is saved inside the default _TestResults_ folder that can be specified through the `--results-directory` command line argument.

## Azure DevOps reports

Azure DevOps report plugin enhances test running for developers that host their code on GitHub, but build on Azure DevOps build agents. It adds additional information to failures to show failure directly in GitHub PR.

![Error annotation in GitHub PR files view](./media/test-azdoreport-failure.png)

The extension is shipped in [Microsoft.Testing.Extensions.AzureDevOpsReport](https://nuget.org/packages/Microsoft.Testing.Extensions.AzureDevOpsReport) package.

The available options as follows:

| Option | Description |
|--|--|
| `--report-azdo` | Enable outputting errors / warnings in CI builds. |
| `--report-azdo-severity` | Severity to use for the reported event. Options are: `error` (default) and `warning`. |

The extension automatically detects that it is running in continuous integration (CI) environment by checking the `TF_BUILD` environment variable.

### Determining the line to report

To highlight the correct line in code where failure occurred, AzureDevOps report plugin searches the error stacktrace for a file that exists in the current repository.

To determine this it:

- finds the repository root, this is done by searching the `.git` directory closest to the location from where the test application is started (as determined by `AppContext.BaseDirectory`).
- finds the first line in stack trace that has file location and line (the library needs to have debug symbols).
- excludes all files that end with `Assert.cs` to avoid showing details of your assertion implementations or wrappers.
- excludes all files that don't exist on disk (typically those are lines from external libraries that ship debug symbols e.g. MSTest).

(These are internal details that serve to aid debugging the behavior and might change in the future.)
25 changes: 14 additions & 11 deletions docs/core/testing/unit-testing-mstest-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ You can set the profile using the property `TestingExtensionsProfile` with one o
* [Hot Reload](./microsoft-testing-platform-extensions-hosting.md#hot-reload)
* [Retry](./microsoft-testing-platform-extensions-policy.md#retry)
* [Trx Report](./microsoft-testing-platform-extensions-test-reports.md#visual-studio-test-reports)
* [AzureDevOpsReport](./microsoft-testing-platform-extensions-test-reports.md#azure-devops-reports)

Here's a full example, using the `None` profile:

Expand All @@ -118,17 +119,19 @@ Here's a full example, using the `None` profile:
</Project>
```

| Extension/Profile | None | Default | AllMicrosoft |
|-------------------------------------------------------------------------------------------|:----:|:------------------:|:--------------------------------------:|
| [Code Coverage](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CodeCoverage) | | :heavy_check_mark: | :heavy_check_mark: |
| [Crash Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CrashDump) | | | :heavy_check_mark: |
| [Fakes](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Fakes) | | | :heavy_check_mark:† |
| [Hang Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HangDump) | | | :heavy_check_mark: |
| [Hot Reload](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HotReload) | | | :heavy_check_mark: |
| [Retry](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Retry) | | | :heavy_check_mark: |
| [Trx](https://www.nuget.org/packages/Microsoft.Testing.Extensions.TrxReport) | | :heavy_check_mark: | :heavy_check_mark: |

† MSTest.Sdk 3.7.0+
| Extension/Profile | None | Default | AllMicrosoft |
| ------------------------------------------------------------------------------------------------- | :---: | :----------------: | :-----------------: |
| [Code Coverage](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CodeCoverage) | | :heavy_check_mark: | :heavy_check_mark: |
| [Crash Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CrashDump) | | | :heavy_check_mark: |
| [Fakes](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Fakes) | | | :heavy_check_mark:¹ |
| [Hang Dump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HangDump) | | | :heavy_check_mark: |
| [Hot Reload](https://www.nuget.org/packages/Microsoft.Testing.Extensions.HotReload) | | | :heavy_check_mark: |
| [Retry](https://www.nuget.org/packages/Microsoft.Testing.Extensions.Retry) | | | :heavy_check_mark: |
| [Trx](https://www.nuget.org/packages/Microsoft.Testing.Extensions.TrxReport) | | :heavy_check_mark: | :heavy_check_mark: |
| [AzureDevOpsReport](./microsoft-testing-platform-extensions-test-reports.md#azure-devops-reports) | | | :heavy_check_mark:² |

¹ MSTest.Sdk 3.7.0+
² MSTest.Sdk 3.11.0+

### Enable or disable extensions

Expand Down