From d20528fb6754776b5661c1be69ece25d3e5e4cc6 Mon Sep 17 00:00:00 2001 From: Nikhil Gupta <108018378+NikhilGupta-Dev@users.noreply.github.com> Date: Thu, 28 Aug 2025 18:45:47 +0530 Subject: [PATCH 1/3] Docs: Add Bazel build support reference.Docs: Add Bazel build support reference.Update running-tests.adoc --- .../asciidoc/user-guide/running-tests.adoc | 1181 +---------------- 1 file changed, 4 insertions(+), 1177 deletions(-) diff --git a/documentation/src/docs/asciidoc/user-guide/running-tests.adoc b/documentation/src/docs/asciidoc/user-guide/running-tests.adoc index 5d5e9ce0c826..4ff3832c4f7b 100644 --- a/documentation/src/docs/asciidoc/user-guide/running-tests.adoc +++ b/documentation/src/docs/asciidoc/user-guide/running-tests.adoc @@ -1,1181 +1,8 @@ -[[running-tests]] -== Running Tests +[[running-tests-build-bazel]] +==== Bazel -[[running-tests-ide]] -=== IDE Support +Bazel provides support for running tests on the JUnit Platform through the rules_jvm_external extension and test configuration. -[[running-tests-ide-intellij-idea]] -==== IntelliJ IDEA - -IntelliJ IDEA supports running tests on the JUnit Platform since version 2016.2. For more -information, please consult this https://jb.gg/junit-idea/[IntelliJ IDEA resource]. Note, -however, that it is recommended to use IDEA 2017.3 or newer since more recent versions of -IDEA download the following JARs automatically based on the API version used in the -project: `junit-platform-launcher`, `junit-jupiter-engine`, and `junit-vintage-engine`. - -In order to use a different JUnit version (e.g., {version}), you may need to -include the corresponding versions of the `junit-platform-launcher`, -`junit-jupiter-engine`, and `junit-vintage-engine` JARs in the classpath. - -.Additional Gradle Dependencies -[source,groovy] -[subs=attributes+] ----- -testImplementation(platform("org.junit:junit-bom:{version}")) -testRuntimeOnly("org.junit.platform:junit-platform-launcher") -testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine") -testRuntimeOnly("org.junit.vintage:junit-vintage-engine") ----- - -.Additional Maven Dependencies -[source,xml] -[subs=attributes+] ----- - - - - org.junit.platform - junit-platform-launcher - test - - - org.junit.jupiter - junit-jupiter-engine - test - - - org.junit.vintage - junit-vintage-engine - test - - - - - - org.junit - junit-bom - {version} - pom - import - - - ----- - -[[running-tests-ide-eclipse]] -==== Eclipse - -Eclipse IDE offers support for the JUnit Platform since the Eclipse Oxygen.1a (4.7.1a) -release. - -For more information on using JUnit Platform in Eclipse consult the official _Eclipse -support for JUnit 5_ section of the -https://www.eclipse.org/eclipse/news/4.7.1a/#junit-5-support[Eclipse Project Oxygen.1a -(4.7.1a) - New and Noteworthy] documentation. - -[[running-tests-ide-netbeans]] -==== NetBeans - -NetBeans offers support for JUnit Jupiter and the JUnit Platform since the -https://netbeans.apache.org/download/nb100/nb100.html[Apache NetBeans 10.0 release]. - -For more information consult the JUnit 5 section of the -https://netbeans.apache.org/download/nb100/index.html#_junit_5[Apache NetBeans 10.0 -release notes]. - -[[running-tests-ide-vscode]] -==== Visual Studio Code - -https://code.visualstudio.com/[Visual Studio Code] supports JUnit Jupiter and the JUnit -Platform via the -https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-test[Java Test -Runner] extension which is installed by default as part of the -https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-pack[Java -Extension Pack]. - -For more information consult the _Testing_ section of the -https://code.visualstudio.com/docs/languages/java#_testing[Java in Visual Studio Code] -documentation. - -[[running-tests-ide-other]] -==== Other IDEs - -If you are using an editor or IDE other than one of those listed in the previous sections, -and it doesn't support running tests on the JUnit Platform, you can use the -<> to run them from the command line. - -[[running-tests-build]] -=== Build Support - -[[running-tests-build-gradle]] -==== Gradle - -Starting with https://docs.gradle.org/4.6/release-notes.html[version 4.6], Gradle provides -https://docs.gradle.org/current/userguide/java_testing.html#using_junit5[native support] -for executing tests on the JUnit Platform. To enable it, you need to specify -`useJUnitPlatform()` within a `test` task declaration in `build.gradle`: - -[source,groovy,indent=0] -[subs=attributes+] ----- -test { - useJUnitPlatform() -} ----- - -Filtering by <>, -<>, or engines is also supported: - -[source,groovy,indent=0] -[subs=attributes+] ----- -test { - useJUnitPlatform { - includeTags("fast", "smoke & feature-a") - // excludeTags("slow", "ci") - includeEngines("junit-jupiter") - // excludeEngines("junit-vintage") - } -} ----- - -Please refer to the -https://docs.gradle.org/current/userguide/java_testing.html[official Gradle documentation] -for a comprehensive list of options. - -[[running-tests-build-gradle-bom]] -===== Aligning dependency versions - -TIP: See <> for details on how to override the version -of JUnit used in your Spring Boot application. - -Unless you're using Spring Boot which defines its own way of managing dependencies, it is -recommended to use the JUnit Platform <> to align the -versions of all JUnit artifacts. - -[source,groovy,indent=0] -[subs=attributes+] -.Explicit platform dependency on the BOM ----- -dependencies { - testImplementation(platform("org.junit:junit-bom:{version}")) - testImplementation("org.junit.jupiter:junit-jupiter") - testRuntimeOnly("org.junit.platform:junit-platform-launcher") -} ----- - -Using the BOM allows you to omit the version when declaring dependencies on all artifacts -with the `org.junit.platform`, `org.junit.jupiter`, and `org.junit.vintage` group IDs. - -Since all JUnit artifacts declare a -https://docs.gradle.org/current/userguide/platforms.html[platform] dependency on the BOM, -you usually don't need to declare an explicit dependency on it yourself. Instead, it's -sufficient to declare _one_ regular dependency that includes a version number. Gradle will -then pull in the BOM automatically so you can omit the version for all other JUnit -artifacts. - -[source,groovy,indent=0] -[subs=attributes+] -.Implicit platform dependency on the BOM ----- -dependencies { - testImplementation("org.junit.jupiter:junit-jupiter:{version}") // <1> - testRuntimeOnly("org.junit.platform:junit-platform-launcher") // <2> -} ----- -<1> Dependency declaration with explicit version. Pulls in the `junit-bom` automatically. -<2> Dependency declaration without version. The version is supplied by the `junit-bom`. - -[WARNING] -.Declaring a dependency on junit-platform-launcher -==== -Even though pre-8.0 versions of Gradle don't require declaring an explicit -dependency on `junit-platform-launcher`, it is recommended to do so to ensure the versions -of JUnit artifacts on the test runtime classpath are aligned. - -Moreover, doing so is recommended and in some cases even required when importing the -project into an IDE like <> or -<>. -==== - -[[running-tests-build-gradle-engines-configure]] -===== Configuring Test Engines - -In order to run any tests at all, a `TestEngine` implementation must be on the classpath. - -To configure support for JUnit Jupiter based tests, configure a `testImplementation` dependency -on the dependency-aggregating JUnit Jupiter artifact similar to the following. - -[source,groovy,indent=0] -[subs=attributes+] ----- -dependencies { - testImplementation("org.junit.jupiter:junit-jupiter:{version}") - testRuntimeOnly("org.junit.platform:junit-platform-launcher") -} ----- - -Alternatively, you can use Gradle's -https://docs.gradle.org/current/userguide/jvm_test_suite_plugin.html[JVM Test Suite] -support. - -[source,kotlin,indent=0] -[subs=attributes+] -.Kotlin DSL ----- -testing { - suites { - named("test") { - useJUnitJupiter("{version}") - } - } -} ----- - -[source,groovy,indent=0] -[subs=attributes+] -.Groovy DSL ----- -testing { - suites { - test { - useJUnitJupiter("{version}") - } - } -} ----- - -The JUnit Platform can run JUnit 4 based tests as long as you configure a `testImplementation` -dependency on JUnit 4 and a `testRuntimeOnly` dependency on the JUnit Vintage `TestEngine` -implementation similar to the following. - -[source,groovy,indent=0] -[subs=attributes+] ----- -dependencies { - testImplementation("junit:junit:{junit4-version}") - testRuntimeOnly("org.junit.vintage:junit-vintage-engine:{version}") - testRuntimeOnly("org.junit.platform:junit-platform-launcher") -} ----- - -[[running-tests-build-gradle-config-params]] -===== Configuration Parameters - -The standard Gradle `test` task currently does not provide a dedicated DSL to set JUnit -Platform <> to influence test -discovery and execution. However, you can provide configuration parameters within the -build script via system properties (as shown below) or via the -`junit-platform.properties` file. - -[source,groovy,indent=0] ----- -test { - // ... - systemProperty("junit.jupiter.conditions.deactivate", "*") - systemProperty("junit.jupiter.extensions.autodetection.enabled", true) - systemProperty("junit.jupiter.testinstance.lifecycle.default", "per_class") - // ... -} ----- - -[[running-tests-build-gradle-logging]] -===== Configuring Logging (optional) - -JUnit uses the Java Logging APIs in the `java.util.logging` package (a.k.a. _JUL_) to -emit warnings and debug information. Please refer to the official documentation of -`{LogManager}` for configuration options. - -Alternatively, it's possible to redirect log messages to other logging frameworks such as -{Log4j} or {Logback}. To use a logging framework that provides a custom implementation of -`{LogManager}`, set the `java.util.logging.manager` system property to the _fully -qualified class name_ of the `{LogManager}` implementation to use. The example below -demonstrates how to configure Log4j{nbsp}2.x (see {Log4j_JDK_Logging_Adapter} for -details). - -[source,groovy,indent=0] -[subs=attributes+] ----- -test { - systemProperty("java.util.logging.manager", "org.apache.logging.log4j.jul.LogManager") - // Avoid overhead (see https://logging.apache.org/log4j/2.x/manual/jmx.html#enabling-jmx) - systemProperty("log4j2.disableJmx", "true") -} ----- - -Other logging frameworks provide different means to redirect messages logged using -`java.util.logging`. For example, for {Logback} you can use the -https://www.slf4j.org/legacy.html#jul-to-slf4j[JUL to SLF4J Bridge] by adding it as a -dependency to the test runtime classpath. - -[[running-tests-build-maven]] -==== Maven - -Maven Surefire and Maven Failsafe provide -https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit-platform.html[native support] -for executing tests on the JUnit Platform. The `pom.xml` file in the -`{junit-jupiter-starter-maven}` project demonstrates how to use the Maven Surefire plugin -and can serve as a starting point for configuring your Maven build. - -[WARNING] -.Minimum required version of Maven Surefire/Failsafe -==== -As of JUnit 6.0, the minimum required version of Maven Surefire/Failsafe is 3.0.0. -==== - -[[running-tests-build-maven-bom]] -===== Aligning dependency versions - -Unless you're using Spring Boot which defines its own way of managing dependencies, it is -recommended to use the JUnit Platform <> to align the -versions of all JUnit artifacts. - -[source,xml,indent=0] -[subs=attributes+] ----- - - - - org.junit - junit-bom - {version} - pom - import - - - ----- - -Using the BOM allows you to omit the version when declaring dependencies on all artifacts -with the `org.junit.platform`, `org.junit.jupiter`, and `org.junit.vintage` group IDs. - -TIP: See <> for details on how to override the version -of JUnit used in your Spring Boot application. - -[[running-tests-build-maven-engines-configure]] -===== Configuring Test Engines - -In order to have Maven Surefire or Maven Failsafe run any tests at all, at least one -`TestEngine` implementation must be added to the test classpath. - -To configure support for JUnit Jupiter based tests, configure `test` scoped dependencies -on the JUnit Jupiter API and the JUnit Jupiter `TestEngine` implementation similar to the -following. - -[source,xml,indent=0] -[subs=attributes+] ----- - - - - - org.junit.jupiter - junit-jupiter - {version} - test - - - - - - - maven-surefire-plugin - {surefire-version} - - - maven-failsafe-plugin - {surefire-version} - - - - ----- - -Maven Surefire and Maven Failsafe can run JUnit 4 based tests alongside Jupiter tests as -long as you configure `test` scoped dependencies on JUnit 4 and the JUnit Vintage -`TestEngine` implementation similar to the following. - -[source,xml,indent=0] -[subs=attributes+] ----- - - - - - junit - junit - {junit4-version} - test - - - org.junit.vintage - junit-vintage-engine - {version} - test - - - - - - - - maven-surefire-plugin - {surefire-version} - - - maven-failsafe-plugin - {surefire-version} - - - - ----- - -[[running-tests-build-maven-filter-test-class-names]] -===== Filtering by Test Class Names - -The Maven Surefire Plugin will scan for test classes whose fully qualified names match -the following patterns. - -- `+++**/Test*.java+++` -- `+++**/*Test.java+++` -- `+++**/*Tests.java+++` -- `+++**/*TestCase.java+++` - -Moreover, it will exclude all nested classes (including static member classes) by default. - -Note, however, that you can override this default behavior by configuring explicit -`include` and `exclude` rules in your `pom.xml` file. For example, to keep Maven Surefire -from excluding static member classes, you can override its exclude rules as follows. - -[source,xml,indent=0] -[subs=attributes+] -.Overriding exclude rules of Maven Surefire ----- - - - - - maven-surefire-plugin - {surefire-version} - - - - - - - - - ----- - -Please see the -https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html[Inclusions and Exclusions of Tests] -documentation for Maven Surefire for details. - -[[running-tests-build-maven-filter-tags]] -===== Filtering by Tags - -You can filter tests by <> or -<> using the following configuration -properties. - -- to include _tags_ or _tag expressions_, use `groups`. -- to exclude _tags_ or _tag expressions_, use `excludedGroups`. - -[source,xml,indent=0] -[subs=attributes+] ----- - - - - - maven-surefire-plugin - {surefire-version} - - acceptance | !feature-a - integration, regression - - - - - ----- - -[[running-tests-build-maven-config-params]] -===== Configuration Parameters - -You can set JUnit Platform <> to -influence test discovery and execution by declaring the `configurationParameters` -property and providing key-value pairs using the Java `Properties` file syntax (as shown -below) or via the `junit-platform.properties` file. - -[source,xml,indent=0] -[subs=attributes+] ----- - - - - - maven-surefire-plugin - {surefire-version} - - - - junit.jupiter.conditions.deactivate = * - junit.jupiter.extensions.autodetection.enabled = true - junit.jupiter.testinstance.lifecycle.default = per_class - - - - - - - ----- - -[[running-tests-build-ant]] -==== Ant - -Starting with version `1.10.3`, link:https://ant.apache.org/[Ant] has a -link:https://ant.apache.org/manual/Tasks/junitlauncher.html[`junitlauncher`] task that -provides native support for launching tests on the JUnit Platform. The `junitlauncher` -task is solely responsible for launching the JUnit Platform and passing it the selected -collection of tests. The JUnit Platform then delegates to registered test engines to -discover and execute the tests. - -The `junitlauncher` task attempts to align as closely as possible with native Ant -constructs such as -link:https://ant.apache.org/manual/Types/resources.html#collection[resource collections] -for allowing users to select the tests that they want executed by test engines. This gives -the task a consistent and natural feel when compared to many other core Ant tasks. - -Starting with version `1.10.6` of Ant, the `junitlauncher` task supports -link:https://ant.apache.org/manual/Tasks/junitlauncher.html#fork[forking the tests in a separate JVM]. - -The `build.xml` file in the `{junit-jupiter-starter-ant}` project demonstrates how to use -the task and can serve as a starting point. - -===== Basic Usage - -The following example demonstrates how to configure the `junitlauncher` task to select a -single test class (i.e., `com.example.project.CalculatorTests`). - -[source,xml,indent=0] ----- - - - - - - - - - - - ----- - -The `test` element allows you to specify a single test class that you want to be selected -and executed. The `classpath` element allows you to specify the classpath to be used to -launch the JUnit Platform. This classpath will also be used to locate test classes that -are part of the execution. - -The following example demonstrates how to configure the `junitlauncher` task to select -test classes from multiple locations. - -[source,xml,indent=0] ----- - - - - - - - - - - - - - - - - ----- - -In the above example, the `testclasses` element allows you to select multiple test -classes that reside in different locations. - -For further details on usage and configuration options please refer to the official Ant -documentation for the -link:https://ant.apache.org/manual/Tasks/junitlauncher.html[`junitlauncher` task]. - -[[running-tests-build-spring-boot]] -==== Spring Boot - -link:https://spring.io/projects/spring-boot[Spring Boot] provides automatic support for -managing the version of JUnit used in your project. In addition, the -`spring-boot-starter-test` artifact automatically includes testing libraries such as JUnit -Jupiter, AssertJ, Mockito, etc. - -If your build relies on dependency management support from Spring Boot, you should not -import JUnit's <> in your build script since that would -result in duplicate (and potentially conflicting) management of JUnit dependencies. - -If you need to override the version of a dependency used in your Spring Boot application, -you have to override the exact name of the -link:https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.dependency-versions.properties[version property] -defined in the BOM used by the Spring Boot plugin. For example, the name of the JUnit -Jupiter version property in Spring Boot is `junit-jupiter.version`. The mechanism for -changing a dependency version is documented for both -link:https://docs.spring.io/spring-boot/docs/current/gradle-plugin/reference/htmlsingle/#managing-dependencies.dependency-management-plugin.customizing[Gradle] -and -link:https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/#using.parent-pom[Maven]. - -With Gradle you can override the JUnit Jupiter version by including the following in your -`build.gradle` file. - -[source,groovy,indent=0] -[subs=attributes+] ----- - ext['junit-jupiter.version'] = '{version}' ----- - -With Maven you can override the JUnit Jupiter version by including the following in your -`pom.xml` file. - -[source,xml,indent=0] -[subs=attributes+] ----- - - {version} - ----- +For a complete working example of how to set up JUnit 5 with Bazel, see the https://github.com/junit-team/junit5-samples/tree/main/junit5-jupiter-starter-bazel[JUnit 5 + Bazel Starter Sample] project. [[running-tests-console-launcher]] -=== Console Launcher - -The `{ConsoleLauncher}` is a command-line Java application that lets you launch the JUnit -Platform from the console. For example, it can be used to run JUnit Vintage and JUnit -Jupiter tests and print test execution results to the console. - -An executable _Fat JAR_ (`junit-platform-console-standalone-{version}.jar`) that -contains the contents of all of its dependencies is published in the {Maven_Central} -repository under the -https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone[junit-platform-console-standalone] -directory. It contains the contents of the following artifacts: - -include::{standaloneConsoleLauncherShadowedArtifactsFile}[] - -[NOTE] -==== -Since the `junit-platform-console-standalone` JAR contains the contents of all of its -dependencies, its Maven POM does not declare any dependencies. - -Furthermore, it is not very likely that you would need to include a dependency on the -`junit-platform-console-standalone` artifact in your project's Maven POM or Gradle build -script. On the contrary, the executable `junit-platform-console-standalone` JAR is -typically invoked directly from the command line or a shell script without a build script. - -If you need to declare dependencies in your build script on some of the artifacts -contained in the `junit-platform-console-standalone` artifact, you should declare -dependencies only on the JUnit artifacts that are used in your project. To simplify -dependency management of JUnit artifacts in your build, you may wish to use the -`junit-jupiter` aggregator artifact or `junit-bom`. See <> for -details. -==== - -You can https://docs.oracle.com/javase/tutorial/deployment/jar/run.html[run] the -standalone `ConsoleLauncher` as shown below. - -[source,console,subs=attributes+] ----- -$ java -jar junit-platform-console-standalone-{version}.jar execute - -├─ JUnit Vintage -│ └─ example.JUnit4Tests -│ └─ standardJUnit4Test ✔ -└─ JUnit Jupiter - ├─ StandardTests - │ ├─ succeedingTest() ✔ - │ └─ skippedTest() ↷ for demonstration purposes - └─ A special test case - ├─ Custom test name containing spaces ✔ - ├─ ╯°□°)╯ ✔ - └─ 😱 ✔ - -Test run finished after 64 ms -[ 5 containers found ] -[ 0 containers skipped ] -[ 5 containers started ] -[ 0 containers aborted ] -[ 5 containers successful ] -[ 0 containers failed ] -[ 6 tests found ] -[ 1 tests skipped ] -[ 5 tests started ] -[ 0 tests aborted ] -[ 5 tests successful ] -[ 0 tests failed ] ----- - -You can also run the standalone `ConsoleLauncher` as shown below (for example, to include -all jars in a directory): - -[source,console,subs=attributes+] ----- -$ java -cp classes:testlib/* org.junit.platform.console.ConsoleLauncher ----- - -[[running-tests-console-launcher-options]] -==== Subcommands and Options - -The `{ConsoleLauncher}` provides the following subcommands: - ----- -include::{consoleLauncherOptionsFile}[] ----- - -[[running-tests-console-launcher-options-discovering-tests]] -===== Discovering tests - ----- -include::{consoleLauncherDiscoverOptionsFile}[] ----- - -[[running-tests-console-launcher-options-executing-tests]] -===== Executing tests - -.Exit Code -NOTE: The `{ConsoleLauncher}` exits with a status code of `1` if any containers or tests -failed. If no tests are discovered and the `--fail-if-no-tests` command-line option is -supplied, the `ConsoleLauncher` exits with a status code of `2`. Otherwise, the exit code -is `0`. - ----- -include::{consoleLauncherExecuteOptionsFile}[] ----- - -[[running-tests-console-launcher-options-listing-test-engines]] -===== Listing test engines - ----- -include::{consoleLauncherEnginesOptionsFile}[] ----- - -[[running-tests-console-launcher-argument-files]] -==== Argument Files (@-files) - -On some platforms you may run into system limitations on the length of a command line when -creating a command line with lots of options or with long arguments. - -The `ConsoleLauncher` supports _argument files_, also known as _@-files_. Argument files -are files that themselves contain arguments to be passed to the command. When the -underlying https://github.com/remkop/picocli[picocli] command line parser encounters an -argument beginning with the character `@`, it expands the contents of that file into the -argument list. - -The arguments within a file can be separated by spaces or newlines. If an argument -contains embedded whitespace, the whole argument should be wrapped in double or single -quotes -- for example, `"-f=My Files/Stuff.java"`. - -If the argument file does not exist or cannot be read, the argument will be treated -literally and will not be removed. This will likely result in an "unmatched argument" -error message. You can troubleshoot such errors by executing the command with the -`picocli.trace` system property set to `DEBUG`. - -Multiple _@-files_ may be specified on the command line. The specified path may be -relative to the current directory or absolute. - -You can pass a real parameter with an initial `@` character by escaping it with an -additional `@` symbol. For example, `@@somearg` will become `@somearg` and will not be -subject to expansion. - -[[running-tests-console-launcher-redirecting-stdout-and-stderr]] -==== Redirecting Standard Output/Error to Files - -You can redirect the `System.out` (stdout) and `System.err` (stderr) output streams to -files using the `--redirect-stdout` and `--redirect-stderr` options: - -[source,console,subs=attributes+] ----- -$ java -jar junit-platform-console-standalone-{version}.jar \ - --redirect-stdout=stdout.txt \ - --redirect-stderr=stderr.txt ----- - -[NOTE] -==== -If the `--redirect-stdout` and `--redirect-stderr` arguments point to the same file, both -output streams will be redirected to that file. - -The default charset is used for writing to the files. -==== - -[[running-tests-console-launcher-color-customization]] -==== Color Customization - -The colors used in the output of the `{ConsoleLauncher}` can be customized. -The option `--single-color` will apply a built-in monochrome style, while -`--color-palette` will accept a properties file to override the -https://en.wikipedia.org/wiki/ANSI_escape_code#Colors[ANSI SGR] color styling. -The properties file below demonstrates the default style: - -[source,properties,indent=0] ----- -SUCCESSFUL = 32 -ABORTED = 33 -FAILED = 31 -SKIPPED = 35 -CONTAINER = 35 -TEST = 34 -DYNAMIC = 35 -REPORTED = 37 ----- - -[[running-tests-discovery-selectors]] -=== Discovery Selectors - -The JUnit Platform provides a rich set of discovery selectors that can be used to specify -which tests should be discovered or executed. - -Discovery selectors can be created programmatically using the factory methods in the -`{DiscoverySelectors}` class, specified declaratively via annotations when using the -<>, via options of the <>, or -generically as strings via their identifiers. - -The following discovery selectors are provided out of the box: - -|=== -| Java Type | API | Annotation | Console Launcher | Identifier - -| `{ClasspathResourceSelector}` | `{DiscoverySelectors_selectClasspathResource}` | `{SelectClasspathResource}` | `--select-resource /foo.csv` | `resource:/foo.csv` -| `{ClasspathRootSelector}` | `{DiscoverySelectors_selectClasspathRoots}` | -- | `--scan-classpath bin` | `classpath-root:bin` -| `{ClassSelector}` | `{DiscoverySelectors_selectClass}` | `{SelectClasses}` | `--select-class com.acme.Foo` | `class:com.acme.Foo` -| `{DirectorySelector}` | `{DiscoverySelectors_selectDirectory}` | `{SelectDirectories}` | `--select-directory foo/bar` | `directory:foo/bar` -| `{FileSelector}` | `{DiscoverySelectors_selectFile}` | `{SelectFile}` | `--select-file dir/foo.txt` | `file:dir/foo.txt` -| `{IterationSelector}` | `{DiscoverySelectors_selectIteration}` | `{Select}("")` | `--select-iteration method=com.acme.Foo#m[1..2]` | `iteration:method:com.acme.Foo#m[1..2]` -| `{MethodSelector}` | `{DiscoverySelectors_selectMethod}` | `{SelectMethod}` | `--select-method com.acme.Foo#m` | `method:com.acme.Foo#m` -| `{ModuleSelector}` | `{DiscoverySelectors_selectModule}` | `{SelectModules}` | `--select-module com.acme` | `module:com.acme` -| `{NestedClassSelector}` | `{DiscoverySelectors_selectNestedClass}` | `{Select}("")` | `--select ` | `nested-class:com.acme.Foo/Bar` -| `{NestedMethodSelector}` | `{DiscoverySelectors_selectNestedMethod}` | `{Select}("")` | `--select ` | `nested-method:com.acme.Foo/Bar#m` -| `{PackageSelector}` | `{DiscoverySelectors_selectPackage}` | `{SelectPackages}` | `--select-package com.acme.foo` | `package:com.acme.foo` -| `{UniqueIdSelector}` | `{DiscoverySelectors_selectUniqueId}` | `{Select}("")` | `--select-unique-id ` | `uid:[engine:Foo]/[segment:Bar]` -| `{UriSelector}` | `{DiscoverySelectors_selectUri}` | `{SelectUris}` | `--select-uri \file:///foo.txt` | `uri:file:///foo.txt` -|=== - - -[[running-tests-config-params]] -=== Configuration Parameters - -In addition to instructing the platform which test classes and test engines to include, -which packages to scan, etc., it is sometimes necessary to provide additional custom -configuration parameters that are specific to a particular test engine, listener, or -registered extension. For example, the JUnit Jupiter `TestEngine` supports _configuration -parameters_ for the following use cases. - -- <> -- <> -- <> -- <> - -_Configuration Parameters_ are text-based key-value pairs that can be supplied to test -engines running on the JUnit Platform via one of the following mechanisms. - -1. The `configurationParameter()` and `configurationParameters()` methods in - `LauncherDiscoveryRequestBuilder` which - is used to build a request supplied to the <>. - + - When running tests via one of the tools provided by the JUnit Platform you can specify - configuration parameters as follows: - * <>: use the `--config` command-line - option. - * <>: use the `systemProperty` or - `systemProperties` DSL. - * <>: use the - `configurationParameters` property. -2. The `configurationParametersResources()` method in `LauncherDiscoveryRequestBuilder`. - + - When running tests via the <> you can - specify custom configuration files using the `--config-resource` command-line option. -3. JVM system properties. -4. The JUnit Platform default configuration file: a file named `junit-platform.properties` - in the root of the class path that follows the syntax rules for Java `Properties` - files. - -NOTE: Configuration parameters are looked up in the exact order defined above. -Consequently, configuration parameters supplied directly to the `Launcher` take -precedence over those supplied via custom configuration files, system properties, and the -default configuration file. Similarly, configuration parameters supplied via system -properties take precedence over those supplied via the default configuration file. - -[[running-tests-config-params-deactivation-pattern]] -==== Pattern Matching Syntax - -This section describes the pattern matching syntax that is applied to the _configuration -parameters_ used for the following features. - -- <> -- <> -- <> -- <> - -If the value for the given _configuration parameter_ consists solely of an asterisk -(`+++*+++`), the pattern will match against all candidate classes. Otherwise, the value -will be treated as a comma-separated list of patterns where each pattern will be matched -against the fully qualified class name (_FQCN_) of each candidate class. Any dot (`.`) in -a pattern will match against a dot (`.`) or a dollar sign (`$`) in a FQCN. Any asterisk -(`+++*+++`) will match against one or more characters in a FQCN. All other characters in a -pattern will be matched one-to-one against a FQCN. - -Examples: - -- `+++*+++`: matches all candidate classes. -- `+++org.junit.*+++`: matches all candidate classes under the `org.junit` base package and - any of its subpackages. -- `+++*.MyCustomImpl+++`: matches every candidate class whose simple class name is exactly - `MyCustomImpl`. -- `+++*System*+++`: matches every candidate class whose FQCN contains `System`. -- `+++*System*+++, +++*Unit*+++`: matches every candidate class whose FQCN contains - `System` or `Unit`. -- `org.example.MyCustomImpl`: matches the candidate class whose FQCN is exactly - `org.example.MyCustomImpl`. -- `org.example.MyCustomImpl, org.example.TheirCustomImpl`: matches candidate classes whose - FQCN is exactly `org.example.MyCustomImpl` or `org.example.TheirCustomImpl`. - -[[running-tests-tags]] -=== Tags - -Tags are a JUnit Platform concept for marking and filtering tests. The programming model -for adding tags to containers and tests is defined by the testing framework. For example, -in JUnit Jupiter based tests, the `@Tag` annotation (see -<>) should be used. For JUnit 4 based tests, the -Vintage engine maps `@Category` annotations to tags (see -<>). Other testing frameworks may define their -own annotation or other means for users to specify tags. - -[[running-tests-tag-syntax-rules]] -==== Syntax Rules for Tags - -Regardless how a tag is specified, the JUnit Platform enforces the following rules: - -* A tag must not be `null` or _blank_. -* A _stripped_ tag must not contain whitespace. -* A _stripped_ tag must not contain ISO control characters. -* A _stripped_ tag must not contain any of the following _reserved characters_. -- `,`: _comma_ -- `(`: _left parenthesis_ -- `)`: _right parenthesis_ -- `&`: _ampersand_ -- `|`: _vertical bar_ -- `!`: _exclamation point_ - -NOTE: In the above context, "stripped" means that leading and trailing whitespace -characters have been removed using `java.lang.String.strip()`. - -[[running-tests-tag-expressions]] -==== Tag Expressions - -Tag expressions are boolean expressions with the operators `!`, `&` and `|`. In addition, -`(` and `)` can be used to adjust for operator precedence. - -Two special expressions are supported, `any()` and `none()`, which select all tests _with_ -any tags at all, and all tests _without_ any tags, respectively. -These special expressions may be combined with other expressions just like normal tags. - -.Operators (in descending order of precedence) -|=== -| Operator | Meaning | Associativity - -| `!` | not | right -| `&` | and | left -| `\|` | or | left -|=== - -If you are tagging your tests across multiple dimensions, tag expressions help you to -select which tests to execute. When tagging by test type (e.g., _micro_, _integration_, -_end-to-end_) and feature (e.g., *product*, *catalog*, *shipping*), the following tag -expressions can be useful. - -[%header,cols="40,60"] -|=== -| Tag Expression -| Selection - -| `+++product+++` -| all tests for *product* - -| `+++catalog \| shipping+++` -| all tests for *catalog* plus all tests for *shipping* - -| `+++catalog & shipping+++` -| all tests for the intersection between *catalog* and *shipping* - -| `+++product & !end-to-end+++` -| all tests for *product*, but not the _end-to-end_ tests - -| `+++(micro \| integration) & (product \| shipping)+++` -| all _micro_ or _integration_ tests for *product* or *shipping* -|=== - -[[running-tests-capturing-output]] -=== Capturing Standard Output/Error - -The JUnit Platform provides opt-in support for capturing output printed to `System.out` -and `System.err`. To enable it, set the `junit.platform.output.capture.stdout` and/or -`junit.platform.output.capture.stderr` <> to `true`. In addition, you may configure the maximum number of buffered bytes -to be used per executed test or container using `junit.platform.output.capture.maxBuffer`. - -If enabled, the JUnit Platform captures the corresponding output and publishes it as a -report entry using the `stdout` or `stderr` keys to all registered -`{TestExecutionListener}` instances immediately before reporting the test or container as -finished. - -Please note that the captured output will only contain output emitted by the thread that -was used to execute a container or test. Any output by other threads will be omitted -because particularly when -<> it would be impossible -to attribute it to a specific test or container. - -[[running-tests-listeners]] -=== Using Listeners and Interceptors - -The JUnit Platform provides the following listener APIs that allow JUnit, third parties, -and custom user code to react to events fired at various points during the discovery and -execution of a `TestPlan`. - -* `{LauncherSessionListener}`: receives events when a `{LauncherSession}` is opened and - closed. -* `{LauncherInterceptor}`: intercepts test discovery and execution in the context of a - `LauncherSession`. -* `{LauncherDiscoveryListener}`: receives events that occur during test discovery. -* `{TestExecutionListener}`: receives events that occur during test execution. - -The `LauncherSessionListener` API is typically implemented by build tools or IDEs and -registered automatically for you in order to support some feature of the build tool or IDE. - -The `LauncherDiscoveryListener` and `TestExecutionListener` APIs are often implemented in -order to produce some form of report or to display a graphical representation of the test -plan in an IDE. Such listeners may be implemented and automatically registered by a build -tool or IDE, or they may be included in a third-party library – potentially registered -for you automatically. You can also implement and register your own listeners. - -For details on registering and configuring listeners, see the following sections of this -guide. - -* <> -* <> -* <> -* <> -* <> -* <> - -The JUnit Platform provides the following listeners which you may wish to use with your -test suite. - -<> :: - `{LegacyXmlReportGeneratingListener}` can be used via the - <> or registered manually to generate XML reports - compatible with the de facto standard for JUnit 4 based test reports. -+ -`{OpenTestReportGeneratingListener}` generates an XML report in the event-based format -specified by {OpenTestReporting}. It is auto-registered and can be enabled and -configured via <>. -+ -See <> for details. - -<> :: - `FlightRecordingExecutionListener` and `FlightRecordingDiscoveryListener` that generate - Java Flight Recorder events during test discovery and execution. - -`{LoggingListener}` :: - `TestExecutionListener` for logging informational messages for all events via a - `BiConsumer` that consumes `Throwable` and `Supplier`. - -`{SummaryGeneratingListener}` :: - `TestExecutionListener` that generates a summary of the test execution which can be - printed via a `PrintWriter`. - -`{UniqueIdTrackingListener}` :: - `TestExecutionListener` that that tracks the unique IDs of all tests that were skipped - or executed during the execution of the `TestPlan` and generates a file containing the - unique IDs once execution of the `TestPlan` has finished. - -[[running-tests-listeners-flight-recorder]] -==== Flight Recorder Support - -The JUnit Platform provides opt-in support for generating Flight Recorder events. -https://openjdk.java.net/jeps/328[JEP 328] describes the Java Flight Recorder (JFR) as -follows. - -> Flight Recorder records events originating from applications, the JVM, and the OS. -Events are stored in a single file that can be attached to bug reports and examined by -support engineers, allowing after-the-fact analysis of issues in the period leading up -to a problem. - -In order to record Flight Recorder events generated while running tests, you need to -start flight recording when launching a test suite via the following java command line -option. - - -XX:StartFlightRecording:filename=... - -Please consult the manual of your build tool for the appropriate commands. - -To analyze the recorded events, use the -https://docs.oracle.com/en/java/javase/14/docs/specs/man/jfr.html[jfr] -command line tool shipped with recent JDKs or open the recording file with -https://jdk.java.net/jmc/[JDK Mission Control]. - -[[stacktrace-pruning]] -=== Stack Trace Pruning - -The JUnit Platform provides built-in support for pruning stack traces produced by failing -tests. This feature is enabled by default but can be disabled by setting the -`junit.platform.stacktrace.pruning.enabled` _configuration parameter_ to `false`. - -When enabled, all calls from the `org.junit`, `jdk.internal.reflect`, and `sun.reflect` -packages are removed from the stack trace, unless the calls occur after the test itself -or any of its ancestors. For that reason, calls to `{Assertions}` or `{Assumptions}` will -never be excluded. - -In addition, all elements prior to and including the first call from the JUnit Platform -`Launcher` will be removed. - -[[running-tests-discovery-issues]] -=== Discovery Issues - -Test engines may encounter issues during test discovery. For example, the declaration of a -test class or method may be invalid. To avoid such issues from going unnoticed, the JUnit -Platform provides a <> to -report them with different severity levels: - -INFO:: -Indicates that the engine encountered something that could be potentially problematic, but -could also happen due to a valid setup or configuration. - -WARNING:: -Indicates that the engine encountered something that is problematic and might lead to -unexpected behavior or will be removed or changed in a future release. - -ERROR:: -Indicates that the engine encountered something that is definitely problematic and will -lead to unexpected behavior. - -If an engine reports an issue with a severity equal to or higher than a configurable -_critical_ severity, its tests will not be executed. Instead, the engine will be reported -as failed during execution with a `{DiscoveryIssueException}` listing all critical issues. -Non-critical issues will be logged but will not prevent the engine from executing its -tests. The `junit.platform.discovery.issue.severity.critical` -<> can be used to set the critical -severity level. Currently, the default value is `ERROR` but it may be changed in a future -release. - -TIP: To surface all discovery issues in your project, it is recommended to set the -`junit.platform.discovery.issue.severity.critical` configuration parameter to `INFO`. - -In addition, registered `{LauncherDiscoveryListener}` implementations can receive -discovery issues via the `issueEncountered()` method. This allows IDEs and build tools to -report issues to the user in a more user-friendly way. For example, IDEs may choose to -display all issues in a list or table. From 62ef59495997999b8dc2bdef9a61845e425a1955 Mon Sep 17 00:00:00 2001 From: Nikhil Gupta <108018378+NikhilGupta-Dev@users.noreply.github.com> Date: Thu, 28 Aug 2025 19:52:15 +0530 Subject: [PATCH 2/3] Update running-tests.adoc --- .../asciidoc/user-guide/running-tests.adoc | 1169 +++++++++++++++++ 1 file changed, 1169 insertions(+) diff --git a/documentation/src/docs/asciidoc/user-guide/running-tests.adoc b/documentation/src/docs/asciidoc/user-guide/running-tests.adoc index 4ff3832c4f7b..ac4963c13a07 100644 --- a/documentation/src/docs/asciidoc/user-guide/running-tests.adoc +++ b/documentation/src/docs/asciidoc/user-guide/running-tests.adoc @@ -1,8 +1,1177 @@ +[[running-tests]] +== Running Tests [[running-tests-build-bazel]] ==== Bazel +[[running-tests-ide]] +=== IDE Support Bazel provides support for running tests on the JUnit Platform through the rules_jvm_external extension and test configuration. +[[running-tests-ide-intellij-idea]] +==== IntelliJ IDEA + +IntelliJ IDEA supports running tests on the JUnit Platform since version 2016.2. For more +information, please consult this https://jb.gg/junit-idea/[IntelliJ IDEA resource]. Note, +however, that it is recommended to use IDEA 2017.3 or newer since more recent versions of +IDEA download the following JARs automatically based on the API version used in the +project: `junit-platform-launcher`, `junit-jupiter-engine`, and `junit-vintage-engine`. + +In order to use a different JUnit version (e.g., {version}), you may need to +include the corresponding versions of the `junit-platform-launcher`, +`junit-jupiter-engine`, and `junit-vintage-engine` JARs in the classpath. + +.Additional Gradle Dependencies +[source,groovy] +[subs=attributes+] +---- +testImplementation(platform("org.junit:junit-bom:{version}")) +testRuntimeOnly("org.junit.platform:junit-platform-launcher") +testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine") +testRuntimeOnly("org.junit.vintage:junit-vintage-engine") +---- + +.Additional Maven Dependencies +[source,xml] +[subs=attributes+] +---- + + + + org.junit.platform + junit-platform-launcher + test + + + org.junit.jupiter + junit-jupiter-engine + test + + + org.junit.vintage + junit-vintage-engine + test + + + + + + org.junit + junit-bom + {version} + pom + import + + + +---- + +[[running-tests-ide-eclipse]] +==== Eclipse + +Eclipse IDE offers support for the JUnit Platform since the Eclipse Oxygen.1a (4.7.1a) +release. + +For more information on using JUnit Platform in Eclipse consult the official _Eclipse +support for JUnit 5_ section of the +https://www.eclipse.org/eclipse/news/4.7.1a/#junit-5-support[Eclipse Project Oxygen.1a +(4.7.1a) - New and Noteworthy] documentation. + +[[running-tests-ide-netbeans]] +==== NetBeans + +NetBeans offers support for JUnit Jupiter and the JUnit Platform since the +https://netbeans.apache.org/download/nb100/nb100.html[Apache NetBeans 10.0 release]. + +For more information consult the JUnit 5 section of the +https://netbeans.apache.org/download/nb100/index.html#_junit_5[Apache NetBeans 10.0 +release notes]. + +[[running-tests-ide-vscode]] +==== Visual Studio Code + +https://code.visualstudio.com/[Visual Studio Code] supports JUnit Jupiter and the JUnit +Platform via the +https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-test[Java Test +Runner] extension which is installed by default as part of the +https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-pack[Java +Extension Pack]. + +For more information consult the _Testing_ section of the +https://code.visualstudio.com/docs/languages/java#_testing[Java in Visual Studio Code] +documentation. + +[[running-tests-ide-other]] +==== Other IDEs + +If you are using an editor or IDE other than one of those listed in the previous sections, +and it doesn't support running tests on the JUnit Platform, you can use the +<> to run them from the command line. + +[[running-tests-build]] +=== Build Support + +[[running-tests-build-gradle]] +==== Gradle + +Starting with https://docs.gradle.org/4.6/release-notes.html[version 4.6], Gradle provides +https://docs.gradle.org/current/userguide/java_testing.html#using_junit5[native support] +for executing tests on the JUnit Platform. To enable it, you need to specify +`useJUnitPlatform()` within a `test` task declaration in `build.gradle`: + +[source,groovy,indent=0] +[subs=attributes+] +---- +test { + useJUnitPlatform() +} +---- + +Filtering by <>, +<>, or engines is also supported: + +[source,groovy,indent=0] +[subs=attributes+] +---- +test { + useJUnitPlatform { + includeTags("fast", "smoke & feature-a") + // excludeTags("slow", "ci") + includeEngines("junit-jupiter") + // excludeEngines("junit-vintage") + } +} +---- + +Please refer to the +https://docs.gradle.org/current/userguide/java_testing.html[official Gradle documentation] +for a comprehensive list of options. + +[[running-tests-build-gradle-bom]] +===== Aligning dependency versions + +TIP: See <> for details on how to override the version +of JUnit used in your Spring Boot application. + +Unless you're using Spring Boot which defines its own way of managing dependencies, it is +recommended to use the JUnit Platform <> to align the +versions of all JUnit artifacts. + +[source,groovy,indent=0] +[subs=attributes+] +.Explicit platform dependency on the BOM +---- +dependencies { + testImplementation(platform("org.junit:junit-bom:{version}")) + testImplementation("org.junit.jupiter:junit-jupiter") + testRuntimeOnly("org.junit.platform:junit-platform-launcher") +} +---- + +Using the BOM allows you to omit the version when declaring dependencies on all artifacts +with the `org.junit.platform`, `org.junit.jupiter`, and `org.junit.vintage` group IDs. + +Since all JUnit artifacts declare a +https://docs.gradle.org/current/userguide/platforms.html[platform] dependency on the BOM, +you usually don't need to declare an explicit dependency on it yourself. Instead, it's +sufficient to declare _one_ regular dependency that includes a version number. Gradle will +then pull in the BOM automatically so you can omit the version for all other JUnit +artifacts. + +[source,groovy,indent=0] +[subs=attributes+] +.Implicit platform dependency on the BOM +---- +dependencies { + testImplementation("org.junit.jupiter:junit-jupiter:{version}") // <1> + testRuntimeOnly("org.junit.platform:junit-platform-launcher") // <2> +} +---- +<1> Dependency declaration with explicit version. Pulls in the `junit-bom` automatically. +<2> Dependency declaration without version. The version is supplied by the `junit-bom`. + +[WARNING] +.Declaring a dependency on junit-platform-launcher +==== +Even though pre-8.0 versions of Gradle don't require declaring an explicit +dependency on `junit-platform-launcher`, it is recommended to do so to ensure the versions +of JUnit artifacts on the test runtime classpath are aligned. +Moreover, doing so is recommended and in some cases even required when importing the +project into an IDE like <> or +<>. +==== + +[[running-tests-build-gradle-engines-configure]] +===== Configuring Test Engines + +In order to run any tests at all, a `TestEngine` implementation must be on the classpath. + +To configure support for JUnit Jupiter based tests, configure a `testImplementation` dependency +on the dependency-aggregating JUnit Jupiter artifact similar to the following. + +[source,groovy,indent=0] +[subs=attributes+] +---- +dependencies { + testImplementation("org.junit.jupiter:junit-jupiter:{version}") + testRuntimeOnly("org.junit.platform:junit-platform-launcher") +} +---- + +Alternatively, you can use Gradle's +https://docs.gradle.org/current/userguide/jvm_test_suite_plugin.html[JVM Test Suite] +support. + +[source,kotlin,indent=0] +[subs=attributes+] +.Kotlin DSL +---- +testing { + suites { + named("test") { + useJUnitJupiter("{version}") + } + } +} +---- + +[source,groovy,indent=0] +[subs=attributes+] +.Groovy DSL +---- +testing { + suites { + test { + useJUnitJupiter("{version}") + } + } +} +---- + +The JUnit Platform can run JUnit 4 based tests as long as you configure a `testImplementation` +dependency on JUnit 4 and a `testRuntimeOnly` dependency on the JUnit Vintage `TestEngine` +implementation similar to the following. + +[source,groovy,indent=0] +[subs=attributes+] +---- +dependencies { + testImplementation("junit:junit:{junit4-version}") + testRuntimeOnly("org.junit.vintage:junit-vintage-engine:{version}") + testRuntimeOnly("org.junit.platform:junit-platform-launcher") +} +---- + +[[running-tests-build-gradle-config-params]] +===== Configuration Parameters + +The standard Gradle `test` task currently does not provide a dedicated DSL to set JUnit +Platform <> to influence test +discovery and execution. However, you can provide configuration parameters within the +build script via system properties (as shown below) or via the +`junit-platform.properties` file. + +[source,groovy,indent=0] +---- +test { + // ... + systemProperty("junit.jupiter.conditions.deactivate", "*") + systemProperty("junit.jupiter.extensions.autodetection.enabled", true) + systemProperty("junit.jupiter.testinstance.lifecycle.default", "per_class") + // ... +} +---- + +[[running-tests-build-gradle-logging]] +===== Configuring Logging (optional) + +JUnit uses the Java Logging APIs in the `java.util.logging` package (a.k.a. _JUL_) to +emit warnings and debug information. Please refer to the official documentation of +`{LogManager}` for configuration options. + +Alternatively, it's possible to redirect log messages to other logging frameworks such as +{Log4j} or {Logback}. To use a logging framework that provides a custom implementation of +`{LogManager}`, set the `java.util.logging.manager` system property to the _fully +qualified class name_ of the `{LogManager}` implementation to use. The example below +demonstrates how to configure Log4j{nbsp}2.x (see {Log4j_JDK_Logging_Adapter} for +details). + +[source,groovy,indent=0] +[subs=attributes+] +---- +test { + systemProperty("java.util.logging.manager", "org.apache.logging.log4j.jul.LogManager") + // Avoid overhead (see https://logging.apache.org/log4j/2.x/manual/jmx.html#enabling-jmx) + systemProperty("log4j2.disableJmx", "true") +} +---- + +Other logging frameworks provide different means to redirect messages logged using +`java.util.logging`. For example, for {Logback} you can use the +https://www.slf4j.org/legacy.html#jul-to-slf4j[JUL to SLF4J Bridge] by adding it as a +dependency to the test runtime classpath. + +[[running-tests-build-maven]] +==== Maven + +Maven Surefire and Maven Failsafe provide +https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit-platform.html[native support] +for executing tests on the JUnit Platform. The `pom.xml` file in the +`{junit-jupiter-starter-maven}` project demonstrates how to use the Maven Surefire plugin +and can serve as a starting point for configuring your Maven build. + +[WARNING] +.Minimum required version of Maven Surefire/Failsafe +==== +As of JUnit 6.0, the minimum required version of Maven Surefire/Failsafe is 3.0.0. +==== + +[[running-tests-build-maven-bom]] +===== Aligning dependency versions + +Unless you're using Spring Boot which defines its own way of managing dependencies, it is +recommended to use the JUnit Platform <> to align the +versions of all JUnit artifacts. + +[source,xml,indent=0] +[subs=attributes+] +---- + + + + org.junit + junit-bom + {version} + pom + import + + + +---- + +Using the BOM allows you to omit the version when declaring dependencies on all artifacts +with the `org.junit.platform`, `org.junit.jupiter`, and `org.junit.vintage` group IDs. + +TIP: See <> for details on how to override the version +of JUnit used in your Spring Boot application. + +[[running-tests-build-maven-engines-configure]] +===== Configuring Test Engines + +In order to have Maven Surefire or Maven Failsafe run any tests at all, at least one +`TestEngine` implementation must be added to the test classpath. + +To configure support for JUnit Jupiter based tests, configure `test` scoped dependencies +on the JUnit Jupiter API and the JUnit Jupiter `TestEngine` implementation similar to the +following. + +[source,xml,indent=0] +[subs=attributes+] +---- + + + + + org.junit.jupiter + junit-jupiter + {version} + test + + + + + + + maven-surefire-plugin + {surefire-version} + + + maven-failsafe-plugin + {surefire-version} + + + + +---- + +Maven Surefire and Maven Failsafe can run JUnit 4 based tests alongside Jupiter tests as +long as you configure `test` scoped dependencies on JUnit 4 and the JUnit Vintage +`TestEngine` implementation similar to the following. + +[source,xml,indent=0] +[subs=attributes+] +---- + + + + + junit + junit + {junit4-version} + test + + + org.junit.vintage + junit-vintage-engine + {version} + test + + + + + + + + maven-surefire-plugin + {surefire-version} + + + maven-failsafe-plugin + {surefire-version} + + + + +---- + +[[running-tests-build-maven-filter-test-class-names]] +===== Filtering by Test Class Names + +The Maven Surefire Plugin will scan for test classes whose fully qualified names match +the following patterns. + +- `+++**/Test*.java+++` +- `+++**/*Test.java+++` +- `+++**/*Tests.java+++` +- `+++**/*TestCase.java+++` + +Moreover, it will exclude all nested classes (including static member classes) by default. + +Note, however, that you can override this default behavior by configuring explicit +`include` and `exclude` rules in your `pom.xml` file. For example, to keep Maven Surefire +from excluding static member classes, you can override its exclude rules as follows. + +[source,xml,indent=0] +[subs=attributes+] +.Overriding exclude rules of Maven Surefire +---- + + + + + maven-surefire-plugin + {surefire-version} + + + + + + + + + +---- + +Please see the +https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html[Inclusions and Exclusions of Tests] +documentation for Maven Surefire for details. + +[[running-tests-build-maven-filter-tags]] +===== Filtering by Tags + +You can filter tests by <> or +<> using the following configuration +properties. + +- to include _tags_ or _tag expressions_, use `groups`. +- to exclude _tags_ or _tag expressions_, use `excludedGroups`. + +[source,xml,indent=0] +[subs=attributes+] +---- + + + + + maven-surefire-plugin + {surefire-version} + + acceptance | !feature-a + integration, regression + + + + + +---- + +[[running-tests-build-maven-config-params]] +===== Configuration Parameters + +You can set JUnit Platform <> to +influence test discovery and execution by declaring the `configurationParameters` +property and providing key-value pairs using the Java `Properties` file syntax (as shown +below) or via the `junit-platform.properties` file. + +[source,xml,indent=0] +[subs=attributes+] +---- + + + + + maven-surefire-plugin + {surefire-version} + + + + junit.jupiter.conditions.deactivate = * + junit.jupiter.extensions.autodetection.enabled = true + junit.jupiter.testinstance.lifecycle.default = per_class + + + + + + + +---- + +[[running-tests-build-ant]] +==== Ant + +Starting with version `1.10.3`, link:https://ant.apache.org/[Ant] has a +link:https://ant.apache.org/manual/Tasks/junitlauncher.html[`junitlauncher`] task that +provides native support for launching tests on the JUnit Platform. The `junitlauncher` +task is solely responsible for launching the JUnit Platform and passing it the selected +collection of tests. The JUnit Platform then delegates to registered test engines to +discover and execute the tests. + +The `junitlauncher` task attempts to align as closely as possible with native Ant +constructs such as +link:https://ant.apache.org/manual/Types/resources.html#collection[resource collections] +for allowing users to select the tests that they want executed by test engines. This gives +the task a consistent and natural feel when compared to many other core Ant tasks. + +Starting with version `1.10.6` of Ant, the `junitlauncher` task supports +link:https://ant.apache.org/manual/Tasks/junitlauncher.html#fork[forking the tests in a separate JVM]. + +The `build.xml` file in the `{junit-jupiter-starter-ant}` project demonstrates how to use +the task and can serve as a starting point. + +===== Basic Usage + +The following example demonstrates how to configure the `junitlauncher` task to select a +single test class (i.e., `com.example.project.CalculatorTests`). + +[source,xml,indent=0] +---- + + + + + + + + + +---- + +The `test` element allows you to specify a single test class that you want to be selected +and executed. The `classpath` element allows you to specify the classpath to be used to +launch the JUnit Platform. This classpath will also be used to locate test classes that +are part of the execution. + +The following example demonstrates how to configure the `junitlauncher` task to select +test classes from multiple locations. + +[source,xml,indent=0] +---- + + + + + + + + + + + + + + + + +---- + +In the above example, the `testclasses` element allows you to select multiple test +classes that reside in different locations. + +For further details on usage and configuration options please refer to the official Ant +documentation for the +link:https://ant.apache.org/manual/Tasks/junitlauncher.html[`junitlauncher` task]. + +[[running-tests-build-spring-boot]] +==== Spring Boot + +link:https://spring.io/projects/spring-boot[Spring Boot] provides automatic support for +managing the version of JUnit used in your project. In addition, the +`spring-boot-starter-test` artifact automatically includes testing libraries such as JUnit +Jupiter, AssertJ, Mockito, etc. + +If your build relies on dependency management support from Spring Boot, you should not +import JUnit's <> in your build script since that would +result in duplicate (and potentially conflicting) management of JUnit dependencies. + +If you need to override the version of a dependency used in your Spring Boot application, +you have to override the exact name of the +link:https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.dependency-versions.properties[version property] +defined in the BOM used by the Spring Boot plugin. For example, the name of the JUnit +Jupiter version property in Spring Boot is `junit-jupiter.version`. The mechanism for +changing a dependency version is documented for both +link:https://docs.spring.io/spring-boot/docs/current/gradle-plugin/reference/htmlsingle/#managing-dependencies.dependency-management-plugin.customizing[Gradle] +and +link:https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/#using.parent-pom[Maven]. + +With Gradle you can override the JUnit Jupiter version by including the following in your +`build.gradle` file. + +[source,groovy,indent=0] +[subs=attributes+] +---- + ext['junit-jupiter.version'] = '{version}' +---- + +With Maven you can override the JUnit Jupiter version by including the following in your +`pom.xml` file. + +[source,xml,indent=0] +[subs=attributes+] +---- + + {version} + +---- For a complete working example of how to set up JUnit 5 with Bazel, see the https://github.com/junit-team/junit5-samples/tree/main/junit5-jupiter-starter-bazel[JUnit 5 + Bazel Starter Sample] project. [[running-tests-console-launcher]] +=== Console Launcher + +The `{ConsoleLauncher}` is a command-line Java application that lets you launch the JUnit +Platform from the console. For example, it can be used to run JUnit Vintage and JUnit +Jupiter tests and print test execution results to the console. + +An executable _Fat JAR_ (`junit-platform-console-standalone-{version}.jar`) that +contains the contents of all of its dependencies is published in the {Maven_Central} +repository under the +https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone[junit-platform-console-standalone] +directory. It contains the contents of the following artifacts: + +include::{standaloneConsoleLauncherShadowedArtifactsFile}[] + +[NOTE] +==== +Since the `junit-platform-console-standalone` JAR contains the contents of all of its +dependencies, its Maven POM does not declare any dependencies. +Furthermore, it is not very likely that you would need to include a dependency on the +`junit-platform-console-standalone` artifact in your project's Maven POM or Gradle build +script. On the contrary, the executable `junit-platform-console-standalone` JAR is +typically invoked directly from the command line or a shell script without a build script. +If you need to declare dependencies in your build script on some of the artifacts +contained in the `junit-platform-console-standalone` artifact, you should declare +dependencies only on the JUnit artifacts that are used in your project. To simplify +dependency management of JUnit artifacts in your build, you may wish to use the +`junit-jupiter` aggregator artifact or `junit-bom`. See <> for +details. +==== + +You can https://docs.oracle.com/javase/tutorial/deployment/jar/run.html[run] the +standalone `ConsoleLauncher` as shown below. + +[source,console,subs=attributes+] +---- +$ java -jar junit-platform-console-standalone-{version}.jar execute +├─ JUnit Vintage +│ └─ example.JUnit4Tests +│ └─ standardJUnit4Test ✔ +└─ JUnit Jupiter + ├─ StandardTests + │ ├─ succeedingTest() ✔ + │ └─ skippedTest() ↷ for demonstration purposes + └─ A special test case + ├─ Custom test name containing spaces ✔ + ├─ ╯°□°)╯ ✔ + └─ 😱 ✔ +Test run finished after 64 ms +[ 5 containers found ] +[ 0 containers skipped ] +[ 5 containers started ] +[ 0 containers aborted ] +[ 5 containers successful ] +[ 0 containers failed ] +[ 6 tests found ] +[ 1 tests skipped ] +[ 5 tests started ] +[ 0 tests aborted ] +[ 5 tests successful ] +[ 0 tests failed ] +---- + +You can also run the standalone `ConsoleLauncher` as shown below (for example, to include +all jars in a directory): + +[source,console,subs=attributes+] +---- +$ java -cp classes:testlib/* org.junit.platform.console.ConsoleLauncher +---- + +[[running-tests-console-launcher-options]] +==== Subcommands and Options + +The `{ConsoleLauncher}` provides the following subcommands: + +---- +include::{consoleLauncherOptionsFile}[] +---- + +[[running-tests-console-launcher-options-discovering-tests]] +===== Discovering tests + +---- +include::{consoleLauncherDiscoverOptionsFile}[] +---- + +[[running-tests-console-launcher-options-executing-tests]] +===== Executing tests + +.Exit Code +NOTE: The `{ConsoleLauncher}` exits with a status code of `1` if any containers or tests +failed. If no tests are discovered and the `--fail-if-no-tests` command-line option is +supplied, the `ConsoleLauncher` exits with a status code of `2`. Otherwise, the exit code +is `0`. + +---- +include::{consoleLauncherExecuteOptionsFile}[] +---- + +[[running-tests-console-launcher-options-listing-test-engines]] +===== Listing test engines + +---- +include::{consoleLauncherEnginesOptionsFile}[] +---- + +[[running-tests-console-launcher-argument-files]] +==== Argument Files (@-files) + +On some platforms you may run into system limitations on the length of a command line when +creating a command line with lots of options or with long arguments. + +The `ConsoleLauncher` supports _argument files_, also known as _@-files_. Argument files +are files that themselves contain arguments to be passed to the command. When the +underlying https://github.com/remkop/picocli[picocli] command line parser encounters an +argument beginning with the character `@`, it expands the contents of that file into the +argument list. + +The arguments within a file can be separated by spaces or newlines. If an argument +contains embedded whitespace, the whole argument should be wrapped in double or single +quotes -- for example, `"-f=My Files/Stuff.java"`. + +If the argument file does not exist or cannot be read, the argument will be treated +literally and will not be removed. This will likely result in an "unmatched argument" +error message. You can troubleshoot such errors by executing the command with the +`picocli.trace` system property set to `DEBUG`. + +Multiple _@-files_ may be specified on the command line. The specified path may be +relative to the current directory or absolute. + +You can pass a real parameter with an initial `@` character by escaping it with an +additional `@` symbol. For example, `@@somearg` will become `@somearg` and will not be +subject to expansion. + +[[running-tests-console-launcher-redirecting-stdout-and-stderr]] +==== Redirecting Standard Output/Error to Files + +You can redirect the `System.out` (stdout) and `System.err` (stderr) output streams to +files using the `--redirect-stdout` and `--redirect-stderr` options: + +[source,console,subs=attributes+] +---- +$ java -jar junit-platform-console-standalone-{version}.jar \ + --redirect-stdout=stdout.txt \ + --redirect-stderr=stderr.txt +---- + +[NOTE] +==== +If the `--redirect-stdout` and `--redirect-stderr` arguments point to the same file, both +output streams will be redirected to that file. +The default charset is used for writing to the files. +==== + +[[running-tests-console-launcher-color-customization]] +==== Color Customization + +The colors used in the output of the `{ConsoleLauncher}` can be customized. +The option `--single-color` will apply a built-in monochrome style, while +`--color-palette` will accept a properties file to override the +https://en.wikipedia.org/wiki/ANSI_escape_code#Colors[ANSI SGR] color styling. +The properties file below demonstrates the default style: + +[source,properties,indent=0] +---- +SUCCESSFUL = 32 +ABORTED = 33 +FAILED = 31 +SKIPPED = 35 +CONTAINER = 35 +TEST = 34 +DYNAMIC = 35 +REPORTED = 37 +---- + +[[running-tests-discovery-selectors]] +=== Discovery Selectors + +The JUnit Platform provides a rich set of discovery selectors that can be used to specify +which tests should be discovered or executed. + +Discovery selectors can be created programmatically using the factory methods in the +`{DiscoverySelectors}` class, specified declaratively via annotations when using the +<>, via options of the <>, or +generically as strings via their identifiers. + +The following discovery selectors are provided out of the box: + +|=== +| Java Type | API | Annotation | Console Launcher | Identifier + +| `{ClasspathResourceSelector}` | `{DiscoverySelectors_selectClasspathResource}` | `{SelectClasspathResource}` | `--select-resource /foo.csv` | `resource:/foo.csv` +| `{ClasspathRootSelector}` | `{DiscoverySelectors_selectClasspathRoots}` | -- | `--scan-classpath bin` | `classpath-root:bin` +| `{ClassSelector}` | `{DiscoverySelectors_selectClass}` | `{SelectClasses}` | `--select-class com.acme.Foo` | `class:com.acme.Foo` +| `{DirectorySelector}` | `{DiscoverySelectors_selectDirectory}` | `{SelectDirectories}` | `--select-directory foo/bar` | `directory:foo/bar` +| `{FileSelector}` | `{DiscoverySelectors_selectFile}` | `{SelectFile}` | `--select-file dir/foo.txt` | `file:dir/foo.txt` +| `{IterationSelector}` | `{DiscoverySelectors_selectIteration}` | `{Select}("")` | `--select-iteration method=com.acme.Foo#m[1..2]` | `iteration:method:com.acme.Foo#m[1..2]` +| `{MethodSelector}` | `{DiscoverySelectors_selectMethod}` | `{SelectMethod}` | `--select-method com.acme.Foo#m` | `method:com.acme.Foo#m` +| `{ModuleSelector}` | `{DiscoverySelectors_selectModule}` | `{SelectModules}` | `--select-module com.acme` | `module:com.acme` +| `{NestedClassSelector}` | `{DiscoverySelectors_selectNestedClass}` | `{Select}("")` | `--select ` | `nested-class:com.acme.Foo/Bar` +| `{NestedMethodSelector}` | `{DiscoverySelectors_selectNestedMethod}` | `{Select}("")` | `--select ` | `nested-method:com.acme.Foo/Bar#m` +| `{PackageSelector}` | `{DiscoverySelectors_selectPackage}` | `{SelectPackages}` | `--select-package com.acme.foo` | `package:com.acme.foo` +| `{UniqueIdSelector}` | `{DiscoverySelectors_selectUniqueId}` | `{Select}("")` | `--select-unique-id ` | `uid:[engine:Foo]/[segment:Bar]` +| `{UriSelector}` | `{DiscoverySelectors_selectUri}` | `{SelectUris}` | `--select-uri \file:///foo.txt` | `uri:file:///foo.txt` +|=== + + +[[running-tests-config-params]] +=== Configuration Parameters + +In addition to instructing the platform which test classes and test engines to include, +which packages to scan, etc., it is sometimes necessary to provide additional custom +configuration parameters that are specific to a particular test engine, listener, or +registered extension. For example, the JUnit Jupiter `TestEngine` supports _configuration +parameters_ for the following use cases. + +- <> +- <> +- <> +- <> + +_Configuration Parameters_ are text-based key-value pairs that can be supplied to test +engines running on the JUnit Platform via one of the following mechanisms. + +1. The `configurationParameter()` and `configurationParameters()` methods in + `LauncherDiscoveryRequestBuilder` which + is used to build a request supplied to the <>. + + + When running tests via one of the tools provided by the JUnit Platform you can specify + configuration parameters as follows: + * <>: use the `--config` command-line + option. + * <>: use the `systemProperty` or + `systemProperties` DSL. + * <>: use the + `configurationParameters` property. +2. The `configurationParametersResources()` method in `LauncherDiscoveryRequestBuilder`. + + + When running tests via the <> you can + specify custom configuration files using the `--config-resource` command-line option. +3. JVM system properties. +4. The JUnit Platform default configuration file: a file named `junit-platform.properties` + in the root of the class path that follows the syntax rules for Java `Properties` + files. + +NOTE: Configuration parameters are looked up in the exact order defined above. +Consequently, configuration parameters supplied directly to the `Launcher` take +precedence over those supplied via custom configuration files, system properties, and the +default configuration file. Similarly, configuration parameters supplied via system +properties take precedence over those supplied via the default configuration file. + +[[running-tests-config-params-deactivation-pattern]] +==== Pattern Matching Syntax + +This section describes the pattern matching syntax that is applied to the _configuration +parameters_ used for the following features. + +- <> +- <> +- <> +- <> + +If the value for the given _configuration parameter_ consists solely of an asterisk +(`+++*+++`), the pattern will match against all candidate classes. Otherwise, the value +will be treated as a comma-separated list of patterns where each pattern will be matched +against the fully qualified class name (_FQCN_) of each candidate class. Any dot (`.`) in +a pattern will match against a dot (`.`) or a dollar sign (`$`) in a FQCN. Any asterisk +(`+++*+++`) will match against one or more characters in a FQCN. All other characters in a +pattern will be matched one-to-one against a FQCN. + +Examples: + +- `+++*+++`: matches all candidate classes. +- `+++org.junit.*+++`: matches all candidate classes under the `org.junit` base package and + any of its subpackages. +- `+++*.MyCustomImpl+++`: matches every candidate class whose simple class name is exactly + `MyCustomImpl`. +- `+++*System*+++`: matches every candidate class whose FQCN contains `System`. +- `+++*System*+++, +++*Unit*+++`: matches every candidate class whose FQCN contains + `System` or `Unit`. +- `org.example.MyCustomImpl`: matches the candidate class whose FQCN is exactly + `org.example.MyCustomImpl`. +- `org.example.MyCustomImpl, org.example.TheirCustomImpl`: matches candidate classes whose + FQCN is exactly `org.example.MyCustomImpl` or `org.example.TheirCustomImpl`. + +[[running-tests-tags]] +=== Tags + +Tags are a JUnit Platform concept for marking and filtering tests. The programming model +for adding tags to containers and tests is defined by the testing framework. For example, +in JUnit Jupiter based tests, the `@Tag` annotation (see +<>) should be used. For JUnit 4 based tests, the +Vintage engine maps `@Category` annotations to tags (see +<>). Other testing frameworks may define their +own annotation or other means for users to specify tags. + +[[running-tests-tag-syntax-rules]] +==== Syntax Rules for Tags + +Regardless how a tag is specified, the JUnit Platform enforces the following rules: + +* A tag must not be `null` or _blank_. +* A _stripped_ tag must not contain whitespace. +* A _stripped_ tag must not contain ISO control characters. +* A _stripped_ tag must not contain any of the following _reserved characters_. +- `,`: _comma_ +- `(`: _left parenthesis_ +- `)`: _right parenthesis_ +- `&`: _ampersand_ +- `|`: _vertical bar_ +- `!`: _exclamation point_ + +NOTE: In the above context, "stripped" means that leading and trailing whitespace +characters have been removed using `java.lang.String.strip()`. + +[[running-tests-tag-expressions]] +==== Tag Expressions + +Tag expressions are boolean expressions with the operators `!`, `&` and `|`. In addition, +`(` and `)` can be used to adjust for operator precedence. + +Two special expressions are supported, `any()` and `none()`, which select all tests _with_ +any tags at all, and all tests _without_ any tags, respectively. +These special expressions may be combined with other expressions just like normal tags. + +.Operators (in descending order of precedence) +|=== +| Operator | Meaning | Associativity + +| `!` | not | right +| `&` | and | left +| `\|` | or | left +|=== + +If you are tagging your tests across multiple dimensions, tag expressions help you to +select which tests to execute. When tagging by test type (e.g., _micro_, _integration_, +_end-to-end_) and feature (e.g., *product*, *catalog*, *shipping*), the following tag +expressions can be useful. + +[%header,cols="40,60"] +|=== +| Tag Expression +| Selection + +| `+++product+++` +| all tests for *product* + +| `+++catalog \| shipping+++` +| all tests for *catalog* plus all tests for *shipping* + +| `+++catalog & shipping+++` +| all tests for the intersection between *catalog* and *shipping* + +| `+++product & !end-to-end+++` +| all tests for *product*, but not the _end-to-end_ tests + +| `+++(micro \| integration) & (product \| shipping)+++` +| all _micro_ or _integration_ tests for *product* or *shipping* +|=== + +[[running-tests-capturing-output]] +=== Capturing Standard Output/Error + +The JUnit Platform provides opt-in support for capturing output printed to `System.out` +and `System.err`. To enable it, set the `junit.platform.output.capture.stdout` and/or +`junit.platform.output.capture.stderr` <> to `true`. In addition, you may configure the maximum number of buffered bytes +to be used per executed test or container using `junit.platform.output.capture.maxBuffer`. + +If enabled, the JUnit Platform captures the corresponding output and publishes it as a +report entry using the `stdout` or `stderr` keys to all registered +`{TestExecutionListener}` instances immediately before reporting the test or container as +finished. + +Please note that the captured output will only contain output emitted by the thread that +was used to execute a container or test. Any output by other threads will be omitted +because particularly when +<> it would be impossible +to attribute it to a specific test or container. + +[[running-tests-listeners]] +=== Using Listeners and Interceptors + +The JUnit Platform provides the following listener APIs that allow JUnit, third parties, +and custom user code to react to events fired at various points during the discovery and +execution of a `TestPlan`. + +* `{LauncherSessionListener}`: receives events when a `{LauncherSession}` is opened and + closed. +* `{LauncherInterceptor}`: intercepts test discovery and execution in the context of a + `LauncherSession`. +* `{LauncherDiscoveryListener}`: receives events that occur during test discovery. +* `{TestExecutionListener}`: receives events that occur during test execution. + +The `LauncherSessionListener` API is typically implemented by build tools or IDEs and +registered automatically for you in order to support some feature of the build tool or IDE. + +The `LauncherDiscoveryListener` and `TestExecutionListener` APIs are often implemented in +order to produce some form of report or to display a graphical representation of the test +plan in an IDE. Such listeners may be implemented and automatically registered by a build +tool or IDE, or they may be included in a third-party library – potentially registered +for you automatically. You can also implement and register your own listeners. + +For details on registering and configuring listeners, see the following sections of this +guide. + +* <> +* <> +* <> +* <> +* <> +* <> + +The JUnit Platform provides the following listeners which you may wish to use with your +test suite. + +<> :: + `{LegacyXmlReportGeneratingListener}` can be used via the + <> or registered manually to generate XML reports + compatible with the de facto standard for JUnit 4 based test reports. ++ +`{OpenTestReportGeneratingListener}` generates an XML report in the event-based format +specified by {OpenTestReporting}. It is auto-registered and can be enabled and +configured via <>. ++ +See <> for details. + +<> :: + `FlightRecordingExecutionListener` and `FlightRecordingDiscoveryListener` that generate + Java Flight Recorder events during test discovery and execution. + +`{LoggingListener}` :: + `TestExecutionListener` for logging informational messages for all events via a + `BiConsumer` that consumes `Throwable` and `Supplier`. + +`{SummaryGeneratingListener}` :: + `TestExecutionListener` that generates a summary of the test execution which can be + printed via a `PrintWriter`. + +`{UniqueIdTrackingListener}` :: + `TestExecutionListener` that that tracks the unique IDs of all tests that were skipped + or executed during the execution of the `TestPlan` and generates a file containing the + unique IDs once execution of the `TestPlan` has finished. + +[[running-tests-listeners-flight-recorder]] +==== Flight Recorder Support + +The JUnit Platform provides opt-in support for generating Flight Recorder events. +https://openjdk.java.net/jeps/328[JEP 328] describes the Java Flight Recorder (JFR) as +follows. + +> Flight Recorder records events originating from applications, the JVM, and the OS. +Events are stored in a single file that can be attached to bug reports and examined by +support engineers, allowing after-the-fact analysis of issues in the period leading up +to a problem. + +In order to record Flight Recorder events generated while running tests, you need to +start flight recording when launching a test suite via the following java command line +option. + + -XX:StartFlightRecording:filename=... + +Please consult the manual of your build tool for the appropriate commands. + +To analyze the recorded events, use the +https://docs.oracle.com/en/java/javase/14/docs/specs/man/jfr.html[jfr] +command line tool shipped with recent JDKs or open the recording file with +https://jdk.java.net/jmc/[JDK Mission Control]. + +[[stacktrace-pruning]] +=== Stack Trace Pruning + +The JUnit Platform provides built-in support for pruning stack traces produced by failing +tests. This feature is enabled by default but can be disabled by setting the +`junit.platform.stacktrace.pruning.enabled` _configuration parameter_ to `false`. + +When enabled, all calls from the `org.junit`, `jdk.internal.reflect`, and `sun.reflect` +packages are removed from the stack trace, unless the calls occur after the test itself +or any of its ancestors. For that reason, calls to `{Assertions}` or `{Assumptions}` will +never be excluded. + +In addition, all elements prior to and including the first call from the JUnit Platform +`Launcher` will be removed. + +[[running-tests-discovery-issues]] +=== Discovery Issues + +Test engines may encounter issues during test discovery. For example, the declaration of a +test class or method may be invalid. To avoid such issues from going unnoticed, the JUnit +Platform provides a <> to +report them with different severity levels: + +INFO:: +Indicates that the engine encountered something that could be potentially problematic, but +could also happen due to a valid setup or configuration. + +WARNING:: +Indicates that the engine encountered something that is problematic and might lead to +unexpected behavior or will be removed or changed in a future release. + +ERROR:: +Indicates that the engine encountered something that is definitely problematic and will +lead to unexpected behavior. + +If an engine reports an issue with a severity equal to or higher than a configurable +_critical_ severity, its tests will not be executed. Instead, the engine will be reported +as failed during execution with a `{DiscoveryIssueException}` listing all critical issues. +Non-critical issues will be logged but will not prevent the engine from executing its +tests. The `junit.platform.discovery.issue.severity.critical` +<> can be used to set the critical +severity level. Currently, the default value is `ERROR` but it may be changed in a future +release. + +TIP: To surface all discovery issues in your project, it is recommended to set the +`junit.platform.discovery.issue.severity.critical` configuration parameter to `INFO`. + +In addition, registered `{LauncherDiscoveryListener}` implementations can receive +discovery issues via the `issueEncountered()` method. This allows IDEs and build tools to +report issues to the user in a more user-friendly way. For example, IDEs may choose to +display all issues in a list or table. From 9430789766803dfb93ee109db97eaea2d229b617 Mon Sep 17 00:00:00 2001 From: Nikhil Gupta <108018378+NikhilGupta-Dev@users.noreply.github.com> Date: Thu, 28 Aug 2025 20:03:22 +0530 Subject: [PATCH 3/3] Update running-tests.adoc