Skip to content

Commit c590031

Browse files
Merge pull request #9780 from circleci/adaptive-testing-smarter-testing
Update and rename adaptive-testing.adoc to smarter-testing.adoc
2 parents 1719767 + fdd4f90 commit c590031

File tree

4 files changed

+43
-87
lines changed

4 files changed

+43
-87
lines changed

antora-playbook.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ antora:
6969
extensions:
7070
- require: ./extensions/unlisted-pages-extension.js
7171
allowedUnlistedPages:
72-
- 'test:adaptive-testing.adoc'
72+
- 'test:smarter-testing.adoc'
7373
- 'test:fix-flaky-tests.adoc'
7474
- require: '@sntke/antora-mermaid-extension'
7575
mermaid_initialize_options:

docs/guides/modules/test/pages/adaptive-testing.adoc renamed to docs/guides/modules/test/pages/smarter-testing.adoc

Lines changed: 41 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,37 @@
1-
= Adaptive testing
1+
= Smarter Testing
22
:page-badge: Preview
33
:page-platform: Cloud
4-
:page-description: This document describes the adaptive testing feature in CircleCI, which enables only running tests that are impacted by code changes and evenly distributes tests across parallel execution nodes.
4+
:page-description: This page describes CircleCI's Smarter Testing. Only run tests that are impacted by code changes and evenly distribute tests across parallel execution nodes.
55
:experimental:
66
:page-noindex: true
7+
:page-aliases: adaptive-testing.adoc
78

8-
CAUTION: *Adaptive testing* is available in closed preview. When the feature is made generally available there will be a cost associated with access and usage.
9+
CAUTION: *Smarter Testing* is available in closed preview. When the feature is made generally available there will be a cost associated with access and usage.
910

1011
NOTE: This page is currently in development and will be updated as the feature is developed.
1112

12-
Use adaptive testing to optimize test runs as follows:
13+
Use Smarter Testing to optimize test runs as follows:
1314

1415
* Run only tests that are impacted by code changes.
1516
* Evenly distribute tests across parallel execution nodes.
1617
17-
Adaptive testing reduces test execution time while maintaining test confidence.
18+
Smarter Testing reduces test execution time while maintaining test confidence.
1819

19-
== Is my project a good fit for adaptive testing?
20+
== Is my project a good fit for Smarter Testing?
2021

21-
The following list shows some examples of where adaptive testing can be most beneficial:
22+
The following list shows some examples of where Smarter Testing can be most beneficial:
2223

2324
* Tests that exercise code within the same repository.
24-
* Projects with comprehensive test coverage. The more thorough your tests, the more precisely adaptive testing can identify which tests are impacted by changes.
25+
* Projects with comprehensive test coverage. The more thorough your tests, the more precisely Smarter Testing can identify which tests are impacted by changes.
2526
* Test frameworks with built-in coverage support (Jest, pytest, Go test, Vitest) where generating coverage reports is straightforward.
2627
+
27-
TIP: In codebases with sparse test coverage, adaptive testing cannot accurately determine which tests cover changed code. This causes the system to run more tests, reducing the benefits of intelligent test selection.
28+
TIP: In codebases with sparse test coverage, Smarter Testing cannot accurately determine which tests cover changed code. This causes the system to run more tests, reducing the benefits of intelligent test selection.
2829

2930
== Limitations
3031

31-
* Generating code coverage data is essential for determining how tests are related to code. If tests are run in a way that makes generating and accessing code coverage data tricky then adaptive testing may not be a good fit.
32-
* Adaptive testing works best when testing a single deployable artifact. For example, a monorepo with integration tests that span multiple packages/services, especially when services run in separate container, makes coverage generation and consolidation difficult.
33-
* Adaptive testing needs to be configured with commands to discover all available tests and run a subset of those tests. If you cannot run commands to discover tests and run a subset of tests on the CLI then adaptive testing may not be a good fit.
32+
* Generating code coverage data is essential for determining how tests are related to code. If tests are run in a way that makes generating and accessing code coverage data tricky then Smarter Testing may not be a good fit.
33+
* Smarter Testing works best when testing a single deployable artifact. For example, a monorepo with integration tests that span multiple packages/services, especially when services run in separate container, makes coverage generation and consolidation difficult.
34+
* Smarter Testing needs to be configured with commands to discover all available tests and run a subset of those tests. If you cannot run commands to discover tests and run a subset of tests on the CLI then Smarter Testing may not be a good fit.
3435

3536
== Key benefits
3637

@@ -40,17 +41,17 @@ TIP: In codebases with sparse test coverage, adaptive testing cannot accurately
4041
* Scale efficiently as test suites grow.
4142

4243
== How it works
43-
Adaptive testing operates through two main components that work together to optimize your test execution:
44+
Smarter Testing operates through two main components that work together to optimize your test execution:
4445

45-
* Dynamic test splitting
46-
* Test impact analysis
46+
* Dynamic test splitting.
47+
* Test impact analysis.
4748

4849
Each component is described in more detail below.
4950

5051
=== Dynamic test splitting
5152
Dynamic test splitting distributes your tests across parallel execution nodes. The system maintains a shared queue that each node pulls from to create a balanced workload.
5253

53-
When you configure parallelism in your job, adaptive testing automatically:
54+
When you configure parallelism in your job, Smarter Testing automatically:
5455

5556
* Retrieves timing data from previous test runs.
5657
* Calculates optimal test distribution across your specified number of parallel nodes.
@@ -293,15 +294,15 @@ options:
293294

294295
If tests are still slower, share the pipeline link in the closed beta slack channel.
295296

296-
== 2. Enable adaptive testing
297+
== 2. Enable Smarter Testing
297298

298-
We recommend following the steps in <<getting-started>> first before enabling the adaptive testing feature to ensure the `discover` and `run` commands are set up correctly.
299+
We recommend following the steps in <<getting-started>> first before enabling the Smarter Testing feature to ensure the `discover` and `run` commands are set up correctly.
299300

300-
The goal of this section is to enable adaptive testing for your test suite.
301+
The goal of this section is to enable Smarter Testing for your test suite.
301302

302303
=== 2.1 Update the test suites file
303304

304-
When using adaptive testing for test impact analysis the following commands are used:
305+
When using Smarter Testing for test impact analysis the following commands are used:
305306

306307
* The `discover` command discovers all tests in a test suite.
307308
* The `run` command runs only impacted tests and a new command.
@@ -518,7 +519,7 @@ The following options are available to be defined in the options map in config:
518519

519520
| `adaptive-testing`
520521
|false
521-
|Enables the adaptive testing features, such as test impact analysis.
522+
|Enables the Smarter Testing features, such as test impact analysis.
522523

523524
| `full-test-run-paths`
524525
a|
@@ -573,9 +574,12 @@ a| * `all` selects and runs all discovered tests, used to run the full test suit
573574
|===
574575
--
575576

576-
== 3. Start using adaptive testing
577+
== 3. Start using Smarter Testing
577578

578-
Now the test suite is set up, test selection is working and the test analysis is up to date with the latest changes from the feature branch that ran the first test analysis.
579+
Now the test suite is set up:
580+
581+
* Test selection is working.
582+
* The test analysis is up to date with the latest changes from the feature branch that ran the first test analysis.
579583

580584
*Action Items*
581585

@@ -683,7 +687,7 @@ options:
683687

684688
== Limitations
685689

686-
The adaptive testing feature has some limitations to consider:
690+
The Smarter Testing feature has some limitations to consider:
687691

688692
*Initial setup period*:: Test impact analysis requires an initial analysis run on all tests before intelligent selection can begin. This first analysis run will be slower than normal test execution.
689693

@@ -703,9 +707,9 @@ The adaptive testing feature has some limitations to consider:
703707

704708
*Debugging steps:*
705709

706-
. Check that all test files are discovered with the discover command
707-
. Verify parallelism is set correctly in your config.yml
708-
. Look for timing data in previous test runs
710+
. Check that all test files are discovered with the discover command.
711+
. Verify parallelism is set correctly in your config.yml.
712+
. Look for timing data in previous test runs.
709713
. Ensure test results are being stored with `store_test_results`.
710714

711715
=== Test impact analysis not selecting expected tests
@@ -716,10 +720,10 @@ The adaptive testing feature has some limitations to consider:
716720

717721
*Debugging steps:*
718722

719-
. Verify analysis has run successfully on your configured branch(es)
720-
. Check that coverage data is being generated correctly
721-
. Review the full-test-run-paths configuration - changes to these paths trigger full test runs
722-
. Confirm the analysis command is producing valid LCOV output
723+
. Verify analysis has run successfully on your configured branch(es).
724+
. Check that coverage data is being generated correctly.
725+
. Review the full-test-run-paths configuration - changes to these paths trigger full test runs.
726+
. Confirm the analysis command is producing valid LCOV output.
723727

724728
*When all tests run:* If no impact data exists or all tests are determined to be affected, the system runs all tests as a safety measure.
725729

@@ -769,9 +773,9 @@ The frequency depends on your test execution speed and development pace:
769773

770774
*Consider re-running analysis:*
771775

772-
* After major refactoring or code restructuring
773-
* When test selection seems inaccurate or outdated
774-
* After adding significant new code or tests
776+
* After major refactoring or code restructuring.
777+
* When test selection seems inaccurate or outdated.
778+
* After adding significant new code or tests.
775779

776780
*Remember:* You can customize which branches run analysis through your CircleCI configuration - it does not have to be limited to the main branch.
777781

@@ -807,7 +811,7 @@ When test selection determines that no existing tests are affected by your chang
807811

808812
*Best practice:* Include relevant paths in `full-test-run-paths` to explicitly trigger full test runs for infrastructure changes.
809813

810-
=== How do I know if adaptive testing is working?
814+
=== How do I know if Smarter Testing is working?
811815

812816
Look for these indicators in your CircleCI build output:
813817

@@ -846,7 +850,7 @@ See the <<run-higher-parallelism-on-the-analysis-branch,Run higher parallelism o
846850
[#baseline-coverage]
847851
=== Why are there so many files impacting a test?
848852

849-
If you see many files impacting each test during analysis (for example, "...found 150 files impacting test..."), this may be caused by shared setup code like global imports or framework initialization being included in coverage.
853+
If you see many files impacting each test during analysis, for example, `...found 150 files impacting test...`, this may be caused by shared setup code like global imports or framework initialization being included in coverage.
850854

851855
This extraneous coverage can be excluded by providing an `analysis-baseline` command to compute the code covered during startup that isn't directly exercised by test code. We call this "baseline coverage data".
852856

@@ -883,7 +887,7 @@ The `analysis-baseline` command will be run just before running analysis. The co
883887

884888
=== What test frameworks are supported?
885889

886-
Adaptive testing is runner-agnostic. We provide default configurations for the following test frameworks:
890+
Smarter Testing is runner-agnostic. We provide default configurations for the following test frameworks:
887891

888892
* Jest (JavaScript/TypeScript)
889893
* gotestsum (Go)

styles/AsciiDoc/UnsetAttributes.yml

Lines changed: 0 additions & 49 deletions
This file was deleted.

styles/config/vocabularies/Docs/accept.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,7 @@ SLAs?
315315
Slanger
316316
[Ss]ignup
317317
SKUs?
318+
Smarter Testing
318319
[Ss]nap
319320
SSH
320321
statsd

0 commit comments

Comments
 (0)