Skip to content

Commit bfb4ad9

Browse files
authored
Add an example about test splitting (#91)
1 parent 4d98ee3 commit bfb4ad9

File tree

1 file changed

+37
-0
lines changed

1 file changed

+37
-0
lines changed

README.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,42 @@ workflows:
188188
os: [linux, windows, macos]
189189
```
190190

191+
### Split Tests Across Runners
192+
The `run-tests` command is compatible with [CircleCI test splitting](https://circleci.com/docs/parallelism-faster-jobs/), which enables you to split your tests by name, file size, or timing data, and run them across parallel execution environments. To split your MATLAB tests across runners:
193+
194+
1. Specify the number of runners across which to split your tests by using the `parallelism` key.
195+
2. Specify the `select-by-name` parameter of the `run-tests` command using one of the options provided by the CircleCI CLI. For more information about the CLI, see [Use the CircleCI environment CLI to split tests](https://circleci.com/docs/use-the-circleci-cli-to-split-tests/).
196+
197+
For example, suppose that your MATLAB test files are located in a folder named `tests` in the root of your repository as well as in its subfolders. In other words, suppose that the glob pattern `"tests/**/*.m"` represents all your test files. Split the tests by name and run them across four runners.
198+
199+
```YAML
200+
version: 2.1
201+
orbs:
202+
matlab: mathworks/matlab@1
203+
jobs:
204+
run-matlab-tests:
205+
machine:
206+
image: ubuntu-2204:current
207+
parallelism: 4
208+
steps:
209+
- checkout
210+
- matlab/install
211+
- matlab/run-tests:
212+
select-by-name: $(circleci tests glob "tests/**/*.m" | circleci tests split | awk -F'[\\\\/.]' '{print $(NF-1) "/*"}')
213+
workflows:
214+
test:
215+
jobs:
216+
- run-matlab-tests
217+
```
218+
219+
To split the tests specified by the `"tests/**/*.m"` glob pattern using file size, you can specify `select-by-name` like this:
220+
221+
`select-by-name: $(circleci tests glob "tests/**/*.m" | circleci tests split --split-by=filesize | awk -F'[\\\\/.]' '{print $(NF-1) "/*"}')`
222+
223+
Timing-based test splitting relies on timing data collected from a previous test run. To use this strategy, you must produce test results in JUnit-style XML format (by specifying the `test-results-junit` parameter of the `run-tests` command) so that CircleCI can access the historic timing data from the test results. To split the tests specified by the `"tests/**/*.m"` glob pattern using timing data, you can specify `select-by-name` like this:
224+
225+
`select-by-name: $(circleci tests glob "tests/**/*.m" | awk -F'[\\\\/.]' '{print $(NF-1)}' | circleci tests split --split-by=timings | awk '{print $0 "/*"}')`
226+
191227
## Commands
192228
You can access the orb commands in the [CircleCI configuration editor](https://circleci.com/docs/config-editor/).
193229

@@ -238,6 +274,7 @@ Parameter | Description
238274
---------------------------- | ---------------
239275
`source-folder` | <p>(Optional) Location of the folder containing source code, specified as a path relative to the project root folder. The specified folder and its subfolders are added to the top of the MATLAB search path. If you specify `source-folder` and then generate a coverage report, the command uses only the source code in the specified folder and its subfolders to generate the report. You can specify multiple folders using a colon-separated or semicolon-separated list.</p><p>**Example:** `source-folder: source`<br/>**Example:** `source-folder: source/folderA; source/folderB`</p>
240276
`select-by-folder` | <p>(Optional) Location of the folder containing the tests to run, specified as a path relative to the project root folder. If you specify this parameter, the command runs only the tests in the specified folder and its subfolders. You can specify multiple folders using a colon-separated or semicolon-separated list.</p><p>**Example:** `select-by-folder: test`<br/>**Example:** `select-by-folder: test/folderA; test/folderB`</p>
277+
`select-by-name` | <p>(Optional) Names of the tests to run, specified as a list of test names separated by spaces. If you specify this parameter, the command runs only the tests with the specified names. You can use the wildcard character (`*`) to match any number of characters and the question mark character (`?`) to match a single character. For example, specify `select-by-name: ExampleTest/*` to select all the tests whose name starts with `ExampleTest/`.</p><p>For a given test file, the name of a test uniquely identifies the smallest runnable portion of the test content. The test name includes the namespace name, filename (excluding the extension), procedure name, and information about parameterization.</p><p>**Example:** `select-by-name: ExampleTest/testA`<br/>**Example:** `select-by-name: ExampleTest/testA ExampleTest/testB(array=3x3_double)`</p>
241278
`select-by-tag` | <p>(Optional) Tag of the tests to run, specified as a string scalar. If you specify this parameter, the command runs only the tests with the specified tag. For more information about test tags, see [Tag Unit Tests](https://www.mathworks.com/help/matlab/matlab_prog/tag-unit-tests.html).</p><p>**Example:** `select-by-tag: Unit`</p>
242279
`strict` | <p>(Optional) Option to apply strict checks when running tests, specified as `false` or `true`. By default, the value is `false`. If you specify a value of `true`, the command generates a qualification failure whenever a test issues a warning.</p><p>**Example:** `strict: true`</p>
243280
`use-parallel` | <p>(Optional) Option to run tests in parallel, specified as `false` or `true`. By default, the value is `false` and tests run in serial. If the test runner configuration is suited for parallelization, you can specify a value of `true` to run tests in parallel. This parameter requires a Parallel Computing Toolbox license.</p><p>**Example:** `use-parallel: true`</p>

0 commit comments

Comments
 (0)