Skip to content

Commit 77de500

Browse files
authored
docs: update run-tests.py documentation with enhanced usage instructions and memory management options (#1250)
…ons and memory management options # Pull Request Description ## Summary [Provide a brief description of the changes in this PR] ### Issue Reference Fixes #[Issue Number] ### Motivation and Context - Why is this change needed? - What problem does it solve? - If it fixes an open issue, please link to the issue here ### Dependencies - List any dependencies that are required for this change - Include any configuration changes needed - Note any version updates required ## Type of Change Please mark the relevant option with an `x`: - [ ] 🐛 Bug fix (non-breaking change which fixes an issue) - [ ] ✨ New feature (non-breaking change which adds functionality) - [ ] 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] 📝 Documentation update (Wiki/README/Code comments) - [ ] ♻️ Refactor (code improvement without functional changes) - [ ] 🎨 Style update (formatting, renaming) - [ ] 🔧 Configuration change - [ ] 📦 Dependency update ## Testing - [ ] I have added unit tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have tested this code in the following browsers/environments: [list environments] ## Quality Checklist - [ ] I have reviewed my own code before requesting review - [ ] I have verified there are no other open Pull Requests for the same update/change - [ ] All CI/CD pipelines pass without errors or warnings - [ ] My code follows the established style guidelines of this project - [ ] I have added necessary documentation (if appropriate) - [ ] I have commented my code, particularly in complex areas - [ ] I have made corresponding changes to the README and other relevant documentation - [ ] My changes generate no new warnings - [ ] I have performed a self-review of my own code - [ ] My code is properly formatted according to project standards ## Screenshots/Recordings (if appropriate) [Add screenshots or recordings that demonstrate the changes] ## Additional Notes [Add any additional information that might be helpful for reviewers]
2 parents f71b655 + d76c863 commit 77de500

File tree

2 files changed

+47
-21
lines changed

2 files changed

+47
-21
lines changed

.github/workflows/bi-weekly-tests.yml

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -215,25 +215,25 @@ jobs:
215215
strategy:
216216
fail-fast: false
217217
matrix:
218-
include: # testing both ubuntu-24.04-arm and ubuntu-22.04-arm-4c-l
218+
include: # testing both ubuntu-24.04-arm and ubuntu-24.04-arm
219219
- service: applications
220-
runner: ubuntu-22.04-arm-4c-l
220+
runner: ubuntu-24.04-arm
221221
- service: device_and_app_management
222-
runner: ubuntu-22.04-arm-4c-l
222+
runner: ubuntu-24.04-arm
223223
- service: device_management
224-
runner: ubuntu-22.04-arm-4c-l
224+
runner: ubuntu-24.04-arm
225225
- service: groups
226-
runner: ubuntu-22.04-arm-4c-l
226+
runner: ubuntu-24.04-arm
227227
- service: identity_and_access
228-
runner: ubuntu-22.04-arm-4c-l
228+
runner: ubuntu-24.04-arm
229229
- service: m365_admin
230-
runner: ubuntu-22.04-arm-4c-l
230+
runner: ubuntu-24.04-arm
231231
- service: multitenant_management
232-
runner: ubuntu-22.04-arm-4c-l
232+
runner: ubuntu-24.04-arm
233233
- service: utility
234-
runner: ubuntu-22.04-arm-4c-l
234+
runner: ubuntu-24.04-arm
235235
- service: windows_365
236-
runner: ubuntu-22.04-arm-4c-l
236+
runner: ubuntu-24.04-arm
237237
steps:
238238
- name: Harden Runner
239239
uses: step-security/harden-runner@95d9a5deda9de15063e7595e9719c11c38c90ae2 # v2.13.2

scripts/pipeline/run-tests.py

Lines changed: 37 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,42 @@
11
#!/usr/bin/env python3
2-
"""Test runner for nightly acceptance tests.
2+
"""Test runner for Terraform provider acceptance tests.
33
4-
This script runs Terraform provider tests sequentially, one package at a time,
5-
to conserve memory and provide better progress visibility.
4+
This script runs Terraform provider tests sequentially, one Go package at a time,
5+
to conserve memory on resource-constrained runners. Supports configurable parallelism
6+
and memory management options.
67
78
Usage:
8-
./run-tests.py <type> [service] [coverage-file] [test-output-file]
9+
./run-tests.py <type> [service] [coverage-file] [test-output-file] [options]
10+
11+
Positional Arguments:
12+
type Test type: provider-core, resources, or datasources
13+
service Service name (required for resources/datasources)
14+
coverage-file Coverage output file (default: coverage.txt)
15+
test-output-file Test log output file (default: test-output.log)
16+
17+
Memory & Parallelism Options:
18+
--max-procs N GOMAXPROCS - max CPU cores for Go (default: 2)
19+
--test-parallel N Tests to run in parallel within a package (default: 1)
20+
--pkg-parallel N Packages to build/test in parallel (default: 1)
21+
--race / --no-race Enable/disable race detection (auto: on for provider-core)
22+
--skip-enumeration Skip test enumeration phase for faster startup
23+
--force-gc Force garbage collection between packages (default: enabled)
24+
--no-force-gc Disable forced garbage collection
925
10-
Types:
11-
provider-core: Core provider tests (client, helpers, provider, utilities)
12-
resources: Resource tests for a specific service
13-
datasources: Datasource tests for a specific service
26+
Examples:
27+
# Low memory mode (8GB runners) - default settings
28+
./run-tests.py resources identity_and_access coverage.txt output.log
29+
30+
# Ultra-conservative mode (minimize memory)
31+
./run-tests.py resources identity_and_access coverage.txt output.log \
32+
--max-procs 1 --test-parallel 1 --pkg-parallel 1 --skip-enumeration --no-race
33+
34+
# Higher performance mode (16GB+ runners)
35+
./run-tests.py resources identity_and_access coverage.txt output.log \
36+
--max-procs 4 --test-parallel 2 --pkg-parallel 1
37+
38+
# Show help
39+
./run-tests.py --help
1440
"""
1541

1642
import os
@@ -230,9 +256,9 @@ def parse_test_results(output_file: str, category: str, service: str) -> None:
230256

231257

232258
def run_provider_core_tests(coverage_file: str, test_output_file: str,
233-
max_procs: int = 2, test_parallel: int = 1,
234-
pkg_parallel: int = 1, use_race: bool = True,
235-
skip_enumeration: bool = False, force_gc: bool = True) -> int:
259+
max_procs: int = 2, test_parallel: int = 1,
260+
pkg_parallel: int = 1, use_race: bool = True,
261+
skip_enumeration: bool = False, force_gc: bool = True) -> int:
236262
"""Run provider core tests sequentially, one package at a time to conserve memory.
237263
238264
Discovers all test packages in core directories (client, helpers, provider, utilities),

0 commit comments

Comments
 (0)