Skip to content

Commit 22b5ce1

Browse files
corylanouclaude
corylanou
and
claude
authored
test: migrate bash integration and soak tests to Go infrastructure (#799)
Co-authored-by: Claude <noreply@anthropic.com>
1 parent f2d217a commit 22b5ce1

29 files changed

+4998
-4256
lines changed

‎.github/workflows/integration-tests.yml‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
name: Integration Tests
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- '**.go'
7+
- 'go.mod'
8+
- 'go.sum'
9+
- 'tests/integration/**'
10+
- '.github/workflows/integration-tests.yml'
11+
workflow_dispatch:
12+
inputs:
13+
test_type:
14+
description: 'Test type to run'
15+
required: false
16+
default: 'quick'
17+
type: choice
18+
options:
19+
- 'quick'
20+
- 'all'
21+
- 'long'
22+
23+
permissions:
24+
contents: read
25+
26+
jobs:
27+
quick-tests:
28+
name: Quick Integration Tests
29+
runs-on: ubuntu-latest
30+
if: github.event_name == 'pull_request' || inputs.test_type == 'quick' || inputs.test_type == 'all'
31+
steps:
32+
- uses: actions/checkout@v4
33+
34+
- uses: actions/setup-go@v5
35+
with:
36+
go-version-file: "go.mod"
37+
38+
- name: Build binaries
39+
run: |
40+
go build -o bin/litestream ./cmd/litestream
41+
go build -o bin/litestream-test ./cmd/litestream-test
42+
43+
- name: Run quick integration tests
44+
run: |
45+
go test -v -tags=integration -timeout=30m ./tests/integration/... \
46+
-run="TestFreshStart|TestDatabaseIntegrity|TestRapidCheckpoints"
47+
env:
48+
CGO_ENABLED: 1
49+
50+
- name: Upload test logs
51+
if: failure()
52+
uses: actions/upload-artifact@v4
53+
with:
54+
name: quick-test-logs
55+
path: |
56+
/tmp/litestream-*/*.log
57+
/tmp/*-test.log
58+
59+
scenario-tests:
60+
name: Scenario Integration Tests
61+
runs-on: ubuntu-latest
62+
if: github.event_name == 'workflow_dispatch' && (inputs.test_type == 'all' || inputs.test_type == 'long')
63+
steps:
64+
- uses: actions/checkout@v4
65+
66+
- uses: actions/setup-go@v5
67+
with:
68+
go-version-file: "go.mod"
69+
70+
- name: Build binaries
71+
run: |
72+
go build -o bin/litestream ./cmd/litestream
73+
go build -o bin/litestream-test ./cmd/litestream-test
74+
75+
- name: Run all scenario tests
76+
run: |
77+
go test -v -tags=integration -timeout=1h ./tests/integration/... \
78+
-run="Test(FreshStart|DatabaseIntegrity|DatabaseDeletion|RapidCheckpoints|WALGrowth|ConcurrentOperations|BusyTimeout)"
79+
env:
80+
CGO_ENABLED: 1
81+
82+
- name: Upload test logs
83+
if: always()
84+
uses: actions/upload-artifact@v4
85+
with:
86+
name: scenario-test-logs
87+
path: |
88+
/tmp/litestream-*/*.log
89+
/tmp/*-test.log
90+
91+
long-running-tests:
92+
name: Long-Running Integration Tests
93+
runs-on: ubuntu-latest
94+
if: github.event_name == 'workflow_dispatch' && inputs.test_type == 'long'
95+
timeout-minutes: 600
96+
steps:
97+
- uses: actions/checkout@v4
98+
99+
- uses: actions/setup-go@v5
100+
with:
101+
go-version-file: "go.mod"
102+
103+
- name: Build binaries
104+
run: |
105+
go build -o bin/litestream ./cmd/litestream
106+
go build -o bin/litestream-test ./cmd/litestream-test
107+
108+
- name: Run long tests
109+
run: |
110+
go test -v -tags="integration,long" -timeout=10h ./tests/integration/... \
111+
-run="TestOvernight|Test1GBBoundary"
112+
env:
113+
CGO_ENABLED: 1
114+
115+
- name: Upload test logs
116+
if: always()
117+
uses: actions/upload-artifact@v4
118+
with:
119+
name: long-test-logs
120+
path: |
121+
/tmp/litestream-*/*.log
122+
/tmp/*-test.log
123+
124+
summary:
125+
name: Test Summary
126+
runs-on: ubuntu-latest
127+
needs: [quick-tests]
128+
if: always() && (github.event_name == 'pull_request' || inputs.test_type == 'quick' || inputs.test_type == 'all')
129+
steps:
130+
- name: Generate summary
131+
run: |
132+
echo "## Integration Test Results" >> $GITHUB_STEP_SUMMARY
133+
echo "" >> $GITHUB_STEP_SUMMARY
134+
135+
if [ "${{ needs.quick-tests.result }}" == "success" ]; then
136+
echo "✅ **Quick Tests:** Passed" >> $GITHUB_STEP_SUMMARY
137+
elif [ "${{ needs.quick-tests.result }}" == "failure" ]; then
138+
echo "❌ **Quick Tests:** Failed" >> $GITHUB_STEP_SUMMARY
139+
elif [ "${{ needs.quick-tests.result }}" == "skipped" ]; then
140+
echo "⏭️ **Quick Tests:** Skipped" >> $GITHUB_STEP_SUMMARY
141+
fi
142+
143+
echo "" >> $GITHUB_STEP_SUMMARY
144+
echo "---" >> $GITHUB_STEP_SUMMARY
145+
echo "**Triggered by:** @${{ github.actor }}" >> $GITHUB_STEP_SUMMARY
146+
147+
# Note: Scenario and long-running tests run independently on workflow_dispatch.
148+
# Check individual job results for those test suites.

‎cmd/litestream-test/scripts/README.md‎

Lines changed: 2 additions & 151 deletions
Original file line numberDiff line numberDiff line change
@@ -13,18 +13,11 @@ go build -o bin/litestream-test ./cmd/litestream-test
1313

1414
## Quick Reference
1515

16+
> **Note:** Some tests have been migrated to Go integration tests in `tests/integration/`. See [tests/integration/README.md](../../tests/integration/README.md) for the Go-based test suite.
17+
1618
| Script | Purpose | Duration | Status |
1719
|--------|---------|----------|--------|
1820
| verify-test-setup.sh | Environment validation | ~5s | ✅ Stable |
19-
| test-fresh-start.sh | Fresh database creation | ~30s | ✅ Stable |
20-
| test-rapid-checkpoints.sh | Checkpoint stress test | ~2min | ✅ Stable |
21-
| test-wal-growth.sh | Large WAL handling (100MB+) | ~5min | ✅ Stable |
22-
| test-concurrent-operations.sh | Multi-database concurrent replication | ~5min | ✅ Stable |
23-
| test-database-integrity.sh | Complex data integrity validation | ~3min | ✅ Stable |
24-
| test-database-deletion.sh | Database deletion scenarios | ~2min | ✅ Stable |
25-
| test-replica-failover.sh | Replica failover testing | ~3min | ✅ Stable |
26-
| test-busy-timeout.sh | Database busy timeout handling | ~2min | ✅ Stable |
27-
| test-1gb-boundary.sh | SQLite 1GB lock page boundary | ~10min | ⚠️ Blocked by #754 |
2821
| reproduce-critical-bug.sh | Checkpoint during downtime bug | ~2min | 🐛 Reproduces #752 |
2922
| test-754-s3-scenarios.sh | Issue #754 S3 vs file replication | ~10min | 🐛 Tests #754 |
3023
| test-754-restore-focus.sh | Issue #754 restore focus | ~5min | 🐛 Tests #754 |
@@ -58,148 +51,6 @@ Verifies that the test environment is properly configured with required binaries
5851
- SQLite3 available
5952
- Python dependencies for S3 mock
6053

61-
### Core Functionality Tests
62-
63-
#### test-fresh-start.sh
64-
Tests replication with a fresh database that doesn't exist when Litestream starts.
65-
66-
```bash
67-
./cmd/litestream-test/scripts/test-fresh-start.sh
68-
```
69-
70-
**Tests:**
71-
- Starting Litestream before database exists
72-
- Database creation while Litestream is running
73-
- Automatic detection of new database
74-
- Replication and restore integrity
75-
76-
#### test-database-integrity.sh
77-
Creates complex data patterns and verifies integrity after restore.
78-
79-
```bash
80-
./cmd/litestream-test/scripts/test-database-integrity.sh
81-
```
82-
83-
**Tests:**
84-
- Complex data patterns (multiple tables, indexes)
85-
- SQLite PRAGMA integrity_check
86-
- Full database restoration
87-
- Data consistency verification
88-
89-
#### test-database-deletion.sh
90-
Tests scenarios where the source database is deleted during replication.
91-
92-
```bash
93-
./cmd/litestream-test/scripts/test-database-deletion.sh
94-
```
95-
96-
**Tests:**
97-
- Database deletion during active replication
98-
- Recovery behavior
99-
- Replica consistency
100-
101-
#### test-replica-failover.sh
102-
Tests replica failover scenarios with multiple replicas.
103-
104-
```bash
105-
./cmd/litestream-test/scripts/test-replica-failover.sh
106-
```
107-
108-
**Tests:**
109-
- Multiple replica configuration
110-
- Failover when primary replica fails
111-
- Data consistency across replicas
112-
113-
### Stress & Performance Tests
114-
115-
#### test-rapid-checkpoints.sh
116-
Tests Litestream under rapid checkpoint pressure with continuous writes.
117-
118-
```bash
119-
./cmd/litestream-test/scripts/test-rapid-checkpoints.sh
120-
```
121-
122-
**Tests:**
123-
- 100+ writes/second
124-
- Forced rapid checkpoints
125-
- Replication under checkpoint pressure
126-
- Data integrity under stress
127-
128-
**Key Metrics:**
129-
- Checkpoint frequency
130-
- WAL file growth
131-
- Replication lag
132-
- Error rates
133-
134-
#### test-wal-growth.sh
135-
Tests handling of large WAL files (100MB+) under sustained write load.
136-
137-
```bash
138-
./cmd/litestream-test/scripts/test-wal-growth.sh
139-
```
140-
141-
**Tests:**
142-
- Sustained high write rates (400+ writes/sec)
143-
- Large WAL file creation and handling
144-
- Checkpoint behavior with large WALs
145-
- Replication performance with large data
146-
147-
**Key Findings:**
148-
- Successfully handles 100MB+ WAL files
149-
- Maintains data integrity
150-
- Handles 400+ writes/second
151-
152-
#### test-concurrent-operations.sh
153-
Tests multiple databases replicating simultaneously with competing operations.
154-
155-
```bash
156-
./cmd/litestream-test/scripts/test-concurrent-operations.sh
157-
```
158-
159-
**Tests:**
160-
- Multiple databases (3-5) replicating concurrently
161-
- Mixed read/write operations
162-
- Competing checkpoints
163-
- Resource contention handling
164-
165-
#### test-busy-timeout.sh
166-
Tests database busy timeout handling with concurrent access.
167-
168-
```bash
169-
./cmd/litestream-test/scripts/test-busy-timeout.sh
170-
```
171-
172-
**Tests:**
173-
- Concurrent database access
174-
- Busy timeout configuration
175-
- Lock contention handling
176-
- Recovery from busy states
177-
178-
### Boundary & Edge Case Tests
179-
180-
#### test-1gb-boundary.sh
181-
Tests SQLite's 1GB lock page boundary handling.
182-
183-
```bash
184-
./cmd/litestream-test/scripts/test-1gb-boundary.sh
185-
```
186-
187-
**Tests:**
188-
- Database growth beyond 1GB (with 4KB pages)
189-
- Lock page at #262145 properly skipped
190-
- Replication across lock page boundary
191-
- Restoration integrity after crossing boundary
192-
193-
**Status:** ⚠️ Currently blocked by ltx v0.5.0 flag compatibility issue (#754)
194-
195-
**Lock Page Numbers by Page Size:**
196-
| Page Size | Lock Page # |
197-
|-----------|-------------|
198-
| 4KB | 262145 |
199-
| 8KB | 131073 |
200-
| 16KB | 65537 |
201-
| 32KB | 32769 |
202-
20354
### Bug Reproduction Scripts
20455

20556
#### reproduce-critical-bug.sh

0 commit comments

Comments
 (0)