Skip to content

Commit f7cb488

Browse files
MickaelCaMickaelCa
committed
chore: integrate MinIO S3-compatible storage for local development, including bucket auto-setup and app credentials
1 parent 77a4b6e commit f7cb488

File tree

4 files changed

+136
-1
lines changed

4 files changed

+136
-1
lines changed

‎.docker/minio/setup.sh

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/sh
2+
3+
# Simple script to set up MinIO bucket and user
4+
# Based on example from MinIO issues
5+
6+
# Format bucket name to ensure compatibility
7+
BUCKET_NAME=$(echo "${S3_BUCKET_NAME}" | tr '[:upper:]' '[:lower:]' | tr '_' '-')
8+
9+
# Configure MinIO client
10+
mc alias set myminio http://minio:9000 ${MINIO_ROOT_USER} ${MINIO_ROOT_PASSWORD}
11+
12+
# Remove bucket if it exists (for clean setup)
13+
mc rm -r --force myminio/${BUCKET_NAME} || true
14+
15+
# Create bucket
16+
mc mb myminio/${BUCKET_NAME}
17+
18+
# Set bucket policy to allow downloads
19+
mc anonymous set download myminio/${BUCKET_NAME}
20+
21+
# Create user with access and secret keys
22+
mc admin user add myminio ${S3_ACCESS_KEY} ${S3_SECRET_KEY} || echo "User already exists"
23+
24+
# Create policy for the bucket
25+
echo '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:*"],"Resource":["arn:aws:s3:::'${BUCKET_NAME}'/*","arn:aws:s3:::'${BUCKET_NAME}'"]}]}' > /tmp/policy.json
26+
27+
# Apply policy
28+
mc admin policy create myminio gitingest-policy /tmp/policy.json || echo "Policy already exists"
29+
mc admin policy attach myminio gitingest-policy --user ${S3_ACCESS_KEY}
30+
31+
echo "MinIO setup completed successfully"
32+
echo "Bucket: ${BUCKET_NAME}"
33+
echo "Access via console: http://localhost:9001"

‎.env.example

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,22 @@ GITINGEST_SENTRY_PROFILE_LIFECYCLE=trace
3333
GITINGEST_SENTRY_SEND_DEFAULT_PII=true
3434
# Environment name for Sentry (default: "")
3535
GITINGEST_SENTRY_ENVIRONMENT=development
36+
37+
# MinIO Configuration (for development)
38+
# Root user credentials for MinIO admin access
39+
MINIO_ROOT_USER=minioadmin
40+
MINIO_ROOT_PASSWORD=minioadmin
41+
42+
# S3 Configuration (for application)
43+
# Endpoint URL for the S3 service (MinIO in development)
44+
S3_ENDPOINT=http://minio:9000
45+
# Access key for the S3 bucket (created automatically in development)
46+
S3_ACCESS_KEY=gitingest
47+
# Secret key for the S3 bucket (created automatically in development)
48+
S3_SECRET_KEY=gitingest123
49+
# Name of the S3 bucket (created automatically in development)
50+
S3_BUCKET_NAME=gitingest-bucket
51+
# Region for the S3 bucket (default for MinIO)
52+
S3_REGION=us-east-1
53+
# Public URL/CDN for accessing S3 resources
54+
S3_ALIAS_HOST=127.0.0.1:9000/gitingest-bucket

‎README.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,7 @@ The application can be configured using the following environment variables:
233233
- **GITINGEST_SENTRY_PROFILE_SESSION_SAMPLE_RATE**: Sampling rate for profile sessions (default: "1.0", range: 0.0-1.0)
234234
- **GITINGEST_SENTRY_PROFILE_LIFECYCLE**: Profile lifecycle mode (default: "trace")
235235
- **GITINGEST_SENTRY_SEND_DEFAULT_PII**: Send default personally identifiable information (default: "true")
236+
- **S3_ALIAS_HOST**: Public URL/CDN for accessing S3 resources (default: "127.0.0.1:9000/gitingest-bucket")
236237

237238
### Using Docker Compose
238239

@@ -256,7 +257,7 @@ x-app-base: &app-base
256257

257258
#### Services
258259

259-
The file defines two services:
260+
The file defines three services:
260261

261262
1. **app**: Production service configuration
262263
- Uses the `prod` profile
@@ -269,6 +270,31 @@ The file defines two services:
269270
- Mounts the source code for live development
270271
- Uses hot reloading for faster development
271272

273+
3. **minio**: S3-compatible object storage for development
274+
- Uses the `dev` profile (only available in development mode)
275+
- Provides S3-compatible storage for local development
276+
- Accessible via:
277+
- API: Port 9000 (http://localhost:9000)
278+
- Web Console: Port 9001 (http://localhost:9001)
279+
- Default admin credentials:
280+
- Username: `minioadmin`
281+
- Password: `minioadmin`
282+
- Configurable via environment variables:
283+
- `MINIO_ROOT_USER`: Custom admin username (default: minioadmin)
284+
- `MINIO_ROOT_PASSWORD`: Custom admin password (default: minioadmin)
285+
- Includes persistent storage via Docker volume
286+
- Auto-creates a bucket and application-specific credentials:
287+
- Bucket name: `gitingest-bucket` (configurable via `S3_BUCKET_NAME`)
288+
- Access key: `gitingest` (configurable via `S3_ACCESS_KEY`)
289+
- Secret key: `gitingest123` (configurable via `S3_SECRET_KEY`)
290+
- These credentials are automatically passed to the app-dev service via environment variables:
291+
- `S3_ENDPOINT`: URL of the MinIO server
292+
- `S3_ACCESS_KEY`: Access key for the S3 bucket
293+
- `S3_SECRET_KEY`: Secret key for the S3 bucket
294+
- `S3_BUCKET_NAME`: Name of the S3 bucket
295+
- `S3_REGION`: Region for the S3 bucket (default: us-east-1)
296+
- `S3_ALIAS_HOST`: Public URL/CDN for accessing S3 resources (default: "127.0.0.1:9000/gitingest-bucket")
297+
272298
#### Usage Examples
273299

274300
To run the application in development mode:

‎compose.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,65 @@ services:
4545
environment:
4646
- DEBUG=true
4747
- GITINGEST_SENTRY_ENVIRONMENT=${GITINGEST_SENTRY_ENVIRONMENT:-development}
48+
# S3 Configuration
49+
- S3_ENDPOINT=http://minio:9000
50+
- S3_ACCESS_KEY=${S3_ACCESS_KEY:-gitingest}
51+
- S3_SECRET_KEY=${S3_SECRET_KEY:-gitingest123}
52+
# Use lowercase bucket name to ensure compatibility with MinIO
53+
- S3_BUCKET_NAME=${S3_BUCKET_NAME:-gitingest-bucket}
54+
- S3_REGION=${S3_REGION:-us-east-1}
55+
# Public URL for S3 resources
56+
- S3_ALIAS_HOST=${S3_ALIAS_HOST:-127.0.0.1:9000/${S3_BUCKET_NAME:-gitingest-bucket}}
4857
volumes:
4958
# Mount source code for live development
5059
- ./src:/app:ro
5160
# Use --reload flag for hot reloading during development
5261
command: ["python", "-m", "uvicorn", "server.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
62+
depends_on:
63+
minio-setup:
64+
condition: service_completed_successfully
65+
66+
# MinIO S3-compatible object storage for development
67+
minio:
68+
image: minio/minio:latest
69+
profiles:
70+
- dev
71+
ports:
72+
- "9000:9000" # API port
73+
- "9001:9001" # Console port
74+
environment:
75+
- MINIO_ROOT_USER=${MINIO_ROOT_USER:-minioadmin}
76+
- MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD:-minioadmin}
77+
volumes:
78+
- minio-data:/data
79+
command: server /data --console-address ":9001"
80+
restart: unless-stopped
81+
healthcheck:
82+
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
83+
interval: 30s
84+
timeout: 30s
85+
start_period: 30s
86+
start_interval: 1s
87+
88+
# MinIO setup service to create bucket and user
89+
minio-setup:
90+
image: minio/mc
91+
profiles:
92+
- dev
93+
depends_on:
94+
minio:
95+
condition: service_healthy
96+
environment:
97+
- MINIO_ROOT_USER=${MINIO_ROOT_USER:-minioadmin}
98+
- MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD:-minioadmin}
99+
- S3_ACCESS_KEY=${S3_ACCESS_KEY:-gitingest}
100+
- S3_SECRET_KEY=${S3_SECRET_KEY:-gitingest123}
101+
- S3_BUCKET_NAME=${S3_BUCKET_NAME:-gitingest-bucket}
102+
volumes:
103+
- ./.docker/minio/setup.sh:/setup.sh:ro
104+
entrypoint: sh
105+
command: -c /setup.sh
106+
107+
volumes:
108+
minio-data:
109+
driver: local

0 commit comments

Comments
 (0)