Skip to content

Commit 1d43bd0

Browse files
abcxffabcxff
committed
feat: k8 run scripts
1 parent 2b1b80f commit 1d43bd0

29 files changed

+1522
-9
lines changed

‎justfile‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,6 @@ docker-build-frontend:
1414
docker-run:
1515
docker run -p 6420:6420 -e RIVET__AUTH__ADMIN_TOKEN=dev -e RUST_LOG=debug rivetkit/engine:local
1616

17+
[group('docker')]
18+
docker-stop:
19+
docker run -p 6420:6420 -e RIVET__AUTH__ADMIN_TOKEN=dev -e RUST_LOG=debug rivetkit/engine:local

‎k8s/DEPLOY.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Rivet Engine Kubernetes Quick Deployment
2+
3+
**For the complete deployment guide, see the [Kubernetes documentation](https://rivet.gg/docs/self-hosting/kubernetes).**
4+
5+
This guide is for developers who have already cloned the repository and want to deploy locally.
6+
7+
## Prerequisites
8+
9+
- Kubernetes cluster (v1.24+)
10+
- `kubectl` configured
11+
- Metrics server (required for HPA) - included by default in most distributions
12+
13+
## Architecture
14+
15+
The Rivet Engine deployment consists of two components:
16+
17+
- **Main Engine Deployment**: Runs all services except singleton services. Configured with Horizontal Pod Autoscaling (HPA) to automatically scale between 2-10 replicas based on CPU (60%) and memory (80%) utilization.
18+
- **Singleton Engine Deployment**: Runs singleton services that must have exactly 1 replica (e.g., schedulers, coordinators).
19+
20+
## Deploy
21+
22+
```bash
23+
# Apply all manifests
24+
kubectl apply -f engine/
25+
26+
# Wait for pods
27+
kubectl -n rivet-engine wait --for=condition=ready pod -l app=postgres --timeout=300s
28+
kubectl -n rivet-engine wait --for=condition=ready pod -l app=rivet-engine --timeout=300s
29+
kubectl -n rivet-engine wait --for=condition=ready pod -l app=rivet-engine-singleton --timeout=300s
30+
```
31+
32+
## Verify
33+
34+
```bash
35+
# Check pods (you should see 2+ engine pods and 1 singleton pod)
36+
kubectl -n rivet-engine get pods
37+
38+
# Check HPA status
39+
kubectl -n rivet-engine get hpa
40+
41+
# Port forward
42+
kubectl -n rivet-engine port-forward svc/rivet-engine 6420:6420 6421:6421
43+
44+
# Test health
45+
curl http://localhost:6421/health
46+
```
47+
48+
Expected response:
49+
```json
50+
{"runtime":"engine","status":"ok","version":"..."}
51+
```
52+
53+
## Logs
54+
55+
```bash
56+
# View main engine logs
57+
kubectl -n rivet-engine logs -l app=rivet-engine -f
58+
59+
# View singleton engine logs
60+
kubectl -n rivet-engine logs -l app=rivet-engine-singleton -f
61+
```
62+
63+
## Cleanup
64+
65+
```bash
66+
kubectl delete namespace rivet-engine
67+
```
68+
69+
## Next Steps
70+
71+
See [README.md](README.md) for more deployment options and [the documentation](https://rivet.gg/docs/self-hosting/kubernetes) for production setup.

‎k8s/README.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Rivet Engine Kubernetes Deployment
2+
3+
Kubernetes manifests for deploying Rivet Engine.
4+
5+
## Quick Start
6+
7+
**For a complete quick start guide with copy-paste YAML manifests, see the [Kubernetes documentation](https://rivet.gg/docs/self-hosting/kubernetes).**
8+
9+
## What's in This Directory
10+
11+
The `engine/` directory contains reference Kubernetes manifests for advanced deployments:
12+
13+
- `00-namespace.yaml` - Namespace definition
14+
- `01-serviceaccount.yaml` - Service account
15+
- `02-engine-configmap.yaml` - Engine configuration
16+
- `03-rivet-engine-deployment.yaml` - Main engine deployment
17+
- `04-rivet-engine-service.yaml` - Service definition
18+
- `05-rivet-engine-hpa.yaml` - Horizontal Pod Autoscaler
19+
- `06-rivet-engine-singleton-deployment.yaml` - Singleton services
20+
- `07-nats-configmap.yaml` - NATS configuration
21+
- `08-nats-statefulset.yaml` - NATS cluster
22+
- `09-nats-service.yaml` - NATS service
23+
- `10-postgres-configmap.yaml` - PostgreSQL configuration
24+
- `11-postgres-secret.yaml` - PostgreSQL credentials
25+
- `12-postgres-statefulset.yaml` - PostgreSQL database
26+
- `13-postgres-service.yaml` - PostgreSQL service
27+
28+
## Local Development
29+
30+
For local development with k3d:
31+
32+
```bash
33+
./scripts/run/k8s/engine.sh
34+
```
35+
36+
This script creates a k3d cluster, builds the image, and deploys everything.
37+
38+
## Production Deployment
39+
40+
For production deployments, see the steps outlined in our [Kubernetes Self-Hosting Guide](https://rivet.gg/docs/self-hosting/kubernetes).

‎k8s/engine/00-namespace.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
apiVersion: v1
2+
kind: Namespace
3+
metadata:
4+
labels:
5+
component: rivet-engine
6+
datacenter: YOUR_DATACENTER_KEY
7+
name: rivet-engine
8+
service: engine
9+
name: rivet-engine

‎k8s/engine/01-serviceaccount.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
apiVersion: v1
2+
kind: ServiceAccount
3+
metadata:
4+
name: rivet-engine
5+
namespace: rivet-engine

‎k8s/engine/02-engine-configmap.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
apiVersion: v1
2+
data:
3+
config.jsonc: |
4+
{
5+
"postgres": {
6+
"url": "postgresql://postgres:postgres@postgres:5432/rivet"
7+
},
8+
9+
"topology": {
10+
"datacenter_label": 1,
11+
"datacenters": [
12+
{
13+
"name": "local",
14+
"datacenter_label": 1,
15+
"is_leader": true,
16+
"public_url": "http://localhost:6420",
17+
"peer_url": "http://rivet-engine.rivet-engine.svc.cluster.local:6421",
18+
"proxy_url": "http://rivet-engine.rivet-engine.svc.cluster.local:6420"
19+
}
20+
]
21+
}
22+
}
23+
kind: ConfigMap
24+
metadata:
25+
labels:
26+
app: rivet-engine
27+
component: rivet-engine
28+
service: engine
29+
name: engine-config
30+
namespace: rivet-engine

‎k8s/engine/03-rivet-engine-deployment.yaml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
labels:
5+
app: rivet-engine
6+
component: rivet-engine
7+
service: engine
8+
name: rivet-engine
9+
namespace: rivet-engine
10+
spec:
11+
replicas: 2
12+
selector:
13+
matchLabels:
14+
app: rivet-engine
15+
template:
16+
metadata:
17+
annotations:
18+
checksum/config: REPLACE_WITH_CONFIG_CHECKSUM
19+
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
20+
labels:
21+
app: rivet-engine
22+
component: rivet-engine
23+
service: engine
24+
spec:
25+
containers:
26+
- args:
27+
- start
28+
- --except-services
29+
- singleton
30+
env:
31+
- name: RIVET_CONFIG_PATH
32+
value: /etc/rivet/config.jsonc
33+
image: rivet-engine:local
34+
imagePullPolicy: Never
35+
livenessProbe:
36+
failureThreshold: 3
37+
httpGet:
38+
path: /health
39+
port: 6421
40+
periodSeconds: 10
41+
timeoutSeconds: 5
42+
name: rivet-engine
43+
ports:
44+
- containerPort: 6420
45+
name: guard
46+
protocol: TCP
47+
- containerPort: 6421
48+
name: api-peer
49+
protocol: TCP
50+
readinessProbe:
51+
failureThreshold: 2
52+
httpGet:
53+
path: /health
54+
port: 6421
55+
periodSeconds: 5
56+
timeoutSeconds: 3
57+
resources:
58+
limits:
59+
cpu: 4000m
60+
memory: 8Gi
61+
requests:
62+
cpu: 2000m
63+
memory: 4Gi
64+
startupProbe:
65+
failureThreshold: 30
66+
httpGet:
67+
path: /health
68+
port: 6421
69+
initialDelaySeconds: 30
70+
periodSeconds: 10
71+
timeoutSeconds: 5
72+
volumeMounts:
73+
- mountPath: /etc/rivet
74+
name: config
75+
readOnly: true
76+
serviceAccountName: rivet-engine
77+
volumes:
78+
- configMap:
79+
name: engine-config
80+
name: config

‎k8s/engine/04-rivet-engine-service.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
apiVersion: v1
2+
kind: Service
3+
metadata:
4+
labels:
5+
app: rivet-engine
6+
component: rivet-engine
7+
service: engine
8+
name: rivet-engine
9+
namespace: rivet-engine
10+
spec:
11+
type: NodePort
12+
ports:
13+
- name: guard
14+
port: 6420
15+
targetPort: 6420
16+
nodePort: 30420
17+
protocol: TCP
18+
- name: api-peer
19+
port: 6421
20+
targetPort: 6421
21+
nodePort: 30421
22+
protocol: TCP
23+
selector:
24+
app: rivet-engine

‎k8s/engine/05-rivet-engine-hpa.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
apiVersion: autoscaling/v2
2+
kind: HorizontalPodAutoscaler
3+
metadata:
4+
name: rivet-engine
5+
namespace: rivet-engine
6+
spec:
7+
maxReplicas: 10
8+
metrics:
9+
- resource:
10+
name: cpu
11+
target:
12+
averageUtilization: 60
13+
type: Utilization
14+
type: Resource
15+
- resource:
16+
name: memory
17+
target:
18+
averageUtilization: 80
19+
type: Utilization
20+
type: Resource
21+
minReplicas: 2
22+
scaleTargetRef:
23+
apiVersion: apps/v1
24+
kind: Deployment
25+
name: rivet-engine

‎k8s/engine/06-rivet-engine-singleton-deployment.yaml‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
labels:
5+
app: rivet-engine-singleton
6+
component: singleton
7+
service: engine
8+
name: rivet-engine-singleton
9+
namespace: rivet-engine
10+
spec:
11+
replicas: 1
12+
selector:
13+
matchLabels:
14+
app: rivet-engine-singleton
15+
template:
16+
metadata:
17+
annotations:
18+
checksum/config: REPLACE_WITH_CONFIG_CHECKSUM
19+
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
20+
labels:
21+
app: rivet-engine-singleton
22+
component: singleton
23+
service: engine
24+
spec:
25+
containers:
26+
- args:
27+
- start
28+
- --services
29+
- singleton
30+
- --services
31+
- api-peer
32+
env:
33+
- name: RIVET_CONFIG_PATH
34+
value: /etc/rivet/config.jsonc
35+
image: rivet-engine:local
36+
imagePullPolicy: Never
37+
livenessProbe:
38+
failureThreshold: 3
39+
httpGet:
40+
path: /health
41+
port: 6421
42+
periodSeconds: 10
43+
timeoutSeconds: 5
44+
name: rivet-engine
45+
ports:
46+
- containerPort: 6421
47+
name: api-peer
48+
protocol: TCP
49+
readinessProbe:
50+
failureThreshold: 2
51+
httpGet:
52+
path: /health
53+
port: 6421
54+
periodSeconds: 5
55+
timeoutSeconds: 3
56+
resources:
57+
limits:
58+
cpu: 4000m
59+
memory: 8Gi
60+
requests:
61+
cpu: 2000m
62+
memory: 4Gi
63+
startupProbe:
64+
failureThreshold: 30
65+
httpGet:
66+
path: /health
67+
port: 6421
68+
initialDelaySeconds: 30
69+
periodSeconds: 10
70+
timeoutSeconds: 5
71+
volumeMounts:
72+
- mountPath: /etc/rivet
73+
name: config
74+
readOnly: true
75+
serviceAccountName: rivet-engine
76+
volumes:
77+
- configMap:
78+
name: engine-config
79+
name: config

0 commit comments

Comments
 (0)