Skip to content

Commit 8a14cc6

Browse files
mzazrivecmzazrivec
committed
Proposal for ROSA network
1 parent 5b02ba0 commit 8a14cc6

File tree

1 file changed

+212
-0
lines changed

1 file changed

+212
-0
lines changed

‎docs/proposal/20250224-rosa-networking.md‎

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
---
2+
title: Networking for ROSA HCP
3+
authors:
4+
- "@mzazrivec"
5+
reviewers:
6+
-
7+
creation-date: 2025-02-24
8+
last-updated: 2025-03-21
9+
status: provisional
10+
---
11+
12+
# Networking for ROSA HCP
13+
14+
## Table of Contents
15+
16+
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
17+
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
18+
19+
- [Glossary](#glossary)
20+
- [Summary](#summary)
21+
- [Motivation](#motivation)
22+
- [Goals](#goals)
23+
- [Non-Goals/Future Work](#non-goalsfuture-work)
24+
- [Proposal](#proposal)
25+
- [User Stories](#user-stories)
26+
- [Functional Requirements](#functional-requirements)
27+
- [Alternatives](#alternatives)
28+
- [Upgrade Strategy](#upgrade-strategy)
29+
30+
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
31+
32+
## Glossary
33+
34+
Refer to the [Cluster API Book Glossary](https://cluster-api.sigs.k8s.io/reference/glossary.html).
35+
36+
## Summary
37+
38+
This proposal defines implementation of networking infrastructure in CAPA for ROSA Hosted Control Plane.
39+
40+
## Motivation
41+
42+
To be able to provision a new ROSA HCP kubernetes cluster using CAPA, one has to create and setup the underlying network infrastructure first: VPC, public and private subnets, internet gateway, routing tables for both subnets, elastic IP address.
43+
44+
All of the above can be currently provisioned and configured via AWS CLI, AWS Management Console or Terraform. Motivation for this work is to be able to provision and configure the network infrastructure for ROSA HCP using CAPI.
45+
46+
### Goals
47+
48+
1. Implement a namespaced new custom resource `RosaNetwork` representing the networking stack for ROSA HCP.
49+
2. It will be possible to reference the new `RosaNetwork` resource from ROSA control plane resource
50+
3. Implement creation and deletion for the new `RosaNetwork` resource.
51+
4. Support the same networking scenarios as [ROSA CLI](https://github.com/openshift/rosa) using the same embeded AWS CloudFormation template that ROSA CLI uses.
52+
53+
### Non-Goals/Future Work
54+
55+
- Modify current networking code in AWS / EKS clusters.
56+
- Support custom CloudFormation template.
57+
58+
## Proposal
59+
60+
The goal of this proposal is to be able to provision the networking infrastructure required for a ROSA HCP cluster.
61+
62+
[ROSA CLI](https://github.com/openshift/rosa) supports creation of the networking infrastructure for ROSA HCP and uses [AWS CloudFormation](https://aws.amazon.com/cloudformation/) template under the hood. The [CloudFormation template used by ROSA CLI](https://github.com/openshift/rosa/blob/master/cmd/create/network/templates/rosa-quickstart-default-vpc/cloudformation.yaml) allows to specify four parameters: CloudFormation stack name, AZ count (between 1 and 3), region and CIDR block for the VPC. The created CloudFormation stack then contains a VPC, public and private subnets (each pair created in separate AZ), internet gateway attached to VPC, elastic IPs, NAT gateways, public and private routing tables and a security group.
63+
64+
Adopting the CloudFormation template used by rosa-cli would mean that CAPA and the `RosaNetwork` custom resource would be relying on a mechanism that is know to work well and any changes or fixes implemented in ROSA CLI would be picked up automatically in CAPA.
65+
66+
In practical terms, implementation of the proposal would mean:
67+
1. A new namespaced custom resource definition `RosaNetwork` in CAPA with four attributes: name, AZ count, region and CIDR block for VPC. `availability_zone_count`, `region` and `cidr_block` will become the `spec` part of the new `RosaNetwork` type, name of the cloudformation stack will be the same as `metadata.name`.
68+
69+
`RosaNetwork` spec example:
70+
```
71+
kind: RosaNetwork
72+
metadata:
73+
name: rosa-network-01
74+
namespace: default
75+
spec:
76+
availability_zone_count: 3
77+
region: us-west-2
78+
cidr_block: 10.0.0.0/16
79+
```
80+
81+
1. A new reconciler for the new custom resource, implementing creation and deletion. The reconciler will be using an existing [CloudFormation template from ROSA CLI](https://github.com/openshift/rosa/blob/master/cmd/create/network/templates/rosa-quickstart-default-vpc/cloudformation.yaml) and will use [AWS CloudFormation API](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/cloudformation) to [create](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/cloudformation#Client.CreateStack) and [delete](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/cloudformation#Client.DeleteStack) the AWS CloudFormation stack.
82+
83+
Outputs and resources created in the cloudformation stack will be tracked under `status` of the `RosaNetwork` type. In particular, the `status` will have the following fields:
84+
* publicSubnets: array of public subnet ids
85+
* privateSubnets: array of private subnet ids
86+
* availabilityZones: array of AZs
87+
88+
Example:
89+
```
90+
kind: RosaNetwork
91+
metadata:
92+
name: rosa-network-01
93+
namespace: default
94+
status:
95+
publicSubnets:
96+
- subnet-0d9f28ba991b93514
97+
- subnet-2d7f18c09f1b43512
98+
- subnet-1d7e19c0af1c4c57f
99+
privateSubnets:
100+
- subnet-1d9f28ba992a83514
101+
- subnet-2d7f58c09f1b43512
102+
- subnet-7d7e19c0af1f4d57f
103+
availabilityZones:
104+
- us-west-2a
105+
- us-west-2b
106+
- us-west-2c
107+
```
108+
109+
All resources created in the cloudformation stack will be tracked under `status.resources` array:
110+
```
111+
kind: RosaNetwork
112+
metadata:
113+
name: rosa-network-01
114+
namespace: default
115+
status:
116+
resources:
117+
- resource: NATGateway1
118+
id:
119+
status: CREATE_IN_PROGRESS
120+
reason: Eventual consistency check initiated
121+
- resource: VPC
122+
id: vpc-055edf3ebf27f18d6
123+
status: CREATE_COMPLETE
124+
reason:
125+
- resource: SecurityGroup
126+
id:
127+
status: CREATE_IN_PROGRESS
128+
reason: Resource creation Initiated
129+
```
130+
and will be reflecting the the values coming from [AWS CloudFormation API](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/cloudformation#Client.DescribeStackEvents) (`resource`, `id`, `status` and `reason`).
131+
132+
`status.conditions` of the `RosaNetwork` resource will be consistent with the CAPA conventions, example of a successful network stack creation:
133+
```
134+
kind: RosaNetwork
135+
metadata:
136+
name: rosa-network-01
137+
namespace: default
138+
status:
139+
conditions:
140+
- lastTransitionTime: "2025-03-20T14:45:26Z"
141+
status: "True"
142+
type: RosaNetworkReady
143+
```
144+
Example of failed network stack creation:
145+
```
146+
kind: RosaNetwork
147+
metadata:
148+
name: rosa-network-01
149+
namespace: default
150+
status:
151+
conditions:
152+
- lastTransitionTime: "2025-03-18T13:25:16Z"
153+
status: "False"
154+
type: RosaNetworkReady
155+
severity: Error
156+
reason: ReconciliationFailed
157+
message: Insufficient privileges for ...
158+
```
159+
Failed deletion:
160+
```
161+
kind: RosaNetwork
162+
metadata:
163+
name: rosa-network-01
164+
namespace: default
165+
status:
166+
conditions:
167+
- lastTransitionTime: "2025-03-18T13:25:16Z"
168+
status: "False"
169+
type: RosaNetworkReady
170+
severity: Error
171+
reason: DeletionFailed
172+
message: ...
173+
```
174+
175+
1. Modifications in the ROSA control plane CRD & reconciler so that it would be possible to reference the `RosaNetwork` resource from control plane:
176+
```
177+
kind: ROSAControlPlane
178+
metadata:
179+
name: hcp01-control-plane
180+
namespace: default
181+
spec:
182+
RosaNetworkRef:
183+
name: hcp01-rosa-network
184+
```
185+
Should the ROSA control plane CR contain reference to ROSA network, the reconciler will read the AWS region, AZ and subnet ids parameters from the ROSA network CR.
186+
187+
1. New tests.
188+
189+
### User Stories
190+
191+
1. As a CAPA user, I want to be able to provision the network infrastructure for ROSA HCP.
192+
2. As a CAPA user, I want to be able to use the provisioned network infrastructure in ROSA HCP control plane.
193+
3. As a CAPA user, I want to be able to delete the network infrastructure previously provisioned by CAPA.
194+
195+
#### Functional Requirements
196+
197+
1. Ability to create a new namespaced custom resource `RosaNetwork` with four attributes: name, AZ count, region and CIDR block for VPC.
198+
2. Reconciler implementing creation and deletion of the `RosaNetwork` resource.
199+
3. Ability to reference the new custom resource from ROSA HCP control plane.
200+
201+
## Alternatives
202+
203+
1. Implement CRDs and reconcilers for each of the atoms of network infrastructure (VPCs, subnets, etc.).
204+
2. Implement the network infrasructure similar to EKS, the network parameters being attributes of the EKS control plane.
205+
3. Not implement anything and rely purely on AWS CLI or Terraform.
206+
207+
## Upgrade Strategy
208+
209+
The implementation will not affect CAPA upgrades.
210+
211+
<!-- Links -->
212+
[community meeting]: https://docs.google.com/document/d/1ushaVqAKYnZ2VN_aa3GyKlS4kEd6bSug13xaXOakAQI/edit#heading=h.pxsq37pzkbdq

0 commit comments

Comments
 (0)