Re-architect workshop infrastructure

[niallthomson]

What would you like to be added?

This proposal would re-design the workshop infrastructure. The main changes would be:

VPC re-architecture

The IDE VPC will be expanded to be able to support an EKS cluster. We'll use the eksctl VPC design as a template in terms of subnets etc. The main change would be to make sure only private subnets are tagged for Karpenter.

IDE updates

The VPC ID should be injected in to the IDEs via bashrc.d mechanism. Other details like subnets should be looked up "just in time" when needed.

IaC updates

The eksctl and Terraform cluster configuration will be changed to use the IDE VPC instead of creating their own.

The eksctl configuration is already templated for the cluster name and region, this will be expanded to include the VPC details.

The Terraform can just look up the VPC by tags.

data "aws_vpc" "selected" {
  tags = [...]
}

This change would be considered "breaking" and will need a full announcement and migration instructions.

Content updates

All relevant content should be updated to make endpoints private, which includes:

1. The sample application when its deployed with a load balancer or ingress
2. 3rd party tools like ArgoCD and Kubecost

Content should be modified to note that we're creating internal load balancers.

Development & tests

Currently the local development experience assumes that endpoints like load balancers are public. Consideration will need to be made for how making everything private can be accounted for.

Why is this needed?

Ideally the sample application and other components like ArgoCD and Grafana are not exposed over the public Internet. This change would allow us to use internal load balancers. Using port forwarding through kubectl is not a great user experience and is error-prone.

[niallthomson]

Pushing this out as its more complex than first assessed.

For example in the Ingress labs we want the user to see the load balancer be created, and be able to access it in their browser using the normal URL. Proxying this away will make it less clear what is happening in terms of ALBs.

If code-server had some sort of browser that we could use which did some natural proxying this would be achievable but so far research has turned up no options for this that will work.