Proposal: Populate repo-server container with ClusterTrustBundle CA certificates #1830
Description
Permit Argo CD administrators to declare additional CA trust anchors for repo-server through ClusterTrustBundles, Secrets or ConfigMaps.
Repository server and its sidecars issues a TLS connection to external hosts where Argo's certificate pinning does not work very well.
Is your feature request related to a problem? Please describe.
Use Cases:
- As an Argo CD administrator, I would like to have a possibility to accept Trust Anchors from the cluster in the same way as for other workloads (instead of re-declaring them in Argo's
argocd-tls-certs-cm). - As an Argo CD administrator, I would like not to worry about explicitly managing trust to well known hosts (raw.githubusercontent.com, etc.) while adding custom CAs.
- As an Argo CD administrator, I would like my repo-server (and its plugin sidecars) trust the hosts on OS level. This is important when my executed commands reach out to further TLS hosts. Such as kustomize, fetching files over TLS, etc.
Describe the solution you'd like
- Introduce a new element in the
ArgoCDconfiguration:
apiVersion: argoproj.io/v1alpha1
kind: ArgoCD
metadata:
name: example-argocd
labels:
example: repo
spec:
repo:
systemCATrust:
secrets:
- name: my-local-cert-secret
items:
- key: key-name-in-the-secret-object
# Must end with .crt
path: desired-file-name-of-the-certificate.crt
configMaps:
- name: my-local-cert-cm
# Map all keys in the ConfigMap to files with the same name
# Key names in the ConfigMap must end with .crt
items: {}
clusterTrustBundles:
- name: my-global-ctb
path: my-global-ctb.crt
optional: true- Have the operator decorate the repo-server pod with needed
initContainers andvolumeMounts (see Additional context) to update the system CA trust. - Have the operator decorate the repo-server pod sidecar container with the needed
volumeMounts.
Describe alternatives you've considered
- Certificate pinning through
argocd-tls-certs-cm- Does not satisfy UC1 and UC3)
- Mounting the pre-build CA bundle, with well known CA + the custom ones.
- This delegates a fair deal of the complexity to the user side
- Implementation is not significantly simpler on the Argo CD side, albeit it will not require new config option
- Not using TLS
- :-/
Additional context
The described can currently be achieved through:
(Though it is long, ugly, and Ubuntu specific. It will also not receive any fixes if users start copy-pasting this)
apiVersion: argoproj.io/v1beta1
kind: ArgoCD
metadata:
name: example-argocd
spec:
repo:
volumeMounts:
- name: ca-trust-target
mountPath: "/etc/ssl/certs/"
readOnly: true
- name: ca-trust-source
mountPath: "/usr/local/share/ca-certificates/"
readOnly: true
volumes:
- name: ca-trust-source
projected:
sources:
- clusterTrustBundle:
name: "acme-internal-ca"
path: "acme-internal-ca.crt" # update-ca-certificates requires .crt suffix
optional: false
- clusterTrustBundle:
signerName: "kubernetes.io/kube-apiserver-serving"
labelSelector: { } # Empty matches everything, relying solely on the signerName
path: "kube-apiserver-serving.crt" # update-ca-certificates requires .crt suffix
optional: false
- name: ca-trust-target
emptyDir: { }
# Take CAs from the `argocd-ca-trust-source` volume and mix it with the distro CAs into `argocd-ca-trust-target` volumes.
# Several ubuntu-specific problems exist:
# 1. /etc/ssl/certs/ cannot be updated by `update-ca-certificates` without root - desirable in the production container.
# 2. /etc/ssl/certs/ symlinkes to /usr/local/share/ca-certificates/, so mounting one without the other is futile.
#
# All source certs are projected into the `argocd-ca-trust-source` volume that is ultimately mounted in the prod container (addresses #2).
#
# To amend content of /etc/ssl/certs/ (ca-trust-target), an init container is used:
# - it mounts `argocd-ca-trust-target` over `/etc/ssl/certs/` (addressing #1 by making it writable volume),
# and `ca-trust-source` over `/usr/local/share/ca-certificates/`,
# and amends it with user-added certs using `update-ca-certificates`.
#
# The production container is then mounted with `/etc/ssl/certs/` (`argocd-ca-trust-target`) and
# `/usr/local/share/ca-certificates/` (`argocd-ca-trust-source`) providing read-only CAs needed.
initContainers:
- name: update-ca-certificates
# TODO: Here we need to use the same image tag/hash than the prod container
image: "quay.io/argoproj/argocd:latest"
volumeMounts:
# Injected source trust anchors from clusterTrustBundles
- name: ca-trust-source
# Mounting directly to the path where update-ca-certificates expect it.
# It does not shadow certs distributed in the image, as here are user additional certificates.
mountPath: "/usr/local/share/ca-certificates/"
readOnly: true
- name: ca-trust-target
# Mounting directly to the path where update-ca-certificates expect it.
mountPath: "/etc/ssl/certs/"
command: [ "/bin/bash", "-c" ]
args:
- |
#!/usr/bin/env bash
# https://github.com/olivergondza/bash-strict-mode
set -eEuo pipefail
trap 's=$?; echo >&2 "$0: Error on line "$LINENO": $BASH_COMMAND"; exit $s' ERR
echo "User defined CA files:"
ls -l /usr/local/share/ca-certificates/
update-ca-certificates --verbose
echo "Resulting /etc/ssl/certs/"
ls -l /etc/ssl/certs/
echo "Done!"flowchart TB
ctbs@{ shape: docs, label: "ClusterTrustBundles"}
vs[(ca-trust-source<br/><i>User trust anchors</i>)]
vt[(ca-trust-target<br/><i>Product of image a user trust anchors</i>)]
ibuild[init container<br/><b>update-ca-certificates</b>]--mix in user trust anchors to /etc/ssl/certs/-->vt
vs--mounts to /usr/local/share/ca-certificates/-->ibuild
ctbs--project bundles to volume-->vs
vt--mounts to /etc/ssl/certs/ -->rs[[quay.io/argoproj/argocd<br/><b>repo-server / plugin sidecars</b>]]
vs--mounts to /usr/local/share/ca-certificates/-->rs