Skip to content

Commit bb3c767

Browse files
justin-leskojustin-lesko
authored
Add new EKS Logging documentation (#20847)
* Add new EKS Logging documentation * Update docs * Update docs * Update operator CR values * Temporarily remove tabs * Updates * More updates * Address comments
1 parent 09b73fa commit bb3c767

File tree

1 file changed

+157
-4
lines changed

1 file changed

+157
-4
lines changed

‎eks_fargate/README.md

Lines changed: 157 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -552,7 +552,7 @@ spec:
552552
- name: "<CONTAINER_NAME>"
553553
image: "<CONTAINER_IMAGE>"
554554
555-
# Running the Agent as a side-car
555+
# Running the Agent as a sidecar
556556
- name: datadog-agent
557557
image: gcr.io/datadoghq/agent:7
558558
env:
@@ -689,7 +689,7 @@ spec:
689689
- name: "<CONTAINER_NAME>"
690690
image: "<CONTAINER_IMAGE>"
691691
692-
# Running the Agent as a side-car
692+
# Running the Agent as a sidecar
693693
- name: datadog-agent
694694
image: gcr.io/datadoghq/agent:7
695695
env:
@@ -852,7 +852,7 @@ spec:
852852
- name: "<CONTAINER_NAME>"
853853
image: "<CONTAINER_IMAGE>"
854854
855-
# Running the Agent as a side-car
855+
# Running the Agent as a sidecar
856856
- name: datadog-agent
857857
image: gcr.io/datadoghq/agent:7
858858
env:
@@ -881,6 +881,158 @@ spec:
881881

882882
## Log collection
883883

884+
### Collecting logs from EKS on Fargate natively with the Agent
885+
886+
Monitor EKS Fargate logs using the Datadog Agent to collect logs from the kubelet and ship them to Datadog.
887+
888+
1. The most convenient way to enable native kubelet logging is through the Cluster Agent's Admission Controller sidecar injection feature. When configured, all subsequent injected Agent containers automatically have kubelet logging enabled. This feature can also be configured manually in your Application's manifest.
889+
890+
<!-- xxx tabs xxx -->
891+
<!-- xxx tab "Enable Logging - Datdog Operator" xxx -->
892+
893+
Set the `DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_KUBELET_API_LOGGING_ENABLED` Cluster Agent environment variable to `true`, so newly injected Agent containers will have kubelet logging enabled.
894+
895+
```yaml
896+
apiVersion: datadoghq.com/v2alpha1
897+
kind: DatadogAgent
898+
metadata:
899+
name: datadog
900+
namespace: datadog
901+
spec:
902+
overrides:
903+
clusterAgent:
904+
env:
905+
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_KUBELET_API_LOGGING_ENABLED
906+
value: "true"
907+
```
908+
909+
<!-- xxz tab xxx -->
910+
<!-- xxx tab "Enable Logging - Helm" xxx -->
911+
912+
Set the `DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_KUBELET_API_LOGGING_ENABLED` Cluster Agent environment variable to `true`, so newly injected Agent containers will have kubelet logging enabled.
913+
914+
```yaml
915+
clusterAgent:
916+
env:
917+
- name: DD_ADMISSION_CONTROLLER_AGENT_SIDECAR_KUBELET_API_LOGGING_ENABLED
918+
value: true
919+
```
920+
921+
<!-- xxz tab xxx -->
922+
<!-- xxx tab "Enable Logging - Manual" xxx -->
923+
924+
To enable Agent logging manually, you must:
925+
1. Attach an [emptyDir][29] volume to your pod and mount it inside the Agent container. This prevents duplicate logs should the Agent container restart.
926+
2. Set `DD_LOGS_ENABLED` to `"true"` - this instructs the Agent to collect logs.
927+
3. Set `DD_LOGS_CONFIG_RUN_PATH` to the emptyDir mount path.
928+
4. Set `DD_LOGS_CONFIG_K8S_CONTAINER_USE_KUBELET_API` to `"true"` - this instructs the Agent on which logging method to use.
929+
930+
```yaml
931+
apiVersion: apps/v1
932+
kind: Deployment
933+
spec:
934+
#(...)
935+
template:
936+
#(...)
937+
spec:
938+
# Empty dir to keep track of logging timestamps in case of agent restart
939+
volumes:
940+
- name: agent-option
941+
emptyDir: {}
942+
containers:
943+
#(...)
944+
# Running the Agent as a sidecar
945+
- name: datadog-agent
946+
image: gcr.io/datadoghq/agent:7
947+
# Mount the empty dir to the Agent
948+
volumeMounts:
949+
- name: agent-option
950+
mountPath: /opt/datadog-agent/run
951+
readOnly: false
952+
env:
953+
#(...)
954+
- name: DD_LOGS_ENABLED
955+
value: "true"
956+
- name: DD_LOGS_CONFIG_K8S_CONTAINER_USE_KUBELET_API
957+
value: "true"
958+
- name: DD_LOGS_CONFIG_RUN_PATH
959+
value: "/opt/datadog-agent/run"
960+
resources:
961+
requests:
962+
memory: "256Mi"
963+
cpu: "200m"
964+
limits:
965+
memory: "256Mi"
966+
cpu: "200m"
967+
```
968+
969+
<!-- xxz tab xxx -->
970+
<!-- xxz tabs xxx -->
971+
972+
2. You can configure the injected Agents to automatically collect logs for all containers by enabling `CONTAINER_COLLECT_ALL`. Alternatively, logs can be filtered through the standard Kubernetes [Autodiscovery annotations](https://docs.datadoghq.com/containers/kubernetes/log/?tab=helm#autodiscovery-annotations).
973+
974+
<!-- xxx tabs xxx -->
975+
<!-- xxx tab "Configure Logging - Datdog Operator" xxx -->
976+
977+
```yaml
978+
#(...)
979+
spec:
980+
#(...)
981+
features:
982+
admissionController:
983+
agentSidecarInjection:
984+
#(...)
985+
profiles:
986+
- env:
987+
# Collect all container logs
988+
- name: DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL
989+
value: "true"
990+
```
991+
992+
<!-- xxz tab xxx -->
993+
<!-- xxx tab "Configure Logging - Helm" xxx -->
994+
995+
```yaml
996+
clusterAgent:
997+
admissionController:
998+
agentSidecarInjection:
999+
# (...)
1000+
profiles:
1001+
- env:
1002+
# Collect all container logs
1003+
- name: DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL
1004+
value: "true"
1005+
```
1006+
1007+
<!-- xxz tab xxx -->
1008+
<!-- xxx tab "Configure Logging - Manual" xxx -->
1009+
1010+
```yaml
1011+
apiVersion: apps/v1
1012+
kind: Deployment
1013+
metadata:
1014+
name: "<APPLICATION_NAME>"
1015+
namespace: default
1016+
spec:
1017+
#(...)
1018+
template:
1019+
#(...)
1020+
spec:
1021+
#(...)
1022+
containers:
1023+
# Running the Agent as a sidecar
1024+
- name: datadog-agent
1025+
image: gcr.io/datadoghq/agent:7
1026+
env:
1027+
# Collect all container logs
1028+
- name: DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL
1029+
value: "true"
1030+
#(...)
1031+
```
1032+
1033+
<!-- xxz tab xxx -->
1034+
<!-- xxz tabs xxx -->
1035+
8841036
### Collecting logs from EKS on Fargate with Fluent Bit
8851037

8861038
Monitor EKS Fargate logs by using [Fluent Bit][14] to route EKS logs to CloudWatch Logs and the [Datadog Forwarder][15] to route logs to Datadog.
@@ -936,7 +1088,7 @@ spec:
9361088
- name: "<CONTAINER_NAME>"
9371089
image: "<CONTAINER_IMAGE>"
9381090
939-
# Running the Agent as a side-car
1091+
# Running the Agent as a sidecar
9401092
- name: datadog-agent
9411093
image: gcr.io/datadoghq/agent:7
9421094
# (...)
@@ -1106,3 +1258,4 @@ Additional helpful documentation, links, and articles:
11061258
[26]: https://kubernetes.io/docs/concepts/configuration/secret/
11071259
[27]: https://helm.sh/docs/intro/install/
11081260
[28]: https://docs.datadoghq.com/tracing/trace_collection/proxy_setup/apigateway
1261+
[29]: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir

0 commit comments

Comments
 (0)