-
Notifications
You must be signed in to change notification settings - Fork 416
Open
Labels
lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.Indicates that an issue or PR should not be auto-closed due to staleness.
Description
A pod took a core dump:
$ oc debug node/$NODE -t
[...]
# ls -l /host/var/lib/systemd/coredump/
total 115824
-rw-r-----+ 1 root root 118596950 Feb 28 16:25 'core.Default\x20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4'
Leave the debug pod up so that we can download the file through the debug pod:
# while true; do echo "hold this pod"; sleep 8; done
Find the debug pod:
$ oc get pods --field-selector=status.phase==Running --all-namespaces | grep debug
admin2-namespace worker1-debug 1/1 Running 0 3m54s
Downloading the core dump directory works although with a strange directory structure:
$ oc cp --namespace admin2-namespace worker1-debug:/host/var/lib/systemd/coredump/ coredumps
tar: Removing leading `/' from member names
$ tree coredumps
coredumps
└── core.Default
└── x20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4
However, downloading the file fails trying with various quoting and escape (client is macOS but same problem on Windows):
- No quoting:
$ oc cp --namespace admin2-namespace worker1-debug:/host/var/lib/systemd/coredump/core.Default\x20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4 core.dmp.lz4 tar: Removing leading `/' from member names tar: /host/var/lib/systemd/coredump/core.Defaultx20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors
- Single quoting:
$ oc cp --namespace admin2-namespace 'worker1-debug:/host/var/lib/systemd/coredump/core.Default\x20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4' core.dmp.lz4 tar: Removing leading `/' from member names tar: /host/var/lib/systemd/coredump/core.Default/x20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors
- Single quoting and converting
\x20
to a space:$ oc cp --namespace admin2-namespace 'worker1-debug:/host/var/lib/systemd/coredump/core.Default Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4' core.dmp.lz4 tar: Removing leading `/' from member names tar: /host/var/lib/systemd/coredump/core.Default Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors
- Trying a wildcard:
$ oc cp --namespace admin2-namespace 'worker1-debug:/host/var/lib/systemd/coredump/core.Default*.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4' core.lz4 tar: Removing leading `/' from member names tar: /host/var/lib/systemd/coredump/core.Default*.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors
- Double quotes doesn't return an error but doesn't download the file:
$ oc cp --namespace admin2-namespace "worker1-debug:/host/var/lib/systemd/coredump/core.Default\x20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4" core.dmp.lz4 $ ls core.dmp.lz4 ls: core.dmp.lz4: No such file or directory
Moving the file using the debug node works but isn't great because now utilities like coredumpctl
think the core dump is gone:
Debug node:
$ mv '/host/var/lib/systemd/coredump/core.Default\x20Executo.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4' '/host/var/lib/systemd/coredump/core.DefaultExecuto.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4'
Client:
$ oc cp --namespace admin2-namespace 'worker1-debug:/host/var/lib/systemd/coredump/core.DefaultExecuto.1001210000.06a2f2e0997c46b2b50bdac3c25ab072.831941.1677601537000000.lz4' core.dmp.lz4
tar: Removing leading `/' from member names
$ ls -l core.dmp.lz4
-rw-r--r-- 1 kevin wheel 118596950 Feb 28 13:29 core.dmp.lz4
oc
version:
$ oc version
Client Version: 4.11.0-0.nightly-2022-05-18-171831
Kustomize Version: v4.5.4
Server Version: 4.11.21
Kubernetes Version: v1.24.6+5658434
Metadata
Metadata
Assignees
Labels
lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.Indicates that an issue or PR should not be auto-closed due to staleness.