-
Notifications
You must be signed in to change notification settings - Fork 5k
17648 Fix to schedule the ingress-dns pod on the minikube primary nod… #21132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…e in multinode clusters
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: fbyrne The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Hi @fbyrne. Thanks for your PR. I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Can one of the admins verify this patch? |
thank you @fbyrne sorry that couldnt get your other PR in my attention, do you mind explaining how does this fix it ? |
Hi @medyagh , the fix does two things
Ill post the before/after now. |
Notice before the fix, the ingress dns pod can be launched on a random node.
Notice after the fix, the ingress dns pod is launched on the primary/master node.
Full before output minikube on HEAD (7498245) via v1.24.0 on ☁️
at 13:26:01 ❯ git log --oneline
7498245a9 (HEAD, upstream/master, upstream/HEAD, origin/master, origin/HEAD, master) replace spinner lib to upstream (#21115)
2629cf7b6 Build(deps): Bump k8s.io/component-base from 0.33.2 to 0.33.3 (#21114)
320095eee Update auto-generated docs and translations (#21121)
70fc56d67 Build(deps): Bump actions/setup-go from 5.2.0 to 5.5.0 (#21116)
9e4cd659c Update auto-generated docs and translations (#21120)
d13f7bbe6 Build(deps): Bump peter-evans/create-pull-request from 7.0.6 to 7.0.8 (#21118)
958ecac9d Refactor table rendering (#20893)
d850b6921 bump winget gihtub action (#21119)
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:27:38 ❯ make clean
rm -rf /home/fergus/src/github/fbyrne/minikube/out
rm -f pkg/minikube/assets/assets.go
rm -f pkg/minikube/translate/translations.go
rm -rf ./vendor
rm -rf /tmp/tmp.*.minikube_*
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:27:54 ❯ make
go build -tags "" -ldflags="-X k8s.io/minikube/pkg/version.version=v1.36.0 -X k8s.io/minikube/pkg/version.isoVersion=v1.36.0-1752940814-21089 -X k8s.io/minikube/pkg/version.gitCommitID="7498245a96aa3e9be815f5f676aad34e9171f6f7-dirty" -X k8s.io/minikube/pkg/version.storageProvisionerVersion=v5" -o out/minikube k8s.io/minikube/cmd/minikube
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:28:10 ❯ out/minikube delete --purge --all
🔥 Deleting "minikube" in docker ...
🔥 Removing /home/fergus/.minikube/machines/minikube ...
💀 Removed all traces of the "minikube" cluster.
🔥 Successfully deleted all profiles
💀 Successfully purged minikube directory located at - [/home/fergus/.minikube]
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:28:18 ❯ out/minikube -p before-bugfix-17648 -n 3 start
😄 [before-bugfix-17648] minikube v1.36.0 on Ubuntu 24.04
✨ Automatically selected the docker driver. Other choices: kvm2, ssh
📌 Using Docker driver with root privileges
👍 Starting "before-bugfix-17648" primary control-plane node in "before-bugfix-17648" cluster
🚜 Pulling base image v0.0.47-1752142599-21053 ...
💾 Downloading Kubernetes v1.33.2 preload ...
> preloaded-images-k8s-v18-v1...: 348.11 MiB / 348.11 MiB 100.00% 24.14 M
> gcr.io/k8s-minikube/kicbase...: 485.70 MiB / 485.70 MiB 100.00% 12.98 M
🔥 Creating docker container (CPUs=2, Memory=5166MB) ...
🐳 Preparing Kubernetes v1.33.2 on Docker 28.3.2 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring CNI (Container Networking Interface) ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
👍 Starting "before-bugfix-17648-m02" worker node in "before-bugfix-17648" cluster
🚜 Pulling base image v0.0.47-1752142599-21053 ...
🔥 Creating docker container (CPUs=2, Memory=5166MB) ...
🌐 Found network options:
▪ NO_PROXY=192.168.49.2
🐳 Preparing Kubernetes v1.33.2 on Docker 28.3.2 ...
▪ env NO_PROXY=192.168.49.2
🔎 Verifying Kubernetes components...
👍 Starting "before-bugfix-17648-m03" worker node in "before-bugfix-17648" cluster
🚜 Pulling base image v0.0.47-1752142599-21053 ...
🔥 Creating docker container (CPUs=2, Memory=5166MB) ...
🌐 Found network options:
▪ NO_PROXY=192.168.49.2,192.168.49.3
🐳 Preparing Kubernetes v1.33.2 on Docker 28.3.2 ...
▪ env NO_PROXY=192.168.49.2
▪ env NO_PROXY=192.168.49.2,192.168.49.3
🔎 Verifying Kubernetes components...
🏄 Done! kubectl is now configured to use "before-bugfix-17648" cluster and "default" namespace by default
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️ took 1m27s
at 13:30:03 ❯ out/minikube -p before-bugfix-17648 addons enable ingress
💡 ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
▪ Using image registry.k8s.io/ingress-nginx/controller:v1.12.3
▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.4
▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.4
🔎 Verifying ingress addon...
🌟 The 'ingress' addon is enabled
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️ took 9s
at 13:30:32 ❯ out/minikube -p before-bugfix-17648 addons enable ingress-dns
💡 ingress-dns is an addon maintained by minikube. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
▪ Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3
🌟 The 'ingress-dns' addon is enabled
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:30:49 ❯ out/minikube -p before-bugfix-17648 kubectl -- describe pod kube-ingress-dns-minikube --namespace=kube-system
> kubectl.sha256: 64 B / 64 B [-------------------------] 100.00% ? p/s 0s
> kubectl: 57.34 MiB / 57.34 MiB [------------] 100.00% 38.74 MiB p/s 1.7s
Name: kube-ingress-dns-minikube
Namespace: kube-system
Priority: 0
Service Account: minikube-ingress-dns
Node: before-bugfix-17648-m02/192.168.49.3
Start Time: Fri, 25 Jul 2025 13:30:49 +0100
Labels: app=minikube-ingress-dns
app.kubernetes.io/part-of=kube-system
Annotations: <none>
Status: Running
IP: 192.168.49.3
IPs:
IP: 192.168.49.3
Containers:
minikube-ingress-dns:
Container ID: docker://f95f54399187e940d012df5384d8a125f36a0de8d4c9b1cf9bcafaf71e111fb4
Image: gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c
Image ID: docker-pullable://gcr.io/k8s-minikube/minikube-ingress-dns@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c
Port: 53/UDP
Host Port: 53/UDP
State: Running
Started: Fri, 25 Jul 2025 13:30:55 +0100
Ready: True
Restart Count: 0
Environment:
DNS_PORT: 53
POD_IP: (v1:status.podIP)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-kb447 (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-kb447:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
Optional: false
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 23s default-scheduler Successfully assigned kube-system/kube-ingress-dns-minikube to before-bugfix-17648-m02
Normal Pulling 22s kubelet Pulling image "gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c"
Normal Pulled 17s kubelet Successfully pulled image "gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c" in 5.586s (5.586s including waiting). Image size: 190556102 bytes.
Normal Created 17s kubelet Created container: minikube-ingress-dns
Normal Started 17s kubelet Started container minikube-ingress-dns
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️ took 2s
at 13:31:12 ❯ kubectl apply -f https://raw.githubusercontent.com/kubernetes/minikube/master/deploy/addons/ingress-dns/example/example.yaml
deployment.apps/hello-world-app created
ingress.networking.k8s.io/example-ingress created
service/hello-world-app created
service/hello-world-app created
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:31:56 ❯ out/minikube -p before-bugfix-17648 kubectl -- get po
NAME READY STATUS RESTARTS AGE
hello-world-app-7d9564db4-f6ffv 1/1 Running 0 19s
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:32:15 ❯ out/minikube -p before-bugfix-17648 kubectl -- get ing --all-namespaces
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
kube-system example-ingress nginx hello-john.test,hello-jane.test 80 35s
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:32:31 ❯ nslookup hello-john.test $(out/minikube -p before-bugfix-17648 ip)
;; communications error to 192.168.49.2#53: connection refused
;; communications error to 192.168.49.2#53: connection refused
;; communications error to 192.168.49.2#53: connection refused
;; no servers could be reached
minikube on HEAD (7498245) [!] via v1.24.0 on ☁️
at 13:32:51 ❯ nslookup hello-jane.test $(out/minikube -p before-bugfix-17648 ip)
;; communications error to 192.168.49.2#53: connection refused
;; communications error to 192.168.49.2#53: connection refused
;; communications error to 192.168.49.2#53: connection refused
;; no servers could be reached Full after output minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:36:37 ❯ git log --oneline -n 5
58bc2bd16 (HEAD -> bugfix-17648, origin/bugfix-17648) 17648 Fix to schedule the ingress-dns pod on the minikube primary node in multinode clusters
7498245a9 (upstream/master, upstream/HEAD, origin/master, origin/HEAD, master) replace spinner lib to upstream (#21115)
2629cf7b6 Build(deps): Bump k8s.io/component-base from 0.33.2 to 0.33.3 (#21114)
320095eee Update auto-generated docs and translations (#21121)
70fc56d67 Build(deps): Bump actions/setup-go from 5.2.0 to 5.5.0 (#21116)
minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:36:40 ❯ make clean
rm -rf /home/fergus/src/github/fbyrne/minikube/out
rm -f pkg/minikube/assets/assets.go
rm -f pkg/minikube/translate/translations.go
rm -rf ./vendor
rm -rf /tmp/tmp.*.minikube_*
minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:36:46 ❯ make
go build -tags "" -ldflags="-X k8s.io/minikube/pkg/version.version=v1.36.0 -X k8s.io/minikube/pkg/version.isoVersion=v1.36.0-1752940814-21089 -X k8s.io/minikube/pkg/version.gitCommitID="58bc2bd16d03f6a9f0bea0abc55166132e65bd2e-dirty" -X k8s.io/minikube/pkg/version.storageProvisionerVersion=v5" -o out/minikube k8s.io/minikube/cmd/minikube
minikube on bugfix-17648 [!] via v1.24.0 on ☁️ took 3s
at 13:36:51 ❯ out/minikube delete --purge --all
🔥 Deleting "before-bugfix-17648" in docker ...
🔥 Removing /home/fergus/.minikube/machines/before-bugfix-17648 ...
🔥 Removing /home/fergus/.minikube/machines/before-bugfix-17648-m02 ...
🔥 Removing /home/fergus/.minikube/machines/before-bugfix-17648-m03 ...
💀 Removed all traces of the "before-bugfix-17648" cluster.
🔥 Successfully deleted all profiles
💀 Successfully purged minikube directory located at - [/home/fergus/.minikube]
📌 Kicbase images have not been deleted. To delete images run:
▪ docker rmi gcr.io/k8s-minikube/kicbase-builds:v0.0.47-1752142599-21053
minikube on bugfix-17648 [!] via v1.24.0 on ☁️ took 6s
at 13:37:27 ❯ out/minikube -p after-bugfix-17648 -n 3 start
😄 [after-bugfix-17648] minikube v1.36.0 on Ubuntu 24.04
✨ Automatically selected the docker driver. Other choices: kvm2, ssh
📌 Using Docker driver with root privileges
👍 Starting "after-bugfix-17648" primary control-plane node in "after-bugfix-17648" cluster
🚜 Pulling base image v0.0.47-1752142599-21053 ...
💾 Downloading Kubernetes v1.33.2 preload ...
> preloaded-images-k8s-v18-v1...: 348.11 MiB / 348.11 MiB 100.00% 34.66 M
🔥 Creating docker container (CPUs=2, Memory=5166MB) ...
🐳 Preparing Kubernetes v1.33.2 on Docker 28.3.2 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring CNI (Container Networking Interface) ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
👍 Starting "after-bugfix-17648-m02" worker node in "after-bugfix-17648" cluster
🚜 Pulling base image v0.0.47-1752142599-21053 ...
🔥 Creating docker container (CPUs=2, Memory=5166MB) ...
🌐 Found network options:
▪ NO_PROXY=192.168.49.2
🐳 Preparing Kubernetes v1.33.2 on Docker 28.3.2 ...
▪ env NO_PROXY=192.168.49.2
🔎 Verifying Kubernetes components...
👍 Starting "after-bugfix-17648-m03" worker node in "after-bugfix-17648" cluster
🚜 Pulling base image v0.0.47-1752142599-21053 ...
🔥 Creating docker container (CPUs=2, Memory=5166MB) ...
🌐 Found network options:
▪ NO_PROXY=192.168.49.2,192.168.49.3
🐳 Preparing Kubernetes v1.33.2 on Docker 28.3.2 ...
▪ env NO_PROXY=192.168.49.2
▪ env NO_PROXY=192.168.49.2,192.168.49.3
🔎 Verifying Kubernetes components...
🏄 Done! kubectl is now configured to use "after-bugfix-17648" cluster and "default" namespace by default
minikube on bugfix-17648 [!] via v1.24.0 on ☁️ took 52s
at 13:38:38 ❯ out/minikube -p after-bugfix-17648 addons enable ingress
💡 ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.4
▪ Using image registry.k8s.io/ingress-nginx/controller:v1.12.3
▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.4
🔎 Verifying ingress addon...
🌟 The 'ingress' addon is enabled
minikube on bugfix-17648 [!] via v1.24.0 on ☁️ took 9s
at 13:39:13 ❯ out/minikube -p after-bugfix-17648 addons enable ingress-dns
💡 ingress-dns is an addon maintained by minikube. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
▪ Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3
🌟 The 'ingress-dns' addon is enabled
minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:43:58 ❯ out/minikube -p after-bugfix-17648 kubectl -- describe pod kube-ingress-dns-minikube --namespace=kube-system
Name: kube-ingress-dns-minikube
Namespace: kube-system
Priority: 0
Service Account: minikube-ingress-dns
Node: after-bugfix-17648/192.168.49.2
Start Time: Fri, 25 Jul 2025 13:39:20 +0100
Labels: app=minikube-ingress-dns
app.kubernetes.io/part-of=kube-system
Annotations: <none>
Status: Running
IP: 192.168.49.2
IPs:
IP: 192.168.49.2
Containers:
minikube-ingress-dns:
Container ID: docker://be746a662af62eef900f0d4a1292213adf483c038c5b1dae7046baca380faa00
Image: gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c
Image ID: docker-pullable://gcr.io/k8s-minikube/minikube-ingress-dns@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c
Port: 53/UDP
Host Port: 53/UDP
State: Running
Started: Fri, 25 Jul 2025 13:39:26 +0100
Ready: True
Restart Count: 0
Environment:
DNS_PORT: 53
POD_IP: (v1:status.podIP)
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-dbpqx (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-dbpqx:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
Optional: false
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: minikube.k8s.io/primary=true
Tolerations: node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 4m44s default-scheduler Successfully assigned kube-system/kube-ingress-dns-minikube to after-bugfix-17648
Normal Pulling 4m44s kubelet Pulling image "gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c"
Normal Pulled 4m38s kubelet Successfully pulled image "gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3@sha256:4211a1de532376c881851542238121b26792225faa36a7b02dccad88fd05797c" in 5.364s (5.364s including waiting). Image size: 190556102 bytes.
Normal Created 4m38s kubelet Created container: minikube-ingress-dns
Normal Started 4m38s kubelet Started container minikube-ingress-dns
minikube on bugfix-17648 [!] via v1.24.0 on ☁️ took 2s
at 13:39:53 ❯ out/minikube -p after-bugfix-17648 kubectl -- apply -f https://raw.githubusercontent.com/kubernetes/minikube/master/deploy/addons/ingress-dns/example/example.yaml
deployment.apps/hello-world-app created
ingress.networking.k8s.io/example-ingress created
service/hello-world-app created
service/hello-world-app created
minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:40:11 ❯ out/minikube -p after-bugfix-17648 kubectl -- get po
NAME READY STATUS RESTARTS AGE
hello-world-app-7d9564db4-v7nf8 1/1 Running 0 21s
minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:40:32 ❯ out/minikube -p after-bugfix-17648 kubectl -- get ing --all-namespaces
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
kube-system example-ingress nginx hello-john.test,hello-jane.test 192.168.49.2 80 45s
minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:40:56 ❯ nslookup hello-john.test $(out/minikube -p after-bugfix-17648 ip)
Server: 192.168.49.2
Address: 192.168.49.2#53
Non-authoritative answer:
Name: hello-john.test
Address: 192.168.49.2
Name: hello-john.test
Address: 192.168.49.2
minikube on bugfix-17648 [!] via v1.24.0 on ☁️
at 13:41:12 ❯ nslookup hello-jane.test $(out/minikube -p after-bugfix-17648 ip)
Server: 192.168.49.2
Address: 192.168.49.2#53
Non-authoritative answer:
Name: hello-jane.test
Address: 192.168.49.2
Name: hello-jane.test
Address: 192.168.49.2 |
@medyagh bump on this |
/cc @kubernetes/sig-cluster-lifecycle |
@fbyrne: GitHub didn't allow me to request PR reviews from the following users: fbyrne. Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Re open of PR #17649
fixes #17648
Add nodeSelector for primary minikube node to ingress-dns pod template.
Add toleration for kubernetes master role.