-
Notifications
You must be signed in to change notification settings - Fork 1k
Description
What happened:
The member cluster client does not have a configured request timeout.
When the creation or request of a member cluster client takes too long, karmada controllers(e.g. execution-controller) has no mechanism to cancel or timeout the request:
clusterObj, err := dynamicClusterClient.DynamicClientSet.Resource(gvr).Namespace(desireObj.GetNamespace()).Create(ctx, desireObj, metav1.CreateOptions{})
As a result, if a member cluster does not respond, the controller’s processing flow is blocked, causing subsequent request events to remain unhandled.
What you expected to happen:
Add a timeout control mechanism for the member cluster client.
When a controller request to a member cluster does not respond within the timeout period, the controller should stop waiting and continue to process other request events.
How to reproduce it (as minimally and precisely as possible):
- Prepare two member clusters: member1 and member2.
- Inject a 300s response delay into member2.
- When deploying workloads:
Requests to member1 respond immediately.
Requests to member2 hang for 5 minutes due to the lack of timeout control.
Anything else we need to know?:
The member cluster clientin Karmada is implemented as a DynamicClient
. DynamicClient
does not set a default timeout internally.
We can refer to the Kubernetes DiscoveryClient
, which uses a default timeout of 32s
.
It is recommended to define a default timeout constant in the membercluster_client file and then pass this timeout value to clusterConfig inside the BuildClusterConfig function.
const (
defaultTimeout = 32 * time.Second
)
clusterConfig := &rest.Config{
BearerToken: string(token),
Host: apiEndpoint,
Timeout: defaultTimeout,
}
Environment:
- Karmada version: alpha
- kubectl-karmada or karmadactl version (the result of
kubectl-karmada version
orkarmadactl version
): - Others:
Metadata
Metadata
Assignees
Labels
Type
Projects
Status