Skip to content

volumeClaimTemplate and exceeded quota for PVC (and requeues ?) #7672

@vdemeester

Description

@vdemeester

In case of using volumeClaimTemplate in a PipelineRun (or TaskRun), the controller is responsible to create the PVC.
If there is resource quota in place in the namespace, and if the quota are exceeded, the creation of the PVC will fail, and thus, the PipelineRun (or TaskRun) will fail immediately.

Failed to create PVC for PipelineRun caas-ocp-tf-prod/caas-ocp-gcp-1031-terraform-preprod-vjpcw correctly: PVC creation error: failed to create PVC pvc-f4c248dec1: persistentvolumeclaims "pvc-f4c248dec1" is forbidden: exceeded quota: caas-ocp-tf-prod-compute-resources, requested: persistentvolumeclaims=1, used: persistentvolumeclaims=20, limited: persistentvolumeclaims=20

This is a slightly inconsistent behavior compared to other objects and I feel we could do a better job here. Nothing should prevent us to requeue the PipelineRun or TaskRun (and pray for the quota to be free now).

What we could do:

  • Change the error (permanent today) to a non permanent error
  • Tell the controller to requeue that object in a minute or so (we could make this configurable, maybe ?)
  • Either define a configurable timeout for this, or let the PipelineRun/TaskRun timeout do its job here. Something to check is, do we start the timeout counter before or after trying to create the PVC, because depending on that, we may have to handle timeouts differently.

cc @tektoncd/core-maintainers

Metadata

Metadata

Assignees

Labels

kind/featureCategorizes issue or PR as related to a new feature.kind/questionIssues or PRs that are questions around the project or a particular featurepriority/important-soonMust be staffed and worked on either currently, or very soon, ideally in time for the next release.

Type

No type

Projects

Status

Todo

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions