Skip to content

Allow user to set an annotation that will specify an existing PVC to … #4221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 30, 2025

Conversation

dsessler7
Copy link
Contributor

…be mounted to cloud backup jobs so that the backup logs can be persisted.

Checklist:

  • Have you added an explanation of what your changes do and why you'd like them to be included?
  • Have you updated or added documentation for the change, as applicable?
  • Have you tested your changes on all related environments with successful results, as applicable?
    • Have you added automated tests?

Type of Changes:

  • New feature
  • Bug fix
  • Documentation
  • Testing enhancement
  • Other

What is the current behavior (link to any open issues here)?

PGO-2578

Currently, the logs from backups that are being made to cloud repos are not persisted.

What is the new behavior (if this is a feature change)?

  • Breaking change (fix or feature that would cause existing functionality to change)

A user can now set an annotation, the value of which can point to a PVC that the cloud backups will use to persist logs to.

Other Information:

@dsessler7 dsessler7 requested review from cbandy and benjaminjb July 25, 2025 23:53
Namespace: postgresCluster.GetNamespace(),
},
}
err := errors.WithStack(r.Client.Get(ctx,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 If we check when generating the job spec, can we assume it exists here for generating the configmap?

Copy link
Contributor Author

@dsessler7 dsessler7 Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, as of right now, if the annotation is in place but the PVC doesn't exist, we still create the backup job, we just don't mount the PVC (and we create a warning event). My thought behind this is that having backups is more important that having logs from backups, so cancelling the backup job just because it can't write the logs seems like overkill. Open to arguments to the contrary though. This does make me wonder though if an event is enough of an alarm to users that something is wrong. Maybe we should log an error too? 🤔 Either way, the backup will still be created. If we don't check for the PVC here where we reconcile the config, the backup job will attempt to write the logs to a directory that doesn't exist. I can't remember if it will fail or not (I'm guessing it won't fail for the same reason as earlier)... If it doesn't fail I guess we could remove the check for the PVC if we want... 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do like to skip an API call if I can -- like, I wish we called the func to generate the jobspec before we generated the configmap, because then we could just pass the jobspec in and if we had the volume on the spec that matched the annotation on the cluster, we could assume the PVC was found.

But because we use the configmap for non-job stuff, that doesn't work. I guess we cold log into the ether if the volume wasn't there and skip a step -- but maybe another event will get through to the user.

(I also don't know the best way to get through to the user for something that's a warning but not an error.)

if logPath != "" {
global.Set("log-path", logPath)
} else {
global.Set("log-level-file", "off")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we just copy the user's options on lines 549-551?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep


func TestAddVolumeAndMountsToPod(t *testing.T) {
pod := &corev1.PodSpec{
Containers: []corev1.Container{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 Might be worth throwing in 2+ initContainers to show that this func adds to those containers as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call.

…be mounted to cloud backup jobs so that the backup logs can be persisted.
@dsessler7 dsessler7 force-pushed the backup-logs-to-pvc branch from f7a8955 to b407f60 Compare July 29, 2025 18:40
@dsessler7 dsessler7 merged commit 6a30d72 into CrunchyData:main Jul 30, 2025
20 checks passed
@dsessler7 dsessler7 deleted the backup-logs-to-pvc branch August 1, 2025 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants