Skip to content
This repository was archived by the owner on Apr 27, 2024. It is now read-only.

Update MySQL Database Backup Script for Improved Functionality and Cleanup #58

Open
hamdallahjodah wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Update MySQL Database Backup Script for Improved Functionality and Cleanup #58

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f5f89d6
BACKUP_TIMESTAMP
hamdallah90 Jul 27, 2023
4a377d2
Enhance Azure support: Updated Dockerfile to include Azure CLI, modif…
hamdallah90 Nov 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ RUN apk -v --update add \
gnupg \
coreutils \
gzip \
tar \
go \
git && \
pip3 install --upgrade awscli s3cmd python-magic && \
pip3 install --upgrade awscli s3cmd python-magic azure-cli && \
rm /var/cache/apk/*

# Set Default Environment Variables
Expand All @@ -30,6 +31,8 @@ ENV CLOUD_SDK_VERSION=367.0.0
# Release commit for https://github.com/FiloSottile/age/tree/v1.0.0
ENV AGE_VERSION=552aa0a07de0b42c16126d3107bd8895184a69e7
ENV BACKUP_PROVIDER=aws
ENV MAX_FILES_TO_KEEP=0
ENV MYSQL_AUTH_PLUGIN=caching_sha2_password

# Install FiloSottile/age (https://github.com/FiloSottile/age)
RUN git clone https://filippo.io/age && \
Expand Down
100 changes: 99 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The below table lists all of the Environment Variables that are configurable for
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| BACKUP_CREATE_DATABASE_STATEMENT | **(Optional - default `false`)** Adds the `CREATE DATABASE` and `USE` statements to the MySQL backup by explicitly specifying the `--databases` flag (see [here](https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html#option_mysqldump_databases)). |
| BACKUP_ADDITIONAL_PARAMS | **(Optional)** Additional parameters to add to the `mysqldump` command. |
| BACKUP_PROVIDER | **(Optional)** The backend to use for storing the MySQL backups. Supported options are `aws` (default) or `gcp` |
| BACKUP_PROVIDER | **(Optional)** The backend to use for storing the MySQL backups. Supported options are `aws` (default), `gcp`, or `azure` |
| AWS_ACCESS_KEY_ID | **(Required for AWS Backend)** AWS IAM Access Key ID. |
| AWS_SECRET_ACCESS_KEY | **(Required for AWS Backend)** AWS IAM Secret Access Key. Should have very limited IAM permissions (see below for example) and should be configured using a Secret in Kubernetes. |
| AWS_DEFAULT_REGION | **(Required for AWS Backend)** Region of the S3 Bucket (e.g. eu-west-2). |
Expand All @@ -27,12 +27,17 @@ The below table lists all of the Environment Variables that are configurable for
| GCP_GCLOUD_AUTH | **(Required for GCP Backend)** Base64 encoded service account key exported as JSON. Example of how to generate: `base64 ~/service-key.json` |
| GCP_BUCKET_NAME | **(Required for GCP Backend)** The name of GCP GCS bucket. |
| GCP_BUCKET_BACKUP_PATH | **(Required for GCP Backend)** Path the backup file should be saved to in GCS. E.g. `/database/myblog/backups`. **Do not put a trailing / or specify the filename.** |
| AZURE_STORAGE_ACCOUNT_NAME | **(Required for Azure Backend)** Azure Storage Account name. |
| AZURE_STORAGE_ACCESS_KEY | **(Required for Azure Backend)** Azure Storage Account access key. Should be configured using a Secret in Kubernetes. |
| AZURE_CONTAINER_NAME | **(Required for Azure Backend)** The name of the Azure Blob Storage container. |
| AZURE_BACKUP_PATH | **(Required for Azure Backend)** Path the backup file should be saved to in the container. E.g. `database/myblog/backups`. **Do not put a trailing / or specify the filename.** |
| TARGET_DATABASE_HOST | **(Required)** Hostname or IP address of the MySQL Host. |
| TARGET_DATABASE_PORT | **(Optional)** Port MySQL is listening on (Default: 3306). |
| TARGET_DATABASE_NAMES | **(Required unless TARGET_ALL_DATABASES is true)** Name of the databases to dump. This should be comma seperated (e.g. `database1,database2`). |
| TARGET_ALL_DATABASES | **(Optional - default `false`)** Set to `true` to ignore `TARGET_DATABASE_NAMES` and dump all non-system databases. |
| TARGET_DATABASE_USER | **(Required)** Username to authenticate to the database with. |
| TARGET_DATABASE_PASSWORD | **(Required)** Password to authenticate to the database with. Should be configured using a Secret in Kubernetes. |
| MYSQL_AUTH_PLUGIN | **(Optional - default `caching_sha2_password`)** MySQL authentication plugin to use. Supported options: `caching_sha2_password`, `mysql_native_password`, `sha256_password`. |
| BACKUP_TIMESTAMP | **(Optional)** Date string to append to the backup filename ([date](http://man7.org/linux/man-pages/man1/date.1.html) format). Leave unset if using S3 Versioning and date stamp is not required. |
| BACKUP_COMPRESS | **(Optional)** (true/false) Enable or disable gzip backup compression - (Default False). |
| BACKUP_COMPRESS_LEVEL | **(Optional - default `9`)** Set the gzip level used for compression. |
Expand All @@ -42,6 +47,7 @@ The below table lists all of the Environment Variables that are configurable for
| SLACK_CHANNEL | **(Required if Slack enabled)** Slack Channel the WebHook is configured for. |
| SLACK_PROXY | **(Optional)** Proxy URL if Slack is behind proxy. |
| SLACK_WEBHOOK_URL | **(Required if Slack enabled)** What is the Slack WebHook URL to post to? Should be configured using a Secret in Kubernetes. |
| MAX_FILES_TO_KEEP | **(Optional - default `0`)** How many backups to keep in the S3 or GCP Bucket. (0 will keep all files) |


## Slack Integration
Expand Down Expand Up @@ -144,8 +150,98 @@ spec:
secretKeyRef:
name: my-database-backup
key: database_password
- name: MYSQL_AUTH_PLUGIN
value: "mysql_native_password"
- name: BACKUP_TIMESTAMP
value: "_%Y_%m_%d"
- name: MAX_FILES_TO_KEEP
value: "70"
- name: SLACK_ENABLED
value: "<true/false>"
- name: SLACK_CHANNEL
value: "#chatops"
- name: SLACK_WEBHOOK_URL
valueFrom:
secretKeyRef:
name: my-database-backup
key: slack_webhook_url
restartPolicy: Never
```

## Azure Backend Configuration

The below subheadings detail how to configure kubernetes-cloud-mysql-backup to backup to an Azure Blob Storage backend.

### Azure - Configuring the Storage Account

By default, kubernetes-cloud-mysql-backup performs a backup to the same path, with the same filename each time it runs. It therefore assumes that you have versioning enabled on your Azure Storage Account. A typical setup would involve Blob versioning, with a Lifecycle Management Policy.

If a timestamp is required on the backup file name, the BACKUP_TIMESTAMP Environment Variable can be set.

A Storage Account should be created in Azure, and you'll need the Storage Account name and one of the access keys. The access key should be stored securely in a Kubernetes Secret.

### Azure - Example Kubernetes Cronjob

An example of how to schedule this container in Kubernetes as a cronjob is below. This would configure a database backup to run each day at 01:00am. The Azure Storage Access Key, Target Database Password and Slack Webhook URL are stored in secrets.

```
apiVersion: v1
kind: Secret
metadata:
name: my-database-backup
type: Opaque
data:
azure_storage_access_key: <Azure Storage Access Key>
database_password: <Your Database Password>
slack_webhook_url: <Your Slack WebHook URL>
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: my-database-backup
spec:
schedule: "0 01 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: my-database-backup
image: ghcr.io/benjamin-maynard/kubernetes-cloud-mysql-backup:v2.6.0
imagePullPolicy: Always
env:
- name: AZURE_STORAGE_ACCOUNT_NAME
value: "<Your Azure Storage Account Name>"
- name: AZURE_STORAGE_ACCESS_KEY
valueFrom:
secretKeyRef:
name: my-database-backup
key: azure_storage_access_key
- name: BACKUP_PROVIDER
value: "azure"
- name: AZURE_CONTAINER_NAME
value: "<Your Azure Container Name>"
- name: AZURE_BACKUP_PATH
value: "<Your Azure Backup Path>"
- name: TARGET_DATABASE_HOST
value: "<Your Target Database Host>"
- name: TARGET_DATABASE_PORT
value: "<Your Target Database Port>"
- name: TARGET_DATABASE_NAMES
value: "<Your Target Database Name(s)>"
- name: TARGET_DATABASE_USER
value: "<Your Target Database Username>"
- name: TARGET_DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: my-database-backup
key: database_password
- name: MYSQL_AUTH_PLUGIN
value: "mysql_native_password"
- name: BACKUP_TIMESTAMP
value: "_%Y_%m_%d"
- name: MAX_FILES_TO_KEEP
value: "70"
- name: SLACK_ENABLED
value: "<true/false>"
- name: SLACK_CHANNEL
Expand Down Expand Up @@ -227,6 +323,8 @@ spec:
secretKeyRef:
name: my-database-backup
key: database_password
- name: MYSQL_AUTH_PLUGIN
value: "mysql_native_password"
- name: BACKUP_TIMESTAMP
value: "_%Y_%m_%d"
- name: SLACK_ENABLED
Expand Down
Loading