Upload on Cloudflare R2 doesn't work

[salvatorecordiano]

I'm experiencing issues with Cloudflare R2. While the data dump generates correctly, the CLI displays unusual characters during the upload. The upload to R2 completes successfully, but the command hangs afterward.

This is my command execution.

    docker run --rm \
        -e DB_DUMP_TARGET=s3://BUCKET_NAME/FOLDER_NAME \
        -e DB_SERVER=MY_IP \
        -e DB_PORT=PORT \
        -e DB_USER=USERNAME \
        -e DB_PASS=PASSWORD \
        -e DB_DUMP_INCLUDE=MY_DATABASE_NAME \
        -e AWS_ACCESS_KEY_ID=CLOUDFLARE_ACCESS_KEY_ID \
        -e AWS_SECRET_ACCESS_KEY=CLOUDFLARE_SECRET_ACCESS_KEY \
        -e AWS_ENDPOINT_URL=https://CLOUDFLARE_ID.r2.cloudflarestorage.com \
        -e AWS_DEFAULT_REGION=eeur \
        databack/mysql-backup:1.1.0 dump -v 2

Thank you.

[deitch]

Is it just waiting for the next loop? I see you set -v 2 (you can do that via environment variables as well, if that is your preference), what is in the output log? It should remove any sensitive information, but please review it yourself again before posting here.

My first thought is that it is working correctly. Unless you explicitly ask it to run just once, it sleeps until the next run time and runs again. There is a default setting for it.

[salvatorecordiano]

This is the output

time="2025-03-12T21:30:19Z" level=debug msg="starting dump"
time="2025-03-12T21:30:19Z" level=info msg="beginning dump 2025-03-12T21:30:19Z" run=06102d3a-4ee3-4618-aaa3-732b66bf728f
time="2025-03-12T21:30:22Z" level=debug msg="uploading via protocol s3 from db_backup_2025-03-12T21:30:19Z.tgz to db_backup_2025-03-12T21:30:19Z.tgz" run=06102d3a-4ee3-4618-aaa3-732b66bf728f
Error: error running command: error running command: error running dump: failed to push file: failed to upload file, operation error S3: PutObject, resolve auth scheme: resolve endpoint: endpoint rule error, Invalid region: region was not a valid DNS name.
time="2025-03-12T21:30:22Z" level=fatal msg="error running command: error running command: error running dump: failed to push file: failed to upload file, operation error S3: PutObject, resolve auth scheme: resolve endpoint: endpoint rule error, Invalid region: region was not a valid DNS name."

[deitch]

Ah, interesting. I haven't dug into the intricacies of using R2 via API. Is eeur a valid region? Or does it need to be an FQDN?

[deitch]

Can you try what is marked here in the R2 docs?

[salvatorecordiano]

I'm sorry, but it doesn't work.

I have tried the following values:

AWS_DEFAULT_REGION=auto
AWS_DEFAULT_REGION=us-east-1
AWS_DEFAULT_REGION=eeur

[deitch]

Did you try AWS_REGION (or the CLI equivalent --aws-region)?

[salvatorecordiano]

Yes, I tried but there is no difference.

[deitch]

I have not been able to reproduce it. I have an R2 account set up, and no issues.

I will try to post the setup and logs here in a little while.

[deitch]

I just ran it on my test account, here is what I got:

$ mysql-backup dump --server localhost --aws-access-key-id ${ACCESS_KEY_ID} --aws-secret-access-key ${SECRET_ACCESS_KEY} --aws-endpoint-url https://${CLOUDFLAREID}.r2.cloudflarestorage.com --aws-region auto --user user --pass pass --retention 1c --target s3://tester/backup --debug

DEBU[0000] starting dump
INFO[0000] beginning dump 2025-05-05T08:33:11Z           run=19817b7b-12da-4c60-92a3-d322c43151a1
DEBU[0000] uploading via protocol s3 from db_backup_2025-05-05T08:33:11Z.tgz to db_backup_2025-05-05T08:33:11Z.tgz  run=19817b7b-12da-4c60-92a3-d322c43151a1
DEBU[0001] completed copying 1343 bytes                  run=19817b7b-12da-4c60-92a3-d322c43151a1
INFO[0001] beginning prune                               run=19817b7b-12da-4c60-92a3-d322c43151a1
DEBU[0001] pruning target s3://tester/backup             run=19817b7b-12da-4c60-92a3-d322c43151a1
DEBU[0001] checking filename that is standard backup pattern: backup/db_backup_2025-05-05T08:33:11Z.tgz  run=19817b7b-12da-4c60-92a3-d322c43151a1
DEBU[0001] pruning 0 files from target s3://tester/backup  run=19817b7b-12da-4c60-92a3-d322c43151a1

And the upload is in my R2 account

[deitch]

I tried it as env vars as well, to be sure it is not an env vars vs CLI flags bug:

DB_DUMP_TARGET=s3://tester/backup AWS_ACCESS_KEY=$ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=$SECRET_ACCESS_KEY AWS_ENDPOINT_URL=https://${CLOUDFLAREID}.r2.cloudflarestorage.com AWS_REGION=auto DB_USER=user DB_PASS=pass mysql-backup dump --server localhost --retention 1c --debug

DEBU[0000] starting dump
INFO[0000] beginning dump 2025-05-05T08:39:40Z           run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
DEBU[0000] uploading via protocol s3 from db_backup_2025-05-05T08:39:40Z.tgz to db_backup_2025-05-05T08:39:40Z.tgz  run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
DEBU[0000] completed copying 1342 bytes                  run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
INFO[0000] beginning prune                               run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
DEBU[0000] pruning target s3://tester/backup             run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
DEBU[0000] checking filename that is standard backup pattern: backup/db_backup_2025-05-05T08:33:11Z.tgz  run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
DEBU[0000] checking filename that is standard backup pattern: backup/db_backup_2025-05-05T08:39:40Z.tgz  run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
DEBU[0000] Adding candidate file backup/db_backup_2025-05-05T08:33:11Z.tgz:  run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9
DEBU[0000] pruning 1 files from target s3://tester/backup  run=1a2cb1ac-fbb1-41d0-b759-003b17fb88f9

What errors do you get now? Can you also try it using CLI flags instead of env vars, just to be sure?

[salvatorecordiano]

Hello there.

This is my command execution. The terminal stays stuck on the last log line, and I have to manually kill the process.

The command is:

docker run --rm \
        -e DB_SERVER=${DB_SERVER} \
        -e DB_PORT=${DB_PORT} \
        -e DB_USER=${DB_USER} \
        -e DB_PASS=${DB_PASS} \
        -e DB_DUMP_INCLUDE=${DB_NAME} \
        -e SINGLE_DATABASE=true \
        -e DB_INCLUDE=$5 \
        -e DB_DUMP_TARGET=s3://${BUCKET_NAME}/$5 \
        -e AWS_REGION=auto \
        -e AWS_ACCESS_KEY_ID=${ACCESS_KEY_ID} \
        -e AWS_SECRET_ACCESS_KEY=${SECRET_ACCESS_KEY} \
        -e AWS_ENDPOINT_URL=https://${ACCOUNT_ID}.r2.cloudflarestorage.com \
        databack/mysql-backup:1.3.0 dump --debug

The output is:

time="2025-10-05T10:23:45Z" level=debug msg="starting dump"
time="2025-10-05T10:23:45Z" level=info msg="beginning dump 2025-10-05T10:23:45Z" run=4b9bf21a-0ff5-4626-9a8a-a4e9a961dfa2
time="2025-10-05T10:23:49Z" level=debug msg="uploading via protocol s3 from db_backup_2025-10-05T10:23:45Z.tgz to db_backup_2025-10-05T10:23:45Z.tgz" run=4b9bf21a-0ff5-4626-9a8a-a4e9a961dfa2
time="2025-10-05T10:23:49Z" level=debug msg="completed copying 3785297 bytes" run=4b9bf21a-0ff5-4626-9a8a-a4e9a961dfa2

After this point, the process never exits or returns control to the shell.

[deitch]

That looks correct to me. It uploaded, and now is waiting for the next time cycle to run. By default, it is once per day (frequency of 1440 mins, which is 24 hours), so it is waiting. It is marked in the docs/configuration.md document. If it is not as clear as it could be, comment here, let me know how you think it could be clearer.

In the meantime, I think it would be helpful to have an info message saying, "dump completed, upload complete, waiting x for the next run." I will add that.

[deitch]

#475 should help with that.

[salvatorecordiano]

Thank you. Is there a way to do a one-shot execution? I want to put the command in my crontab.