Skip to content

Custom S3-Compatible Storage Support other than localhost #219

@kesuskim

Description

@kesuskim

Hello. I was trying to use litestream with custom minio storage, but was not working- so I checked why, and figured it out there was some regex match check on parsing s3 host.

https://github.com/benbjohnson/litestream/blob/main/s3/replica_client.go#L684-L701

	if a := localhostRegex.FindStringSubmatch(host); a != nil {
		bucket, region = a[1], "us-east-1"
		scheme, endpoint = "http", "localhost"
	} else if a := backblazeRegex.FindStringSubmatch(host); a != nil {
		bucket, region = a[1], a[2]
		endpoint = fmt.Sprintf("s3.%s.backblazeb2.com", region)
	} else if a := filebaseRegex.FindStringSubmatch(host); a != nil {
		bucket, endpoint = a[1], "s3.filebase.com"
	} else if a := digitalOceanRegex.FindStringSubmatch(host); a != nil {
		bucket, region = a[1], a[2]
		endpoint = fmt.Sprintf("%s.digitaloceanspaces.com", region)
	} else if a := linodeRegex.FindStringSubmatch(host); a != nil {
		bucket, region = a[1], a[2]
		endpoint = fmt.Sprintf("%s.linodeobjects.com", region)
	} else {
		bucket = host
		forcePathStyle = false
	}

As for I know, forcePathStyle should be true in s3-compatible storage like minio, but it seems if the url does not contain localhost, it cannot be used.

For the REPLICA_URL I used this value: s3://backup.s3.mysite.com/litestream-testing/file.db.

Metadata

Metadata

Assignees

Labels

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions