Off-site backups to S3-compatible storage

Any storage that speaks the S3 API works: AWS S3, Cloudflare R2, Backblaze B2, Wasabi, Scaleway, Hetzner, or a MinIO server of your own. This article covers what to fill in, which permissions the access key needs, and what to check when the connection test fails.

If you haven't read it yet, Off-site backups to your own storage explains what gets sent and how many copies are kept.

Before you start

You need a bucket, and an access key that can write to it. Make a separate key for this, limited to that one bucket. If it ever leaks, it can't touch anything else, and you can revoke it without breaking other things.

Filling in the form

In the Off-Site Backups tab, click Configure Destination (or Edit Configuration), pick S3-Compatible Storage, and fill in:

• Endpoint URL: leave it empty for AWS S3. For anything else, enter the provider's S3 endpoint. It has to start with https:// and be reachable from the internet.
• Region: the region your bucket is in, for example eu-central-1. On AWS it has to match the bucket's region exactly.
• Bucket: just the name, without s3:// or a URL.
• Access Key ID and Secret Access Key.
• Path Prefix: optional. A folder inside the bucket, for example ghost-backups. Leave it empty to put the files at the top of the bucket.
• Backup retention (days): how many copies of each backup to keep. 0 keeps all of them.

Then click Test Connection, and Save once it passes.

Settings for common providers

• AWS S3: endpoint empty, region as shown on the bucket, for example ap-southeast-1.
• Cloudflare R2: endpoint https://<account-id>.r2.cloudflarestorage.com, region auto. Create an R2 API token with Object Read & Write access to the bucket.
• Backblaze B2: endpoint https://s3.<region>.backblazeb2.com, region as in the endpoint, for example eu-central-003. Use an application key with read and write access to the bucket.

For other providers, look for "S3 endpoint" in their docs. Both values usually appear on the bucket's settings page.

Permissions the key needs

The connection test checks that the bucket exists and the key can see it. Then it writes a small file called .magicpages-write-test-… under your path prefix, and deletes it again.

After that, every copy uploads two files, in parts because they can be large, then lists your site's earlier backups under the prefix and deletes the oldest ones.

On AWS, this policy covers all of it. Replace your-bucket-name, and ghost-backups with your path prefix:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SeeTheBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::your-bucket-name"
    },
    {
      "Sid": "WriteAndPruneBackups",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload"
      ],
      "Resource": "arn:aws:s3:::your-bucket-name/ghost-backups/*"
    }
  ]
}

Two things in this policy that are easy to get wrong:

• s3:ListBucket goes on the bucket itself, with no s3:prefix condition. The check that the bucket exists is a request on the bucket, not on a folder in it, and a prefix condition makes it fail.
• The second Resource ends in /*. Without the asterisk, it only allows writing a file literally called ghost-backups/.

Without a path prefix, the second resource is arn:aws:s3:::your-bucket-name/*. If you set retention to 0, the key never deletes anything and you can leave out s3:DeleteObject.

If your bucket encrypts with your own KMS key (SSE-KMS), the key also needs kms:GenerateDataKey and kms:Decrypt on that KMS key. Uploads in parts need both.

If versioning is on

Deleting an old backup from a versioned bucket only hides it. The older versions stay, and you keep paying for them. Add a lifecycle rule that expires noncurrent versions after a few days.

When the test fails

"Failed to connect to bucket …" means the first check failed: we couldn't see the bucket at all. Usually one of these:

• The key doesn't have s3:ListBucket on the bucket, or has it with a prefix condition.
• The region doesn't match the bucket's region.
• The bucket name has a typo, or includes s3:// or a path.
• The endpoint is wrong, or is set for an AWS bucket. For AWS, leave it empty.
• The access key is wrong, inactive, or belongs to a different account.

"Cannot write to bucket …" means we could see the bucket but not write to it. The key is missing s3:PutObject for the path prefix, or the resource is missing its /*.

A message about a blocked address means the endpoint points at a private network. It has to be a public https:// address.

If none of that fits, email help@magicpages.co with the bucket name, the region and your provider. I can see the exact error on our side. Never send the secret key.