There are two ways to go about backing up a MongoDB database - either taking a snapshot of the underling filesystem or taking a binary dump of the database. For various reasons we needed to go the binary dump route. We use AWS so the logical place to store the dump was in a dedicated S3 bucket. But how to get it there easily? I’ve used S3fuse in the past and that seemed like a decent option. S3fuse mounts a bucket on your local filesystem so you can move objects in and out. Unfortunately it requires some configuration as well as compiling the driver (some attempts at creating a Debian package have gotten hung up with openssl). I also looked at using the great tools that Amazon provides, but it was a tremendous (for me) amount of work for a seemingly simple task.
Then I stumbled upon s3cmd which is available in the Debian repos. It’s quite simple to use, well documented, and does exactly what I was looking for.
The basic workflow is:
Create the bucket via the AWS console or via aws-cli.
Create a user (IAM) via the AWS console or cli . The user needs read, write, and list access to the S3 bucket.
apt-get install s3cmd
Get the access and secret keys for the new user
Here you’ll need to enter the keys as well as an encryption key of your choosing.
Create a dump of the MongoDB database or collection. See MongoDB’s website for options.
Use s3cmd to move the dump to S3:
s3cmd put --recursive dump/%COLLECTION% s3://%BUCKET%/
That’s it. Easily put into a script to run periodically. You’ll want to create a folder with a timestamp or enable versioning on the bucket.