Backing up your own server: what really breaks

Why a backup that runs is not enough

Backing up a self-hosted server comes down to one well-known rule: three copies, two media, one copy kept apart. What really protects you is adding restores that have actually been tried. Everything else is execution detail, but that is where data gets lost.

Our server runs a dozen services in Docker containers, behind a Traefik router: the Twenty CRM, n8n automation, Matomo analytics, electronic signature, invoicing and AI models. What follows comes from operating it, our mistakes included, and these are the rules behind our Leaving SaaS offer, delivered with automatic backups off the server and a tested restore.

The 3-2-1 rule, and what it does not say

The CNIL, France’s data protection authority, presents it as the state of the art in its factsheet on data backup: three copies of the data, on two different media, one of them offline. The same factsheet recommends keeping at least one at a geographically separate site, and warns against backups kept on the same systems as the data without being isolated: ransomware could then reach both.

For a rented server, this means in practice three copies:

This last point matters: if an attacker or a script error gains rights on the server, it must not be able to delete the remote copy with the same keys.

What the rule does not say is how to copy a database that is in use, or how to know that the copy is of any use. That is where things break.

What really breaks

1. The database copied like an ordinary folder

Copying the data directory of a PostgreSQL database while it runs does not give a reliable backup. The PostgreSQL documentation on file system level backup says so plainly: the database server must be shut down to get a usable copy, unless you use a consistent snapshot of the volume. A database is therefore backed up with its own tools: a logical export (pg_dump) for small databases, or a base backup complemented by continuous archiving of the transaction log (WAL) for larger ones.

This second mechanism makes it possible to go back to a precise moment. The PostgreSQL documentation on continuous archiving explains it: by replaying the log up to the desired moment, you get the database back as it was at any instant since the base backup. It draws an instruction from this:

The word “test” is in the official documentation: it is not an agency precaution.

2. The forgotten volume

With Docker, the data that matters lives in volumes or mounted folders. A service added in a hurry, with a volume that nobody added to the backup script, and on the day of the outage you discover it was never copied. The fix is simple and effective: a written list of the services, their volumes and their backup method, reviewed whenever a service is added.

3. The global clean-up command

This is our own mistake, and we tell it because it is instructive.

The documentation for docker system prune is clear, though: by default, it removes all stopped containers, networks that no container uses, dangling (untagged) images and unused build cache; with the --all option, all unused images. On a server where a service is briefly stopped, “unused” can mean “in production, but not at this moment”.

Volumes are not removed by default, so as not to delete important data that no container is using at that moment: that is what saved our data. With the --volumes option, that is no longer true for anonymous volumes.

Our advice: on a production server, no global clean-up, a targeted clean-up instead, and an up-to-date inventory of the services. An incident that leaves no written rule will happen again.

4. The backup that was never restored

The quietest case. The script runs every night, reports no error, and nobody has ever checked that the archive unpacks, that the database reloads, that the application starts on it. A backup only exists from the day it was restored successfully.

And success is judged on the content, not on the return code. On one of the projects we operate, the restore tool returned an error for mere warnings: we therefore validate a restore on the number of tables and records recovered.

5. The alert that never goes out

A backup script that fails without warning is worse than no script, because it gives false assurance. Every backup must send a signal on failure, and ideally a signal when it did not run at all.

The procedure we recommend

For an SME running a few services on a server of its own:

• Every night: export of each database with its own tool, copy of the listed volumes, encrypted upload to remote storage with its own credentials.
• Every week: automatic check that the night’s files exist, have a plausible size and unpack.
• Every quarter: full restore of one service in a disposable container or on a test machine, timed and validated on the content. The measured time becomes your real recovery time, the one you can announce to management.
• For each new service: update the inventory and the script, before going into production, not after.
• Retention: daily copies over a few weeks, monthly copies for longer. The period depends on your obligations and your personal data: keeping data longer than necessary is not security, it is a liability.

Honest limits

This procedure protects against data loss. It does not guarantee availability: restoring a service takes time, and during that time it is down. If your business cannot tolerate a few hours of interruption, you need redundancy (a second server ready to take over), which changes the cost. This is a line to include in the calculation of the real cost of leaving SaaS over three years.

Where to start

Write the inventory: each service, its database, its volumes and its backup method. Then restore one complete service, even a small one, and note how long it takes. Backups are only one building block of the infrastructure foundation shared by your projects, alongside access, monitoring and budgets. If you do not have a server yet, start with a low-criticality tool, as our guide to the tools to self-host first advises. To delegate, the managed operations of our Leaving SaaS offer cover monitoring, updates and tested backups. The other operations topics are in our infrastructure and cloud section.

Frequently asked questions

Sources

1. Factsheet on data backup (new tab)cnil.fr
2. PostgreSQL documentation on file system level backup (new tab)postgresql.org
3. PostgreSQL documentation on continuous archiving (new tab)postgresql.org
4. Documentation for docker system prune (new tab)docs.docker.com