How BorgBackup Deduplication Works

BorgBackup stores your files as content-defined chunks and keeps exactly one copy of each identical chunk in the repository. That single rule is why a daily "full" backup costs little more than the changes since yesterday, while every historical version stays complete and browsable. The mechanism applies across files, across machines, and across years of history — so the savings compound the longer you use one repository.

The core mechanism: content-defined chunking

Borg does not treat a file as one indivisible object. It splits file content into variable-sized chunks, where the boundaries are determined by the content itself rather than by fixed offsets. This is what "content-defined" means: the same byte sequence tends to produce the same chunk boundaries regardless of where it sits in a file or which file it came from.

Each chunk is then identified by a hash. Before writing anything, Borg checks whether a chunk with that identity already exists in the repository:

• Chunk already present → it is not stored again; the archive simply references the existing copy.
• Chunk is new → it is compressed, encrypted, and written once.

Because identity is based on content, two identical chunks are indistinguishable to Borg whether they came from the same file, a different file, a different machine, or a backup taken three years ago.

What this looks like over time

The site's own visualization makes the arithmetic concrete. Three nightly backups of the same machine:

Across the three nights, 24 chunk occurrences were seen, but duplicates collapse into a single stored copy each. Night 2 reuses seven chunks and adds one; night 3 reuses six and adds one. The repository grows by the genuinely new data, not by the total data touched.

Two consequences follow directly:

• Daily full backups stay cheap. You can take a complete backup every night without the storage cost scaling with your total data size — it scales with your changes.
• History stays complete. Deduplication removes redundant storage, not redundant versions. Old chunks are retained, so you can still recover the state of a file from any prior archive.

Why it works across machines and years

The deduplication domain is the repository, not the individual backup run. Once a chunk exists in a repository, any later archive — from any source — that contains the same content will reference it rather than store it. This is why the site describes chunks as stored once "across files, across machines, across years of history."

A practical implication: pointing several machines at one repository lets them share chunks. If two machines hold identical files (a shared document set, the same OS packages, a common codebase), that content is stored once for both. The benefit is largest where content overlaps — repeated backups of similar data — and smallest where every byte is unique.

What Borg adds around the chunks

Deduplication is the storage-efficiency layer; the surrounding features determine what you can do with the result:

• Compression — each new chunk is compressed before storage. You choose the trade-off: lz4 for speed, zstd for balance, zlib and lzma for maximum squeeze, or none.
• Authenticated encryption — chunks are encrypted with AES plus cryptographic authentication. The backup server only ever sees ciphertext, so it does not need to be trusted.
• Mountable archives — any archive can be mounted as a regular filesystem via FUSE, so you can browse old versions of your files like any folder.
• No daemons or agents — a backup is one command against a repository on a local disk, a USB drive, or any machine reachable over SSH.

What to check before relying on it

• Repository scope determines savings. Chunks deduplicate only within a single repository. Splitting backups across separate repositories forfeits cross-source sharing.
• Compression choice affects cost, not correctness. It changes speed and size, not whether deduplication happens.
• Encryption means the server can't deduplicate for you. Because the server sees only ciphertext, all deduplication decisions happen client-side before data leaves your machine.

If your goal is to understand whether Borg will save you space: it will, in proportion to how much repeated content your backups contain. The more often you back up similar data to the same repository, the closer a "full" backup gets to costing only its changes.