Pilotbase showing the measured PostgreSQL 18 backups: a 3 GB full, four incrementals from 37 MB to 1.4 GB, pg_combinebackup and pgBackRest side by side

A nightly full backup of a big database copies every file, every night, even when one table changed. Since PostgreSQL 17 the server can do better on its own: pg_basebackup --incremental copies only the 8 kB blocks that changed since an earlier backup, and pg_combinebackup stitches a full backup and its incrementals back into a data directory you can start. No extension, no extra tool.

We ran it on PostgreSQL 18.6 against a 2,998 MB pgbench database taking 500 transactions a second: one full backup, four incrementals after different kinds of write load, a combine, a restore that we checked row for row, three ways to break it, and pgBackRest 2.60 doing the same job next to it. Here is what came out.

text
                                         time      size        share of a full
full backup (pg_basebackup)               17.1 s   3,062 MB    100%
incremental after 2 min, uniform writes    6.1 s     765 MB    25%
incremental after 2 min, hot rows          0.3 s      44 MB    1%
incremental after no writes                0.2 s      37 MB    1%
incremental after 5 min, uniform writes   20.9 s   1,404 MB    46%
pg_combinebackup (full + 4)               21.8 s   3,317 MB written
start the combined copy                    0.3 s   every row as at the last incremental

The short version: an incremental is small when writes keep landing on the same rows and surprisingly large when they're spread out, because it is priced in blocks touched, not rows changed. Combining is a plain file copy at disk speed. And the chain is only as good as its weakest piece, with nothing in PostgreSQL itself that keeps the pieces for you. The details follow.

Turning it on

Incremental backups need WAL summaries: a background process reads the write-ahead log and records which blocks of which files changed between two points. It is off by default. Turning it on needs a reload, not a restart:

sql
ALTER SYSTEM SET summarize_wal = on;
SELECT pg_reload_conf();

-- wal_summary_keep_time (default 10 days) decides how long summaries are kept;
-- an incremental needs every summary since the backup it builds on

The summaries are tiny. Over the whole run (about 13 minutes of writes and 448 MB of WAL) the summarizer wrote 20 files taking 308 kB in pg_wal/summaries. pg_walsummary shows what one holds:

text
TS 1663, DB 16384, REL 16400, FORK main: block 10
TS 1663, DB 16384, REL 16400, FORK main: block 20
TS 1663, DB 16384, REL 16400, FORK main: block 37
TS 1663, DB 16384, REL 16400, FORK main: block 40
...
57,980 lines for the first 2 minutes of writes: 45,790 blocks of pgbench_accounts (relation 16400),
12,163 of its primary key (16408), a handful of catalog blocks

Then the backups themselves. The full one is an ordinary base backup; each incremental names the manifest of the backup before it:

bash
pg_basebackup -D /backup/full -c fast
pg_basebackup -D /backup/i1 -c fast --incremental=/backup/full/backup_manifest
pg_basebackup -D /backup/i2 -c fast --incremental=/backup/i1/backup_manifest

# restore: newest last, into an empty directory
pg_combinebackup -o /restore/pgdata /backup/full /backup/i1 /backup/i2

The test

pgbench -i -s 200: 20,000,000 rows in pgbench_accounts, 2,998 MB in all, loaded in 13.6 s. A pgbench client then played the application at a steady 500 transactions a second from 8 connections. Each transaction updates one account, one teller and one branch and inserts a history row. Between backups we ran four kinds of load:

text
i1  2 min, accounts picked at random        60,035 transactions
i2  2 min, hot accounts (Zipf, s = 1.1)      60,302 transactions, most of them on a few thousand accounts
i3  nothing at all
i4  5 min, accounts picked at random        150,006 transactions

Writes stopped for each backup so we could record exactly what it should contain: the sum of every balance, the history row count and the newest history timestamp. The machine was a Mac mini (Apple M4, 16 GB, internal SSD) running Homebrew's PostgreSQL 18.6 with shared_buffers at 1 GB, max_wal_size 4 GB and the defaults otherwise. The internal SSD is fast; on a cloud volume with a throughput cap, expect the times to scale with size.

Why 2 minutes of writes cost a quarter of the database

The first incremental, after 60,035 transactions spread at random over 20 million accounts, was 765 MB: 25% of the full backup for two minutes of a modest load. The five-minute run made the next one 1,404 MB, 46%.

The reason is how PostgreSQL stores an update. The new version of the row goes wherever there is room (on a freshly loaded table with no free space, that is the end of the table), the old version on its original page gets marked dead, and the primary key index gains an entry on its own leaf page. With accounts picked at random, nearly every update dirties a different old heap page and a different index page. The WAL summary records each of those 8 kB blocks, and the incremental copies each one in full:

text
files in the 5-minute incremental (base/16384)   size     what
INCREMENTAL.16400, .16400.1, .16400.2            953 MB   pgbench_accounts (2.6 GB table)
INCREMENTAL.16408                                378 MB   its primary key index (428 MB)
INCREMENTAL.16402                                  8 MB   pgbench_history (appends only)
16400_fsm, 16400_vm                              0.8 MB   free space and visibility maps, always copied whole

Two minutes of uniform updates gave 749 MB of data blocks, about 95,872 of the database's 383,744 pages. In the five-minute run, 150,006 random updates reached 378 MB of the 428 MB primary key index: nearly every leaf page got a new entry. History rows, all appended at the end, cost 8 MB.

The same number of updates aimed at hot rows gave a 44 MB incremental, only 28 MB of it data: the same few thousand pages took most of the writes, and a block that changes a thousand times is still copied once. That is the case incrementals are built for, and most real tables look more like it than like pgbench's uniform random: recent orders, active sessions, this month's events.

The quiet incremental was 37 MB with nothing changed. 16 MB of that is the WAL segment every streamed backup includes so it can be made consistent; the rest is the files an incremental always copies whole (the transaction status files, configuration, small catalogs). Budget roughly 20 to 40 MB per incremental as a floor.

Rule of thumb

Estimate an incremental as distinct blocks written since the last backup times 8 kB, not rows changed times row size. Spread-out updates on a large table push it toward the size of a full backup quickly, and at that point a new full costs about the same and shortens the chain. Inserts that only append are cheap: they fill new pages at the end and touch few old ones.

Restoring: pg_combinebackup and a check

Combining the full backup with all four incrementals took 21.8 s and wrote 3,317 MB, a complete data directory as of the last incremental. pg_verifybackup checked it against the manifest that pg_combinebackup writes in 1.3 s. The server started in 0.3 s, and the three values we recorded before the last backup came back identical:

text
                    sum(abalance) / history rows / newest history row
recorded at i4      289248/270343/2026-10-30 17:02:31.840443
restored copy       289248/270343/2026-10-30 17:02:31.840443

An amcheck pass over every B-tree index in the restored database (bt_index_check(..., true), which also checks that each heap row has its index entry) took 8.9 s and found nothing. Run something like it on every test restore: matching counts don't prove the indexes are sound.

--clone and --link

PostgreSQL 18's pg_combinebackup can clone files instead of copying them (--clone, on filesystems with reflinks: APFS, XFS, Btrfs) or hard-link files that come unchanged from one backup (--link). Both only help for files no incremental touched. Here every 1 GB segment of pgbench_accounts and its index had changed blocks, so they still had to be rebuilt: --clone took 32.5 s and used 3,048 MB, --link 22.6 s and 3,038 MB. On a database where most tables are quiet the saving is much larger. One warning for --link: the output shares files with the backups, so starting a server on it changes the backup files too. Use it only for a copy you'll throw away, or after you've stopped needing the inputs.

Plan the restore disk for a full copy either way. pg_combinebackup can't write into the old data directory, and you need the backups readable on local disk, not in object storage.

Three ways to break it

An incremental is useless without every backup it builds on. We deleted the first incremental and tried to combine the rest:

text
pg_combinebackup: error: backup at "/backup/full" starts at LSN 0/9C000028, but expected 0/CC000028

Good: pg_combinebackup checks that each backup starts where the previous one ended and refuses a gap. It can't tell you that newer incrementals exist, though. Combine the full backup with only the first two and you get a perfectly valid database from an older point in time, with no warning.

Starting an incremental directory as if it were a backup fails just as clearly:

text
FATAL:  this is an incremental backup, not a data directory
HINT:  Use pg_combinebackup to reconstruct a valid data directory.

And with summarize_wal switched off (or the summaries since the last backup already removed by wal_summary_keep_time), the next incremental is refused at the start:

text
pg_basebackup: error: could not initiate base backup: ERROR:  incremental backups cannot be taken unless WAL summarization is enabled

Two habits follow. Take a new full backup more often than wal_summary_keep_time (10 days by default), so a missed night can never leave you with summaries already deleted. And keep a full backup and its incrementals together as one unit: delete them together, copy them off the machine together, and don't let a cleanup script remove an old full that newer incrementals still need. PostgreSQL has no catalog of backups, so that bookkeeping is yours.

Next to pgBackRest

pgBackRest has had differential and incremental backups for years, and our point-in-time recovery article uses it. We took its backups at the same moments, with its defaults (gzip, one process):

text
                              pg_basebackup        pgBackRest (default: whole changed files, gzip)
full                           17.1 s, 3,062 MB    39.9 s, 165.9 MB in the repo (read 3 GB)
after 2 min uniform             6.1 s,   765 MB    25.2 s, 166.8 MB (read 2.9 GB)
after 2 min hot rows            0.3 s,    44 MB    20.0 s, 130.2 MB (read 1.9 GB)
after no writes                 0.2 s,    37 MB     1.9 s, 472 bytes
after 5 min uniform            20.9 s, 1,404 MB    37.0 s, 176.4 MB (read 3 GB)
restore                        21.8 s combine        13.0 s

pgBackRest restored faster (13.0 s against 21.8 s): it read about 180 MB of compressed repository, where pg_combinebackup read the full backup plus 2.2 GB of incrementals. And with nothing changed it stored 472 bytes, because it skips files whose size and timestamp are the same.

By default pgBackRest's incremental is file-level: any data file whose size or timestamp changed is copied whole, and PostgreSQL splits tables into 1 GB files. After hot-row updates it still read 1.9 GB where pg_basebackup copied 44 MB. Its repository looks tiny only because pgbench rows are mostly blank padding that gzip squeezes about 18 to 1. Real tables usually compress 2 to 4 to 1, so on real data its incrementals would weigh hundreds of megabytes here, not tens.

pgBackRest with block incremental

pgBackRest 2.46 added block-level incrementals (repo1-bundle=y and repo1-block=y). We repeated two minutes of uniform writes against a fresh full backup with both tools on the same cluster. pg_basebackup's incremental was 641 MB (16 MB of it WAL) in 3.1 s. pgBackRest's added 173 MB to its repository in 24.6 s, and its log still shows 3 GB read: without WAL summaries it has to read every changed file in full to find the changed blocks, and it stores them in blocks that grow with the file size (its repo-block-size-map), not 8 kB pages. On padding this compressible the result is about the same as file-level (166.8 MB). The gain shows on real, less compressible data and over long chains, where unchanged parts of big files are no longer stored again.

What pgBackRest adds is everything around the backup: the WAL archive for point-in-time recovery, retention that never deletes a full backup a newer one depends on, S3/GCS/Azure repositories, encryption, parallel processes, and pgbackrest verify. The built-in tools give you consistent backups and a combine step; archiving, retention, copies off the machine and restore testing are scripts you write and keep working.

Which one to use

text
one server, backups to local or mounted disk, a small cron     pg_basebackup --incremental nightly,
script is fine                                                  a full every week (< wal_summary_keep_time)
need point-in-time recovery to any second                       pgBackRest (or WAL archiving you run yourself)
backups to object storage, retention, encryption                pgBackRest with repo1-block=y
updates spread evenly over a big table                          incrementals near full size: shorter chains,
                                                                more frequent fulls
mostly appends and hot recent rows                              incrementals stay small, the built-in tools shine

Whichever you pick, a backup you haven't restored is only a hope. The combine-start-check loop above took under a minute for 3 GB; put it in the same script that takes the backups. Our pg_dump restore test covers the logical side.

Pilotbase connected to the PostgreSQL 18 backup lab: the measured full, incremental, combine and pgBackRest runs in a result grid, next to the pgbench tables

Check a restored copy the same way we did: open it in Pilotbase, run the counts and the amcheck query, and compare with the source in a second tab. Free for Windows, macOS and Linux, or Docker on a server.

Get Pilotbase

Related: Point-in-time recovery in Postgres · Deleting millions of rows without hurting production · Upgrading Postgres 16 to 18

Sources: our own runs on PostgreSQL 18.6 and pgBackRest 2.60, 30 October 2026. PostgreSQL 18 documentation: making an incremental backup, pg_basebackup (--incremental), pg_combinebackup (--clone, --link and its warning), WAL summarization settings (summarize_wal, wal_summary_keep_time), pg_walsummary, amcheck. pgBackRest: repository block incremental and backup types.