Optimizing SkelaDB's branch path
Tracing slow branches, removing unnecessary waits, and measuring what actually improved.
A database branch can be cheap to create and still slow to use. In SkelaDB, I measured the path through parent ingestion, durable fork publication, PostgreSQL startup, login, reads, and a committed write. Timing only the fork would have missed much of the work.
One page, 127 calls#
An early trace found a visibility-map page replaying 16,149 write-ahead log records through 127 SQL helper calls. That reconstruction took about 108 ms.
I increased the bounded redo batches and removed per-byte formatting allocations. A regression test replayed 2,340 records in three calls instead of nineteen, producing the same page bytes as a stock PostgreSQL standby.
The next million-row pilot measured a median of 182 ms, down from 201 ms, and a p95 of 240 ms, down from 302 ms. Those were successive single cohorts, so I treated them as a promising observation rather than a precise measurement of the change's effect. The smaller workload's median got worse.
Remove waits that don't protect correctness#
The later traces pointed to storage reads and publication overhead. I made four changes:
- Load up to four required storage layers concurrently, with a serial fallback under memory pressure.
- Give validated bootstrap status layers an eviction preference inside the existing cache budget.
- Queue advisory WAL-retention reports after durable fork publication, rather than delaying the branch response.
- Move obsolete WAL-object deletion to a bounded worker after the replacement archive is durable.
Acknowledgments still wait for quorum, fsync, and synchronous archive publication. Checksums, authorization, and memory limits stay in place. Moving cleanup out of the response path does not make the required archive writes optional.
What the latest run says#
The completed local comparison retained three alternating pairs at each dataset size:
| Dataset | SkelaDB median / p95 | Local Neon median / p95 |
|---|---|---|
| 100K rows | 118 / 150 ms | 139 / 161 ms |
| 1M rows | 151 / 197 ms | 124 / 143 ms |
All 240 branches, 12 graceful storage-loss recovery checks, and 226,800 warm queries passed across the comparison. The latest bundle did not demonstrate an overall speedup: SkelaDB's previous million-row median was 140 ms. The runs were not interleaved across revisions, so the difference does not establish a cause.
These are roughly 15 MiB and 150 MiB tables, including indexes, compared against a pinned August 2025 local Neon build. The timer excludes deployment scheduling, and SQL runs without TLS. Neon archives asynchronously in this setup; SkelaDB waits for archive publication. This is not a comparison with current hosted Neon.
The next experiment is to separate small bootstrap status pages from large table-data layers, then isolate the remaining commit delay. A faster stage is useful evidence. The complete branch operation still has to get faster.