Performance

An indexed window needs equality filters on the index’s leading fields, followed by matching sort fields. An unrelated filter or a range comparison on its own does not get the same shortcut. Full-text, vector and count indexes are not used for this list lookup.

A historical Rust engine experiment registered twenty windows over one million rows in 14.6 seconds without the index and 0.58 seconds with it, including the first build. The fixture has 420,000 rows in the selected collection. This measured in-process engine registration; it did not include a database changefeed, publication or a browser.

Indexes are built when needed, kept for an hour after their last view leaves, and prewarmed for restored active views before the SSP reports ready. Schema changes prewarm replacements too. A live first build still scans its table, but targets 25 ms per step and releases the circuit between steps. Writes keep an unfinished index correct; queries only read it once complete. A separate one-million-row test reduced the longest lock hold from 0.88 seconds to 74 ms, while the complete build took about twice as long. These are engine observations, not the WhitePawn publication times above.

The scheduler batches already queued ordinary changes up to 128 records or 1 MiB, without waiting to fill a batch. The SSP applies their source changes in order and updates affected query computations once per batch. Older SSPs, replay, jobs, users and internal lifecycle records keep their single-record paths.

For implementation details, see registration and ingest stages and batch ingestion. Rust embedders can use prewarm_query_indexes, prewarm_active_indexes, prewarm_index and the step-based variants. Their IndexBuildStats report index count, rows indexed and build time. Query stage timings are shared measurements for affected views, rather than exclusive CPU costs for each view.

Slow-operation evidence, publication wait times and queue diagnostics are explained in the SSP API and scheduler API. Permission-join maintenance still costs work proportional to the set of records a changed join key affects.

Dataset and startup

“Standard” and “large repeat” below identify two separate runs of the same 100K compact case. Their samples are not pooled. Both larger profiles remained incomplete because later tiers failed capacity preflight; the completed 100K cases passed final membership and version verification.

Serialized bytes sum the actual JSON fixture records, including players and collections; they are separate from database storage on disk. _00_version metadata also covers players and collections and is counted before mutations. The PGN payload averages about 1.2 KiB per game.

Startup durations are single observations in seconds. Database readiness precedes seeding. Backend readiness starts after seeding and includes scheduler/SSP startup and the wait for an available SSP; it is separate from database and seed time. The individual process readiness measurements are included within that backend interval.

Window registration and mutation latency

All latency values are p50 / p95 / p99 in milliseconds, using nearest-rank percentiles. Registration response ends at the scheduler’s HTTP response. Published readiness starts before that request and ends when committed root and child membership matches and record versions meet the required versions. It includes response time. Polls run every 10 ms plus database request time.

Each case has 99 new scroll registrations and one retained-window hit across 100 steps; the separate warmup is excluded here.

Mutation probes run individually after the previous mutation has settled. Write acknowledgement ends at SurrealDB’s transaction response. Published readiness starts before the write and includes that acknowledgement. Source queries used to construct expected results run outside the timed interval.

The full cases contain twenty probes per category; smoke contains two. No registration or mutation samples failed in these completed cases. The category breakdown below shows published readiness for the two 100K cases. With only fourteen smoke probes, both p95 and p99 select the maximum observed value; these short runs provide limited evidence about tail latency.

Unrelated collection writes measure publication to a separate one-row query, then verify that the main 50-row window remains unchanged. The unchanged-window check is outside the reported latency.

Write rate and catch-up

Full cases write 3,000 rows at a target of 50 rows/s for 60 seconds, then import 30,000 rows at a target of 1,000 rows/s for 30 seconds. Imports use 100-row transactions with at most four in flight. Smoke runs each phase for two seconds, committing 100 and 2,000 rows respectively. All completed cases had zero failed load-phase rows.

Throughput counts committed source rows per write interval. It does not establish the rate at which the SSP publishes them. After writes finish, the driver reads expected source windows, then times residual publication catch-up. The pipeline can drain while those source queries run, so the two durations below must be read together; residual catch-up alone is not total propagation delay.

Write acknowledgement under load is shown separately below, in milliseconds. Steady samples each commit one row; import samples each commit 100 rows.

Memory, CPU and storage

Scheduler/SSP process samples run every 250 ms; database container samples run roughly every five seconds plus request time. These are observed peaks over the whole case. CPU is the OS ps or Docker percentage: 100% represents one core, and a process can exceed it. Mean CPU averages those observations and is not a per-request CPU cost.

The combined database, scheduler and SSP work directory occupied 355.20 MiB for 10K compact, 363.82 MiB for 10K PGN, 941.75 MiB for 100K standard and 940.91 MiB for the 100K repeat. Serialized fixture bytes above do not include indexes, changefeeds, replica state or other platform storage.

Replica and publication backlog

Scheduler telemetry runs every second; SSP information is sampled roughly every five seconds. Peak values below can come from different observations. The last sample can precede final verification, and successful window verification does not prove every queue is empty.

No scheduler sampling errors or parked publication batches were observed. All four cases verified the exact nineteen retained registrations at completion, with a maximum of twenty during scrolling.

Graph sharing and memory attribution

With view sharing enabled, adding the identical subscriber kept graph count at 17 and increased merged followers from 0 to 1 in every case. A follower shares another registration’s computation; the graph owner is excluded from the follower count. After import, the smoke cases retained sharing, while both 100K cases had nineteen independent graphs and zero merged followers. Later memory observations include that change in graph sharing.

Expensive memory attribution runs between measured phases. These heap ownership estimates are separate from process RSS:

Resource-limited tiers

In these reports, neither 1M case nor the 10M case ran. The standard run excluded 1M compact on projected host memory and 1M PGN on projected disk, host memory and Docker memory. The large run excluded all three larger cases on those three checks. Both commands exited nonzero, retaining their completed 100K results.

The following are capacity projections from the large run, not measured consumption or minimum hardware requirements. They scale the completed compact case by populated rows, including the 33,000 load inserts, with a 1.25× margin. PGN projections also include payload allowances.

Capacity checks reserve 5 GiB free disk and cap projected native plus database memory at 80% of host RAM, with database memory additionally capped at 80% of Docker RAM. The native projection sums individual process peaks, which need not occur simultaneously. There are no measured latency or throughput results for these excluded tiers.

Report provenance

The main page calls the standard 100K case A and the large-profile repeat B. Both have the same initial dataset. Numbers come from these local report directories under apps/benchmark/results/:

Raw directories are local outputs and are not published with this site. Each contains run.json, samples.json, summary.md and failure logs when applicable. The recorded checkout base was 83273b8c, with benchmark changes in the working tree. The benchmark adds no arbitrary latency pass/fail budget.

These May 1, 2026 microbenchmarks start at POST /ingest and stop when the SSP’s in-memory result hash changes. They bypass database write acknowledgement and changefeed transport, and do not measure durable publication or client visibility. They should not be compared directly with the WhitePawn times.

For example, a 1,000-row result changed in about 3 ms, while a 30,000-row result changed in about 27 ms in the flat-query fixture:

Values are means of three per-run p50 measurements, rather than pooled percentiles. The tracked aggregate contains the source data for query shapes, row sizes, view counts and request rates. Those small fixtures do not establish capacity for arbitrary queries.

pnpm -F @spooky-sync/benchmark build
node apps/benchmark/dist/run-3x.js
node apps/benchmark/dist/verify-bootstrap.js --rows 1000 --row-bytes 2000000