CRDT fields

Collaborative text fields backed by Loro. Clients send only their own edits, the server folds them into one canonical document, and the field itself stays plain, queryable text.

Two users. Same thread. Both typing at the same time.

With a normal database, somebody’s changes get silently overwritten. Locking makes the UX terrible. Server-side merging is a nightmare for character-level edits.

CRDTs fix this. A Conflict-free Replicated Data Type guarantees that any two replicas merge automatically, no matter what order edits arrive in. Every peer converges on the same result.

Sp00ky uses Loro under the hood. You annotate a field, and you get real-time collaborative editing with cursor presence. Each client sends only its own edits, through the same outbox and sync pipeline as every other write. The server folds those edits into the canonical document and writes the field back as plain text, so lists, search and filters keep working on it.

1. Annotate your schema

Put -- @crdt <kind> above a TYPE string field, and -- @cursor if you want remote cursors:

DEFINE TABLE thread SCHEMAFULL
  PERMISSIONS
    FOR select WHERE published = true OR author = $auth.id
    FOR create, update, delete WHERE author = $auth.id;

-- @crdt prosemirror
-- @cursor
DEFINE FIELD content ON TABLE thread TYPE string;

-- @crdt text
DEFINE FIELD summary ON TABLE thread TYPE option<string>;

The rules, all checked by spky generate:

• The field must be TYPE string (or option<string>) with no VALUE or ASSERT clause. The server owns its value.
• Top-level fields only, and not on TYPE RELATION tables.
• The surrealism deploy mode is refused: the fold runs in the SSP, and that mode has none.
• The names <table>__crdt_log, <table>__crdt_presence, _00_crdt_doc and fields ending in __crdt are reserved.

Then run spky generate and apply the schema (spky migrate / spky deploy). Who may read a document’s edits, and who may write them, is exactly who may read and update the record: the generated tables reuse the parent table’s select and update permissions.

2. The field is plain text

The annotated column no longer holds anything a client writes. It is the document’s plain text, written by the server each time it folds the edit log. Read it like any other column: render it in a list, search it, filter or order on it.

// The column is the document's plain text: query it like any other.
const threads = useQuery(() =>
  db.query('thread').orderBy('created_at', 'desc').limit(20).build()
);

<For each={threads.data() ?? []}>
  {(thread) => <p class="line-clamp-2">{thread.content}</p>}
</For>

• Write it only when creating the record. A db.create may set the initial text, so a new record shows it before the first fold (put the same text in the field’s draft document). After that a client write keeps the old value; leave the field out of db.update.
• It trails live edits. The column changes when the server folds, typically a few seconds after the last edit (at most one fold per field every 10 s by default) (see the fold). Inside an open editor you read the live document instead.
• prosemirror renders as lines. Block nodes (paragraphs, headings, list items) become lines, hard breaks become newlines, and formatting is dropped.
• Keep it out of the sync engine with @opaque. Add -- @opaque next to @crdt when the text is large or sensitive and you never filter on it; see @opaque.

Each fold also writes <field>__crdt on the record, a stamp of the folded version (its rev and the Loro version it covers). You don’t need to read it.

3. Open the field

useCrdtField opens the field’s document, keeps it syncing, and closes it on cleanup. It reopens when the record id changes.

import { useCrdtField } from '@spooky-sync/client-solid';

const content = useCrdtField('thread', () => thread()?.id, 'content');
// content() is null while the field opens, then a CrdtField

It returns an accessor that is null until the field is open, then a CrdtField. It must live inside a <Sp00kyProvider>, and has the same shape in @spooky-sync/client-solid and @spooky-sync/client-solid2.

Without Solid, open and close the field on the client yourself. Every open of the same field in a tab shares one document, so pair each open with a close:

const field = await client.openCrdtField('thread', 'thread:abc', 'content');
bindEditor(field.getDoc());

// later, when the editor goes away
await client.closeCrdtField('thread', 'thread:abc', 'content');

closeCrdtField resolves once the edits not sent yet are handed to the outbox, so awaiting it before navigating away loses nothing. Offline it resolves at once and the edits stay on the device until the field is next opened online; on a connection that has not noticed it is down it waits at most 4 seconds.

When a device holds nothing for a field yet, the open first fetches the server’s published snapshot and waits briefly (at most 4 seconds, not at all offline) for the log’s first answer, so the editor starts from the real document rather than an empty one. An open with a draft skips both: the record was just created there. Set crdt: true in the client config to load the Loro engine at startup instead of on the first open.

4. Create a record with content

A “new thread” form needs an editor before the record exists. Give it a draft:

import { Uuid, useDb } from '@spooky-sync/client-solid';
import type { CrdtField } from '@spooky-sync/core';

const db = useDb<typeof schema>();
const sp00ky = db.getSp00ky();

// 1. A detached document the editor writes into before the record exists.
const draft: CrdtField = await sp00ky.createCrdtDraft('thread', 'content');
// ... render <CollaborativeEditor field={draft} /> and let the user type ...

// 2. On submit, create the record. Leave `content` out: the server owns it.
const id = 'thread:' + Uuid.v4().toString().replace(/-/g, '');
await db.create(id, { title, author, published: false });

// 3. Hand the draft over. Its edits become the record's first edits.
await sp00ky.openCrdtField('thread', id, 'content', draft);
await sp00ky.closeCrdtField('thread', id, 'content');

createCrdtDraft returns a detached CrdtField that sends nothing. Passing it to openCrdtField after db.create merges its edits into the record’s document, where they go out as the record’s first edits. The example app’s CreateThreadDialog.tsx does exactly this.

5. Bind an editor

CrdtField.getDoc() returns a standard Loro LoroDoc.

prosemirror fields

Loro ships ProseMirror plugins in loro-prosemirror, so any editor built on ProseMirror works:

A minimal TipTap editor. Disable TipTap’s own history and let LoroUndoPlugin handle undo, so undo only ever reverts your own edits:

import { onMount, onCleanup } from 'solid-js';
import { Editor, Extension } from '@tiptap/core';
import StarterKit from '@tiptap/starter-kit';
import { LoroSyncPlugin, LoroUndoPlugin, undo, redo } from 'loro-prosemirror';
import { keymap } from '@tiptap/pm/keymap';
import type { CrdtField } from '@spooky-sync/core';

export function CollaborativeEditor(props: { field: CrdtField }) {
  let ref: HTMLDivElement | undefined;
  let editor: Editor | undefined;

  onMount(() => {
    const doc = props.field.getDoc();

    editor = new Editor({
      element: ref,
      extensions: [
        StarterKit.configure({ history: false }), // Loro handles undo
        Extension.create({
          name: 'loroSync',
          addProseMirrorPlugins: () => [
            LoroSyncPlugin({ doc }),
            LoroUndoPlugin({ doc }),
            keymap({ 'Mod-z': undo, 'Mod-y': redo, 'Mod-Shift-z': redo }),
          ],
        }),
      ],
    });
  });

  onCleanup(() => editor?.destroy());
  return <div ref={ref} />;
}

Show the plain-text column while the field opens:

const content = useCrdtField('thread', () => thread()?.id, 'content');

<Show when={content()} fallback={<p class="whitespace-pre-wrap">{thread()?.content}</p>}>
  {(field) => <CollaborativeEditor field={field()} />}
</Show>

For a full-featured version with cursor presence, single-line mode and a placeholder, see CollaborativeEditor.tsx in the example app.

text fields

The document holds one LoroText named after the field. Bind it to an input by diffing into it, so concurrent edits merge instead of replacing each other:

const field = useCrdtField('note', () => props.id, 'summary');
const [value, setValue] = createSignal('');

// Follow the document: local and remote edits alike re-render the input.
createEffect(() => {
  const doc = field()?.getDoc();
  if (!doc) return;
  const text = doc.getText('summary');
  setValue(text.toString());
  onCleanup(doc.subscribe(() => setValue(text.toString())));
});

const onInput = (e: InputEvent & { currentTarget: HTMLTextAreaElement }) => {
  const doc = field()?.getDoc();
  if (!doc) return;
  doc.getText('summary').update(e.currentTarget.value); // minimal diff
  doc.commit();
};

<textarea value={value()} onInput={onInput} />

spky recipe crdt-text-field --table <table> --field <field> renders this as a ready-made component.

6. Cursors (@cursor)

A @cursor field carries one presence blob per editing session. The CrdtField moves the bytes; what they mean is up to the editor. With TipTap, loro-prosemirror’s CursorEphemeralStore and LoroEphemeralCursorPlugin do the drawing:

import { CursorEphemeralStore, LoroEphemeralCursorPlugin } from 'loro-prosemirror';
import { cursorColorFromName } from '@spooky-sync/core';

const doc = field.getDoc();
const user = { name: username, color: cursorColorFromName(username) };
const presence = new CursorEphemeralStore(doc.peerIdStr);
presence.setLocal({ user });

// Draw remote cursors: add this plugin next to LoroSyncPlugin.
const cursorPlugin = LoroEphemeralCursorPlugin(presence, { user });

// Send: publish the local cursor whenever it moves.
const stop = presence.subscribeBy((by) => {
  if (by === 'local') field.pushCursorState(presence.encodeAll());
});

// Receive: every other session's presence bytes.
field.onCursorUpdate = (data) => presence.apply(data);

Presence is the latest state only: it is not queued offline, it is sent at most every 250 ms, and a session’s row is removed when its field closes (the SSP sweeps rows a crashed tab left behind once they are older than the presence TTL).

The CrdtField API

import type { CrdtField } from '@spooky-sync/core';

const field: CrdtField = content();

field.getDoc();                 // LoroDoc: bind your editor to it
field.hasContent();             // true once the document holds anything
field.pushCursorState(bytes);   // publish this session's presence (`@cursor` only)
field.onCursorUpdate = (bytes) => { /* another session's presence */ };
// A blob that arrives before you set onCursorUpdate is replayed when you do.

// On the client (useDb().getSp00ky() in Solid):
client.openCrdtField(table, recordId, field, draft?);  // Promise<CrdtField>
client.closeCrdtField(table, recordId, field);         // Promise<void>
client.createCrdtDraft(table, field);                  // Promise<CrdtField>

import { cursorColorFromName, CURSOR_COLORS } from '@spooky-sync/core';

const color = cursorColorFromName('ada');  // a stable color per user name

How it syncs

Nothing about a CRDT field bypasses the sync pipeline. For a table with @crdt fields, spky generate adds:

Up. Typing changes the tab’s local Loro document right away. 50 ms after the first keystroke (crdtDebounceMs), and then every 200 ms for as long as the typing goes on (crdtSendIntervalMs), the tab exports only its own new operations as one Loro update and creates one <table>__crdt_log row through the ordinary outbox, so edits survive offline, reloads and retries like any other write. Each tab edits under a random Loro peer id that it claims for the signed-in user in _00_crdt_peer; the database refuses a log row whose peer the writer does not own.

Down. An open field is an ordinary query on its log rows, so other clients’ edits arrive through the normal sync pipeline on the same SurrealDB connection, with no extra LIVE query, and are merged into the open document.

The acceptance rule. Before any client or the server imports a log row, it checks that the blob is a Loro update (not a snapshot), at most 256 KiB, and that every operation in it was made by the row’s peer. A row that fails never enters any document, anywhere, so a client cannot inject edits under someone else’s name or replace history with a crafted snapshot.

Locally. The device keeps its merged document and how far each of its peers’ edits got into the outbox in _00_crdt_local. Edits made offline, or just before a reload, are sent later under the peer they were made with. Signing out does not send them under the next user: they wait in the old account’s local store.

The server fold

The SSP is the only writer of the canonical document. To keep its CPU flat it folds rarely: a field is folded once its log has been idle for a while, or once enough rows or bytes are pending, and never more often than a minimum gap. A fold takes a lease on the field’s _00_crdt_doc row, imports the accepted log rows into the snapshot, writes the new snapshot and the record’s plain-text column in one transaction, and then deletes the rows it folded. Several SSPs in a cluster are safe: the lease and a compare-and-swap on rev mean exactly one of them publishes each version.

Nothing waits on a fold. A late fold only delays the plain-text column and the snapshot new devices start from; live editors exchange edits through the log directly.

The thresholds are SSP environment variables:

Self-hosted, set them on the SSP process (Self-hosting). On Sp00ky Cloud, set them for the ssp role in deployment.env. The SSP’s /info reports fold counters under crdt (SSP API).

Limitations

• Two kinds. text and prosemirror. The plain-text column renders those shapes only.
• The column trails the editor. List previews update when the server folds, also for your own edits.
• Every fold rewrites the record. The column and its __crdt stamp are an update of the parent row, so VALUE time::now() fields such as updated_at move on each fold, and the row syncs to everyone watching it.
• Impersonation is read-only here. An impersonation session cannot write edits or presence.
• JavaScript clients only. The Dart client (spooky_core, Flutter) throws UnimplementedError for CRDT fields.
• API may change while experimental.

Next steps