Self-hosting

Run sp00ky on your own server with Docker Compose. SurrealDB and one SSP is a complete setup, big projects included.

A self-hosted sp00ky is two containers. SurrealDB holds your data, and the SSP keeps every live query up to date. Your app connects to SurrealDB directly, so the SSP never has to be reachable from the internet.

sequenceDiagram
    participant App as Your app
    participant DB as SurrealDB
    participant SSP
    App->>DB: reads and writes over wss://db.example.com
    DB->>SSP: every change, through the sync events
    SSP->>DB: updated query results
    DB-->>App: live update

What you need

• A Linux server with Docker and Docker Compose, ports 80 and 443 open.
• A domain for the database, for example db.example.com, with its DNS pointing at the server. Caddy uses it to get a TLS certificate automatically.
• Your sp00ky project and the spky CLI on your own machine.

Set it up

1. Write the .env file

   On the server, create an empty directory with a .env file in it. Compose reads it automatically, and every value below is used in the next steps.

   .env

   # Your images and your CLI must be the same version (spky --version)
   SPOOKY_VERSION=0.0.1-canary.283
   
   # The domain your app connects to. Point its DNS at this server.
   DB_DOMAIN=db.example.com
   
   # Long random values, e.g. from: openssl rand -hex 32
   SURREAL_PASS=change-me
   SPKY_AUTH_SECRET=change-me

2. Add the Caddyfile and docker-compose.yml

   Caddy terminates TLS for your domain and forwards it to SurrealDB, WebSockets included. If you already run a reverse proxy, drop the caddy service and point your proxy at surrealdb:8000 instead.

   Caddyfile

   {$DB_DOMAIN} {
   	reverse_proxy surrealdb:8000
   }

   docker-compose.yml

   services:
     surrealdb:
       image: surrealdb/surrealdb:v3.1.0-beta.3
       user: root
       command: start --allow-all rocksdb:/data/db
       environment:
         SURREAL_USER: root
         SURREAL_PASS: ${SURREAL_PASS}
         SURREAL_CAPS_ALLOW_EXPERIMENTAL: files,surrealism
       volumes:
         - surrealdb:/data
       restart: unless-stopped
   
     ssp:
       image: mono424/spooky-ssp:${SPOOKY_VERSION}
       environment:
         SPKY_DB_URL: http://surrealdb:8000
         SPKY_DB_NS: main
         SPKY_DB_NAME: main
         SPKY_DB_USER: root
         SPKY_DB_PASS: ${SURREAL_PASS}
         SPKY_AUTH_SECRET: ${SPKY_AUTH_SECRET}
         # Row bytes on disk. With a scheduler, SPKY_SSP_SNAPSHOT_DIR also keeps
         # row checkpoints across restarts; a standalone SSP rebuilds from the
         # database on every start.
         SPKY_SSP_SNAPSHOT_DIR: /data/snapshot
         SPKY_SSP_ARENA_DIR: /data/arena
       volumes:
         - ssp:/data
       ports:
         - "127.0.0.1:8667:8667" # health checks from the server only
       depends_on:
         - surrealdb
       restart: unless-stopped
   
     caddy:
       image: caddy:2
       environment:
         DB_DOMAIN: ${DB_DOMAIN}
       ports:
         - "80:80"
         - "443:443"
       volumes:
         - ./Caddyfile:/etc/caddy/Caddyfile:ro
         - caddy:/data
       restart: unless-stopped
   
   volumes:
     surrealdb:
     ssp:
     caddy:

   SPKY_DB_NS and SPKY_DB_NAME must match surrealdb.namespace and surrealdb.database in your sp00ky.yml. Both default to main.

3. Start it

   docker compose up -d
   curl localhost:8667/health

   The SSP answers with a short JSON status. Once it contains "status":"ready", it is up.

4. Apply your schema

   From your project directory on your own machine, apply your migrations together with the sync events sp00ky adds to every table:

   export SURREAL_URL=https://db.example.com
   export SURREAL_PASS=<SURREAL_PASS from .env>
   
   spky migrate apply --endpoint http://ssp:8667 --secret <SPKY_AUTH_SECRET from .env>

   • SURREAL_URL is how your machine reaches SurrealDB.
   • --endpoint is how SurrealDB reaches the SSP. The sync events call it on every write, from inside the Docker network, so it is the service name ssp and not your domain.
   • --secret must be the exact SPKY_AUTH_SECRET from .env, or the SSP rejects every change.
5. Point your app at it

   Set the client’s endpoint to your domain. Namespace and database are the same as above.

   const dbConfig: SyncedDbConfig<typeof schema> = {
     // ...
     database: {
       endpoint: 'wss://db.example.com/rpc',
       namespace: 'main',
       database: 'main',
     },
   };

   If your client sets enableAnonymousLiveQueries: true, add SPKY_SSP_ANON_LIVE_QUERIES: "true" to the SSP as well. The two must agree, or signed-out users get no live updates.

That is a complete production setup. Everything below is optional.

Everyday tasks

Change the schema

Create a migration as usual, then apply it with the same command as the first time:

spky migrate create add_due_date
spky migrate apply --endpoint http://ssp:8667 --secret <SPKY_AUTH_SECRET>

The SSP notices new and changed tables on its own within 15 seconds. No restart needed.

Upgrade

Update the CLI, set SPOOKY_VERSION in .env to the version it prints, then pull the new images and re-apply so the sync events match the new version:

docker compose pull
docker compose up -d
spky migrate apply --endpoint http://ssp:8667 --secret <SPKY_AUTH_SECRET>

To upgrade the scheduler or an SSP without downtime, start the new container next to the old one and let it take over, see Blue/green handover.

Back up

Add the S3_* variables from Backup storage to the SSP and it can export SurrealDB to any S3-compatible bucket and restore from it, through the backup routes. With the scheduler, the dashboard does this for you.

Backends and jobs

If your sp00ky.yml has backends that take jobs from an outbox table, run each one as another service in the same compose file and tell the SSP where to send the jobs:

  ssp:
    environment:
      # ...
      SPKY_JOB_CONFIG: '[{"name":"api","base_url":"http://api:3000","table":"job"}]'

  api:
    image: registry.example.com/my-api:latest
    environment:
      SPKY_DB_URL: http://surrealdb:8000
      SPKY_DB_NS: main
      SPKY_DB_NAME: main
      SPKY_DB_USER: root
      SPKY_DB_PASS: ${SURREAL_PASS}
    restart: unless-stopped

table is the backend’s outbox table (method.table in sp00ky.yml) and base_url is how the SSP reaches the backend on the Docker network. With no backends, leave SPKY_JOB_CONFIG out.

Add the scheduler

The scheduler sits between SurrealDB and your SSPs. You still run one SSP. What you get:

• The operator dashboard and its MCP server.
• A replica of your database, so an SSP restart loads its own row checkpoints and repairs only what changed against the replica, instead of reading everything from SurrealDB again.
• An end-to-end heartbeat that can page you through a webhook, and drift checks that repair the replica on their own.
• The option to run more SSPs later.

To switch:

1. Set mode: cluster in your sp00ky.yml.
2. Add SPKY_ADMIN_PASSWORD (another long random value) to .env.
3. Add the scheduler service and point the SSP at it:

  scheduler:
    image: mono424/spooky-scheduler:${SPOOKY_VERSION}
    environment:
      SPKY_DB_URL: http://surrealdb:8000
      SPKY_DB_NS: main
      SPKY_DB_NAME: main
      SPKY_DB_USER: root
      SPKY_DB_PASS: ${SURREAL_PASS}
      SPKY_AUTH_SECRET: ${SPKY_AUTH_SECRET}
      SPKY_ADMIN_PASSWORD: ${SPKY_ADMIN_PASSWORD}
    volumes:
      - scheduler:/data # replica and write-ahead log
    ports:
      - "127.0.0.1:9668:9668" # dashboard. Never publish 9667.
    depends_on:
      - surrealdb
    restart: unless-stopped

  ssp:
    image: mono424/spooky-ssp:${SPOOKY_VERSION}
    environment:
      # ...everything from before, plus:
      SPKY_SCHEDULER_URL: http://scheduler:9667
      SPKY_SSP_ID: ssp-0
      SPKY_SSP_ADVERTISE_ADDR: ssp:8667
    depends_on:
      - scheduler

volumes:
  # ...
  scheduler:

4. Run docker compose up -d, then apply again with the scheduler as the endpoint, since the sync events now go to the scheduler instead of the SSP:

spky migrate apply --endpoint http://scheduler:9667 --secret <SPKY_AUTH_SECRET>

Operator dashboard

The dashboard shows the health of the scheduler and every SSP, end-to-end sync latency, backend health, live logs and workflow runs. It can restart SSPs, cancel and retry workflow runs, pause schedules and take backups.

The compose file above binds it to the server itself. Reach it over an SSH tunnel:

ssh -L 9668:localhost:9668 you@your-server
# then open http://localhost:9668/admin

Or give it its own domain by adding a second block to the Caddyfile:

{$DB_DOMAIN} {
	reverse_proxy surrealdb:8000
}

admin.example.com {
	reverse_proxy scheduler:9668
}

Sign in with SPKY_ADMIN_PASSWORD, the break-glass login that works even when the database is down, or with a normal app account after spky admin add <user> puts it on the admin roster.

The same port serves the MCP server at /admin/api/mcp, so an AI assistant can read and operate the deployment with tokens minted on the dashboard’s Access page.

A few actions only work on Sp00ky Cloud, because they manage containers rather than processes: pulling newer images, wiping the scheduler volume, restarting SurrealDB, and the managed backup catalog, schedule and retention. The dashboard shows them greyed out with the reason Not linked to Sp00ky Cloud. See Admin dashboard for the full surface.

Run more than one SSP

Only when one SSP runs out of memory or CPU. Each extra SSP needs its own id, address and volume:

  ssp-1:
    image: mono424/spooky-ssp:${SPOOKY_VERSION}
    environment:
      # ...same as ssp, except:
      SPKY_SSP_ID: ssp-1
      SPKY_SSP_ADVERTISE_ADDR: ssp-1:8667
    volumes:
      - ssp-1:/data
    depends_on:
      - scheduler
    restart: unless-stopped

volumes:
  # ...
  ssp-1:

The scheduler spreads views across the SSPs by load. Keep SPKY_CLEAR_VIEWS_ON_START at its default (true) while you run more than one.

Environment variables

Every variable the SSP and the scheduler read, with its default. The setups above only need the handful they show; reach for these when you tune or debug.

SSP (Sidecar)

Read straight from the process environment.

Row checkpoints

Without a snapshot directory every restart of a cluster SSP is a cold rebuild: the SSP re-pages the whole database while the scheduler holds sync frozen. Point these at a volume and a restart loads the rows it kept, verifies them against the scheduler and repairs only what changed. Both are caches of SurrealDB, never the source of truth, so deleting them is always safe. A standalone SSP rebuilds from the database on every start; a snapshot.json an older build left in the directory is removed by the next clean restart. See State persistence.

Scheduler

The scheduler reads most of its config from sp00ky.yml, and the SPKY_* vars in the first table below override the corresponding YAML field. The fields ingest_host, ingest_port, load_balance, replica_db_path, wal_path, ssp_poll_interval_ms and job_tables are YAML only; max_buffer_per_ssp also answers to SPKY_MAX_BUFFER_PER_SSP.

The tables after it are env only and have no YAML equivalent: the admin plane, backup storage, the dead-man heartbeat, and the tuning knobs.

Core

Admin plane

The dashboard and its MCP server, on their own port. See Admin dashboard.

Machine pools

The listener machine pool agents connect to, and the docker provider a self-hosted scheduler can create machines with.

Backup storage

These carry no SPKY_ prefix. The backup plane is enabled as soon as any of S3_ENDPOINT, S3_ACCESS_KEY, S3_SECRET_KEY or S3_BUCKET is present, so a partial configuration turns backups on with defaults filling the rest. Any S3-compatible store works. Also read by a standalone SSP, which serves the same backup routes.

Heartbeat and alerting

An end-to-end probe: the scheduler writes _00_heartbeat:probe upstream and times the full round trip through /ingest, the WAL, the broadcast and every SSP’s circuit. It is what the dashboard’s latency reading and the /health staleness flag are built on.

SurrealDB WebSocket watchdog

Each cycle the scheduler opens a new WebSocket to SurrealDB and waits for the upgrade. A cycle counts as wedged when /health answers but the upgrade gets no response at all, the SurrealDB 3.x state where open sockets keep working and no new client can connect. A database that is down is left to the control plane. Needs a plain http:// or ws:// SPKY_DB_URL; restarts need the Sp00ky Cloud link, without it the watchdog only reports.

Tuning

Web Push

The push engine runs in the scheduler in cluster mode and in the SSP in singlenode mode (an SSP behind a scheduler never runs it), and reads the same variables in both. It needs no configuration: its VAPID key is derived from SPKY_AUTH_SECRET, and the rules come from push: in sp00ky.yml. See Push notifications.

Native push needs no variable either: the APNs and FCM credentials come from push.apns / push.fcm, which spky migrate stores in the database (_00_push_credential). The host needs outbound HTTPS (HTTP/2 for Apple) to api.push.apple.com, api.sandbox.push.apple.com, fcm.googleapis.com and oauth2.googleapis.com. See Native push.

Next steps