sluice

Online schema changes on PlanetScale with sluice

The expand→migrate→contract pattern as one gated command, the standalone resumable backfill with its verify gate, the deploy-ddl governed channel for safe-migrations branches, and the freshness gates that guard your production schema.

sluice ships a family of schema-change commands that drive the classic expand → migrate → contract pattern against PlanetScale: expand-contract (all three legs, one command), backfill (the data-migration middle step on its own — this one also works on plain MySQL, Vitess, and Postgres), and deploy-ddl + control-tables ddl (the governed DDL channel and bootstrap for branches with safe migrations enabled). Every leg was live-validated against real PlanetScale, and the pattern has been exercised at 131,072-row backfill scale. What sluice deliberately is not: a versioned-migration tool — no history table, no down-migrations. Atlas / Flyway / sqitch own that layer; your migration tool decides what changes, and these commands are a safe way to execute it.

The full pattern: sluice expand-contract #

Add the new column (expand), backfill the data (migrate), drop the old column (contract) — each DDL leg shipped through a PlanetScale dev branch + deploy request, the data leg run as a resumable keyset-chunked backfill, and a verify gate between backfill and the destructive drop:

export PLANETSCALE_SERVICE_TOKEN_ID='...'   # service token: branch + deploy-request scopes
export PLANETSCALE_SERVICE_TOKEN='...'      # env, never argv — these never land in shell history

sluice expand-contract \
    --org myorg --database mydb --branch main \
    --dsn "$PROD_BRANCH_DSN" --table users \
    --expand-ddl   'ALTER TABLE users ADD COLUMN full_name VARCHAR(255)' \
    --set          'full_name = CONCAT(first_name, " ", last_name)' \
    --where        'full_name IS NULL' \
    --contract-ddl 'ALTER TABLE users DROP COLUMN first_name' \
    --yes

The load-bearing semantics:

The middle step alone: sluice backfill #

When the schema change itself is already handled — or you're not on PlanetScale — the batched, resumable, online-safe in-place data migration stands alone. It is single-endpoint (reads and updates one database) and walks the table's primary key issuing one bounded UPDATE per chunk, so no statement approaches PlanetScale/Vitess's synchronous-transaction wall (errno 3024) or holds long locks on any engine:

sluice backfill \
    --driver planetscale --dsn "$DSN" \
    --table users \
    --set   'full_name = CONCAT(first_name, " ", last_name)' \
    --where 'full_name IS NULL' \
    --verify

Safe-migrations branches: deploy-ddl + the control-tables bootstrap #

A PlanetScale branch with safe migrations enabled refuses every direct DDL statement (Error 1105, “direct DDL is disabled”) — including sluice's own CREATE TABLE IF NOT EXISTS for its control tables, and the user-table CREATEs a fresh migrate or sync cold-start issues. sluice's ensure paths are detect-first (no DDL at all when the tables are current), and when DDL is genuinely needed the refusal is the coded SLUICE-E-PS-DIRECT-DDL-BLOCKED, echoing the exact refused statement. The way through is the governed channel, one command per statement:

# 1. Print the exact CREATE statements for sluice's control tables (read-only, no credentials)
sluice control-tables ddl

# 2. Ship each statement via a deploy request (dev branch -> apply -> deploy -> cleanup, one command)
sluice deploy-ddl --org myorg --database mydb --ddl '<one statement from step 1>'

# 3a. For sync: pre-create the USER tables the same way, then skip schema-apply
sluice schema preview --source-driver mysql --source "$SRC" --target-driver planetscale --target "$DST"
sluice deploy-ddl --org myorg --database mydb --ddl '<one CREATE from the preview>'   # per table
sluice sync start ... --schema-already-applied

# 3b. For a one-time migrate: pre-create the user tables via deploy-ddl as in 3a, then just run it
sluice migrate --source-driver mysql --source "$SRC" --target-driver planetscale --target "$DST"

deploy-ddl wraps ONE verbatim statement in the full safety machinery — safe-migrations preflight, dev branch with the freshness gate below, apply, deploy request, deploy, skip-revert finalize, always-cleanup — and --dry-run makes zero control-plane calls. It is also the general escape hatch for any ad-hoc schema change on a safe-migrations branch. Ship the control-table statements once, and backfill runs normally against the branch.

The shape gate: bootstrap → fresh migrate just works. In step 3b, migrate needs no flag at all: its pre-create shape gate detects each pre-created table, verifies its column shape matches what the migration would create (names, types, nullability — deploy-ddl-shipped indexes are fine, they're outside the compare), and skips the refused CREATE with an INFO. A pre-existing table whose shape does NOT match refuses upfront with SLUICE-E-TARGET-TABLE-SHAPE-MISMATCH, before any data moves. sync takes --schema-already-applied instead, skipping its schema-apply phase.

One more place safe migrations can bite: the deferred index build after a large copy (on migrate, restore, and sync cold-start alike). Arm the automatic deploy-request index-build fallback with --planetscale-org plus the service-token env vars on whichever command you're running, and still-pending indexes build through a dev branch + deploy request on the already-copied data — no re-copy. Unarmed, the refusal is SLUICE-E-INDEX-DIRECT-DDL-DISABLED (or SLUICE-E-INDEX-STATEMENT-TIME-LIMIT for the ~900 s statement wall), and --resume finishes just the indexes.

Safe-migrations posture: sluice never touches the toggle #

sluice never enables or disables safe migrations on your branch. It is a behaviour change on production (direct DDL becomes blocked from then on), and the enable/disable propagation lag makes toggling it around a run unsafe. So:

The stale-base freshness gate #

A newly created PlanetScale dev branch's schema can lag the production branch it was created from — observed live: a branch created 14 minutes after a deploy still lacked the deployed column, while another created 1 minute after was current; the lag is intermittent and its timing undocumented. A deploy request from such a branch silently proposes reverting the missing schema — on the contract leg, that would drop the freshly backfilled expand column. So expand-contract, deploy-ddl, and the index-build fallback all compare every dev branch's schema against production before applying any DDL, self-heal a stale base once (delete the branch → take an on-demand backup → recreate), and refuse with SLUICE-E-PS-BRANCH-STALE-BASE only if it's still stale after the rebase. Two more gates ride the same machinery: the deploy request's computed diff is fetched and refused if it touches any object the leg never intended, and production's schema is re-verified after a long review wait (a request that sat in a review queue while production moved would otherwise deploy against the old schema).

What sluice checks for you #

Next steps #