Migrate many databases or schemas at once

Fan a whole MySQL server or a multi-schema Postgres source out to same-named target namespaces in one run.

By default migrate and sync start move the one database (MySQL) or schema (Postgres) named in the source DSN. The multi-namespace flags move all of a server's databases, or all of a Postgres source's schemas, in a single run — snapshot and CDC both — fanning each source namespace out to a same-named target namespace. Reach for this with a multi-tenant MySQL server (one database per tenant), a Postgres database holding several application schemas, or any "migrate the whole server" job.

The unifying idea is that a MySQL database is the rough equivalent of a Postgres schema. So there's one internal routing with two spellings: use the --*-database form on a MySQL source and the --*-schema form on a Postgres source. They're synonyms — mixing both spellings in one invocation is a loud error.

Within a form, include / exclude / all are mutually exclusive. System namespaces are always excluded (information_schema, performance_schema, mysql, sys on MySQL; pg_catalog, information_schema, pg_toast, pg_temp* on Postgres). When any namespace-scope flag is set, the source DSN's database/schema is optional — sluice connects to the server (or, on PG, to the database) rather than a single namespace.

Postgres source: every schema in one run #

A Postgres database holding sales, billing, inventory → one Postgres target, each schema recreated (auto-created if absent) under its own name:

sluice migrate \
    --source-driver postgres --source 'postgres://user:pw@src/appdb?sslmode=require' \
    --target-driver postgres --target 'postgres://user:pw@dst/appdb?sslmode=require' \
    --all-schemas

Continuous sync is identical — just sync start with a --stream-id. Scope with globs, or take everything except a couple:

# only the app_* schemas (plus public)
sluice migrate ... --include-schema 'app_*,public'

# everything except the staging schemas
sluice migrate ... --exclude-schema 'scratch,tmp_load'

MySQL server: every database → Postgres in one run #

A MySQL server hosting one database per tenant/service → a single Postgres target, each MySQL database recreated as a same-named PG schema (auto-created). Note the source DSN has no database after the / — with --all-databases it's a server connection:

sluice migrate \
    --source-driver mysql    --source 'root:pw@tcp(src:3306)/' \
    --target-driver postgres --target 'postgres://user:pw@dst/warehouse?sslmode=require' \
    --all-databases

MySQL shop / crm / analytics land as PG schemas shop / crm / analytics under warehouse. When the target is also MySQL, each source database is recreated via CREATE DATABASE IF NOT EXISTS — same names, no manual pre-creation.

Fan-IN: many sources → one target namespace #

The reverse shape — several independent source databases (e.g. per-microservice MySQL databases) consolidated into one Postgres analytics schema. This isn't a --all-* fan-out; it's N separate runs, each pinned to the same target namespace with --target-schema (Postgres-target-only; it prefixes every emitted object and auto-creates the schema):

# service A → warehouse.analytics
sluice migrate --source-driver mysql --source 'root:pw@tcp(svc-a:3306)/orders' \
    --target-driver postgres --target 'postgres://user:pw@dst/warehouse?sslmode=require' \
    --target-schema analytics

# service B → the SAME warehouse.analytics (run separately)
sluice migrate --source-driver mysql --source 'root:pw@tcp(svc-b:3306)/users' \
    --target-driver postgres --target 'postgres://user:pw@dst/warehouse?sslmode=require' \
    --target-schema analytics

Rename a namespace on the way #

By default every source namespace lands in a same-named target. To route one to a differently-named target — consolidating legacy_app into app, or namespacing each tenant under a prefix — pass --map-database SRC=DST (MySQL source) or --map-schema SRC=DST (Postgres source), repeatable, for both snapshot and CDC:

# MySQL databases shop / crm → PG schemas storefront / sales (analytics keeps its name)
sluice migrate \
    --source-driver mysql    --source 'root:pw@tcp(src:3306)/' \
    --target-driver postgres --target 'postgres://user:pw@dst/warehouse?sslmode=require' \
    --all-databases \
    --map-database shop=storefront \
    --map-database crm=sales

The spelling rule matches the fan-out flags — --map-database on a MySQL source, --map-schema on a Postgres source; mixing both in one run is a loud error. The rename is purely a target-side routing: source-keyed --redact and --type-override still match on the original source name, so a remap never quietly disables a redaction rule.

Views, and namespaces that hold only views (v0.161.0) #

A database or schema that holds views but no tables is migrated. Through v0.160.4 migrate returned early from such a namespace with source schema has no tables; nothing to migrate, before its views phase, so in an --include-database, --include-schema or --all-* fan-out a reporting schema of views was left out at exit 0, a single-namespace migrate of a views-only database did nothing, and the sync start fan-out cold start skipped it the same way (MEDIUM, silent omission, Bug 317). Only a namespace with neither tables nor views returns early now.

• The deferred views pass. migrate and the sync start fan-out copy every namespace's tables first, then create every namespace's views in one final pass, so a view reading a namespace that sorts after its own finds its tables. The pass retries a view that reads a view in another namespace while any view is still being created, and fails loudly, naming every pending view, when a round creates none. If migrate's final views pass fails, re-run it with --resume; each namespace is already recorded complete, so a plain re-run says "already complete".
• Materialized views. The Postgres writer refreshes a materialized view that already exists on the target with the same definition, instead of failing "already exists", so a resumed or pre-created target ends with current rows. One with a different definition refuses, naming it. The comparison is of deparsed text, so different server major versions, or an equal body that deparses differently, refuse loudly too.
• An existing target view is replaced (open, MEDIUM, silent; VIEW-REPLACES-EXISTING-TARGET-VIEW). A plain view of the same name already on the target is replaced by the source's body without warning; the populated-target check grades tables only, and a views-only namespace has none. Postgres materialized views refuse a different definition instead (above).
• SYNC-VIEWS-ONLY-NAMESPACE. A single-namespace sync start of a views-only schema has no table to stream, so its views would not be created; it used to exit 0 with an INFO line. It now refuses, naming the views, and so does its --dry-run. Create them with sluice migrate once the tables they read exist on the target, or pass --skip-views to accept a run that streams nothing (it exits 0 with "nothing to stream"). The check runs after --skip-views and --include-view / --exclude-view, so a run that filtered every view out gets the plain "nothing to stream".
• Table filters. --include-table / --exclude-table, or PlanetScale's default _vt_* exclusion, do not refuse a views-only namespace, which has no tables to exclude. A namespace that has tables and loses all of them to the filter still refuses if it has views; one with no views left is skipped on a Postgres source and refused on a MySQL source. The skip is per namespace, because an --include-table naming one schema's table is a normal fan-out. But when the filter matches no table in any selected namespace (a typo in --include-table, say), a Postgres sync start fan-out now refuses ("in every selected namespace") instead of starting CDC over nothing at exit 0 (a Postgres fan-out migrate already refused that case on v0.160.4, and still does). migrate refuses after the per-namespace loop, before the deferred foreign-key and views passes; the sync start fan-out refuses before the deferred views, the run's completion record and CDC. A MySQL source already refused at the first namespace.
• View refusals are per namespace. VIEW-QUALIFIER-AMBIGUOUS and VIEW-BODY-CROSS-ENGINE (v0.160.4+) fire before their own namespace copies anything, but namespaces earlier in the order have already copied by then, as with every per-namespace preflight. Re-run with --exclude-view against a fresh target, or with --reset-target-data --yes for a sync. VIEW-BODY-CROSS-ENGINE refuses a non-public view of a Postgres multi-schema fan-out into MySQL, which used to create there.
• Renamed namespaces. Since v0.160.4 a view copied under --map-database or --map-schema reads the renamed target namespace, not the original (Bug 314). Still open: a view over a different schema that the same run also renames keeps naming the old name (--include-schema app,public --map-schema app=app2 with a public view over app.t).

The documented edges #

• --slot-name applies to a multi-schema Postgres run, on both its cold start and its warm resume (since v0.139.0). A logical slot is database-wide, so a spanning multi-schema sync uses exactly one — and before v0.139.0 the fan-out ignored the flag and always used the default name, which meant two multi-schema streams against one database could not coexist and the slot recorded in the CDC state row was not the one you named. Name a slot per stream when you run more than one.
• A multi-schema sync start refuses a table with no usable replica identity, before it publishes anything (since v0.139.0). Postgres will not UPDATE or DELETE a published table that lacks one, so publishing first and checking later would break your own application's writes. The refusal carries SLUICE-E-SOURCE-REPLICA-IDENTITY and names the offending tables; the remedy is a primary key or a REPLICA IDENTITY FULL. --exclude-table is NOT a remedy here. It is one on a single-schema sync, whose publication is scoped to the tables you named — but this path's slot is database-wide, so its publication is FOR ALL TABLES and an excluded table is still published, its writes still broken. Since v0.141.0 that case is warned rather than silent: UNSELECTED-NAMESPACE-EXPOSURE names every at-risk table the refusal did not grade, including the ones you excluded. Since v0.148.3 the same marker also fires when the check could not RUN — most often a role that cannot read pg_publication_tables — and says "could not check" rather than naming tables. Read that as exposure UNKNOWN, not exposure absent.
• Cross-database / cross-schema foreign keys are refused loudly. A fan-out validates that FK referents are inside the selected set; an out-of-scope FK fails loudly at the deferred FK pass (after the copy), never silently dropped.
• Separate Postgres databases are one run each. A PG connection is scoped to a single database, so --all-schemas covers every schema within the connected database; moving N separate PG databases is N runs (one --source DSN each).
• PlanetScale-MySQL is a single keyspace and isn't a multi-namespace target — fanning several source databases into one PS-MySQL branch would collapse and collide. PlanetScale-Postgres behaves like regular Postgres and takes --all-schemas fine.
• Default routing is same-name; rename with --map-database / --map-schema. Each source namespace lands in a target namespace of the same name unless you remap it (see below). For the fan-IN shape use --target-schema.