Data migrationsDefined in TypeScript
Migrate data between APIs, databases, files, and CMSes. Define the work in TypeScript and Effect. Keep migration progress between runs and resume where you left off.
pnpm add migrate-sdk effect01 / define
Your source. Your mapping.
Connect source data, destination calls, and saved results in one migration definition.
Emit typed source items.
A Source declares the payload schema, stable identity, and version of each item it emits.
const = .({ : ., : ., : .,});const = .({ ..., : "./authors.csv", : , : .({ : "id", : "author-id@v1" }),});Call a typed destination helper.
Processing receives decoded source data. The destination helper accepts and validates its own schema-defined input.
const = .({ : ., : .,});const = (: typeof .) => .()().(.());Preserve destination results.
A Tracking Record Contract defines the durable result each successful item must preserve.
const : = .("processAuthor")( function* () { const = .; const = yield* ({ : ., : `${.} ${.}`, }); yield* .({ }); });Compensate from saved state.
A Rollback Pipeline uses saved tracking evidence to undo destination work without reading the source again.
const : = () => { const = .?.; return === ? . : ();};Compose your capabilities.
A Migration Definition connects the Source, processing, tracking, rollback, and Migration Store.
export const = .({ : "authors", : , : , : , : , : .({ : "./.migrate" }),});02 / run
Run migrations your way.
Choose what runs.
We’ll work out the order.
Choose migrations individually, by group, or all at once. Declared requirements determine what runs first.
Content group
Routing group
migrate run articles --with-dependencies --planChoose how much to run.
Or exactly which items.
Limit the number of items, target specific source IDs, or run every eligible item.
0 of 20 items processed
migrate run articles --limit 5Run on your server.
Follow from your terminal.
Follow remote runs, including work started by cron or another client. With Workflow SDK, execution keeps going while you disconnect and come back.
240 / 300
items completed · run-018240 / 300
items completed · run-018Following a run started by cron.
migrate-tui --server https://migrations.example.com03 / deploy
Your code. Your infrastructure.
Start in your application. Add a server for remote access, or Workflow SDK for durable execution.
Use the SDK directly in your application. A server is optional.
Run from codeBuilt with Effect.
Typed errors, schema validation, and retries with exponential backoff are part of your TypeScript code.
Effect platformNode.js · Bun · Deno · Browser ↗Runtime support depends on the clients, sources, and stores you use.
Terraform-style state. For content and data.
Keep a record of what ran and what it created. Store that record in your infrastructure so the next run can use it, wherever it starts.
Choose a migration storeGet started with Migrate SDK.
Define a migration, or work through a complete recipe.