On this page

Migration runtime

Seagull's migration runner applies SQL files in version order, records a checksum for each one, and refuses to run when an applied file has changed on disk. The same migrations feed code generation: the schema Seagull checks your queries against is the one your migrations build.

Migration input

Up and down files use {version}_{name}.{up|down}.sql names in one directory. Versions are numeric and applied in order; down files are optional. The file reference defines naming, lexical boundaries and migration directives.

CLI operations

OperationBehaviour
sgl migrate upApply pending versions and validate recorded applied files.
sgl migrate down NRoll back the newest N applied versions.
sgl migrate to VApply or roll back towards numeric version V.
sgl migrate statusRead file/history status without running DDL.
sgl migrate validateCompare recorded applied files with current files.
sgl migrate force V, repair [V]Edit recorded history without executing migration SQL; see recovery below.

The CLI reference lists flags and exit statuses. Follow CLI migration for the deployment procedure.

Go operations and backends

migrate.NewMigrator(database, backend, files, directory, options...) returns a *migrate.Migrator. Up, UpTo, Down and DownTo return the number of versions completed and an error (Down refuses a negative count with an error matching seagull.ErrInvalidConfig); Status returns per-file status, and Validate checks applied history. The startup guide shows embedding and readiness.

sdk/migrate and its migrate_backend_* providers do not link the analyser or generator.

BackendUse
migrate_backend_postgres.New()PostgreSQL, with advisory locks.
migrate_backend_timescaledb.New()TimescaleDB: PostgreSQL's backend, which also runs continuous-aggregate statements outside a transaction.
migrate_backend_cockroachdb.New()CockroachDB, with a durable lock table and no migration transactions: CockroachDB has no advisory locks, and does not make schema changes atomic with a transaction. See CockroachDB.
migrate_backend_postgres.NewPgBouncer()PostgreSQL behind PgBouncer in transaction mode, with a lock table.
migrate_backend_mysql.New()MySQL and MariaDB, with GET_LOCK locks keyed within the current schema and no migration transactions, since MySQL commits DDL implicitly. See MySQL and MariaDB.
migrate_backend_sqlite.New()SQLite, with no lock: SQLite serialises writers itself.
migrate_backend_clickhouse.New()ClickHouse, with no lock and no transactions: coordinate concurrent runs yourself.
migrate_backend_duckdb.New()DuckDB, with transactions and no lock: a DuckDB file is opened for writing by one process at a time. See DuckDB.
migrate.CustomBackend(spec)Any other engine, described by a migrate.BackendSpec. See Other databases and drivers.

A backend keeps no state between runs, so one value can be built once and shared by any number of migrators, including ones running at the same time against different databases.

Persisted names and locks

By default the history table is sgl_migrations, seeds are recorded in sgl_seeds, and locks use the sgl_migration_lock and sgl_seed_lock tables, or advisory locks keyed on the history table names. Every backend constructor takes options that change them:

backend := migrate_backend_postgres.New(
    migrate.WithNamespace("app"),            // app_migrations, app_seeds, app_migration_lock, ...
    migrate.WithHistoryTable("app_history"), // rename only the history table
    migrate.WithLock(migrate.LockTable),     // LockAdvisory, LockTable or LockNone
)
fmt.Println(backend.Names()) // every table, constraint and lock key the backend touches

WithHistoryTable renames only the history table, and the migration lock key follows it; the seed table and seed lock stay namespace-derived. The result does not depend on the order of the options. PostgreSQL supports all three lock modes, CockroachDB table and none, MySQL advisory and none, and SQLite, ClickHouse and DuckDB none only.

MySQL's named locks are global to the server, so its lock keys are scoped to the current schema: the migration lock is named CONCAT(COALESCE(DATABASE(), ''), '.', 'sgl_migrations'), replaced by its SHA2-256 digest when that is longer than MySQL's 64-character limit, and Names().LockKey reports that expression (Names().SeedLockKey the seed lock's). Migrators of different schemas on one server therefore never wait for each other. migrate_backend_mysql.WithServerWideLockKey() keeps the bare sgl_migrations key used by earlier releases. Use it while those releases may still migrate against the same server, so every process takes the same lock. Other engines' locks already cover one database, so they ignore the option.

A constructor never fails, so it can be passed inline. An invalid option (a namespace that is not a plain SQL identifier, a history table that would share a name with the seed or lock tables, or a lock mode the database cannot provide) is recorded in the backend and returned by NewMigrator, NewSeeder and NewExecutor, wrapping migrate.ErrInvalidIdentifier or migrate.ErrUnsupportedLock.

Statements

The runner splits each file into statements exactly as the dialect does when sgl generate and sgl vet read it: under the engine's lexical rules, with the splitter the generator uses. Quoted identifiers such as "a;b", MySQL's double-quoted strings and # comments, PostgreSQL's E'...' strings, nested comments and dollar-quoted bodies, and procedural BEGIN ... END bodies all stay within their statement, and a column named begin is just a column. MySQL's DELIMITER lines change the terminator, as in the mysql client, and are not sent to the server:

DELIMITER //
CREATE FUNCTION double_it(x INT) RETURNS INT DETERMINISTIC
BEGIN
    RETURN x * 2;
END //
DELIMITER ;

On PostgreSQL, TimescaleDB and CockroachDB a line that starts with a backslash where a statement could begin is a psql meta-command, such as the \restrict and \unrestrict lines pg_dump 17.6 and later write around a dump: psql runs it itself, so it belongs to no statement, and a dumped schema replays and runs without it.

Backend.Split(content) returns the statements the runner would run for a file.

Transactions

Each migration runs in a transaction with its history row where the database supports transactional DDL: PostgreSQL, TimescaleDB, SQLite and DuckDB. The runner recognises the statements an engine refuses inside a transaction, and runs a file holding one without one: CREATE INDEX CONCURRENTLY, REINDEX CONCURRENTLY and VACUUM on PostgreSQL, a leading VACUUM on SQLite and DuckDB, and on migrate_backend_timescaledb.New() a continuous aggregate created with data or refreshed with refresh_continuous_aggregate. A mention inside a string, a quoted identifier or a comment does not count. For anything else, the directive at the top of a file turns the transaction off:

-- sgl.migration(no_transaction: true)
CREATE INDEX CONCURRENTLY tasks_title_idx ON tasks (title);

A directive the runner cannot read for certain is refused rather than read as asking for nothing: a misspelt key (no_transacton: true), a value other than true or false, a call left unclosed or continued on the next line, text after the call, or a misspelt role such as sgl.migraton. Running a migration inside a transaction it asked to avoid, or the reverse, cannot be undone, so Up and UpTo check every migration they would run, and Down and DownTo every down file, and return an error that names the file, the line and the problem before anything runs. sgl generate and sgl vet report the same directive where it is written.

MySQL, MariaDB, CockroachDB and ClickHouse run no migration in a transaction, so on those backends every file behaves as if it carried the directive; see MySQL and MariaDB and CockroachDB.

A migration that runs without a transaction is not atomic, so keep a statement that needs one in a file of its own. A TimescaleDB continuous aggregate can also be created WITH NO DATA, which runs in a transaction, and filled by a refresh policy:

CREATE MATERIALIZED VIEW hourly_temps
WITH (timescaledb.continuous)
AS SELECT time_bucket('1 hour', ts) AS bucket, avg(temperature) AS avg_temp
FROM readings
GROUP BY bucket
WITH NO DATA;

Data-only seeds and their history rows share a transaction on every built-in backend except ClickHouse. Schema-changing SQL can break that atomicity on engines whose DDL commits implicitly.

When a migration or seed fails inside its transaction, the runner rolls the transaction back; if the rollback fails too, that failure is joined to the error it returns.

Locking

Advisory and table locks coordinate runners using the same lock identity. LockNone adds no cross-process lock; concurrent runs require external coordination. By default acquisition waits for as long as the call's context allows. migrate.WithLockTimeout(d) bounds the wait: when another process still holds the lock after d, the call fails with an error wrapping migrate.ErrLockTimeout and changes nothing. Only the wait is bounded, not the migrations that run once the lock is held. migrate.WithNonBlockingLock() makes it fail immediately with migrate.ErrLockNotAcquired, which suits short-lived jobs that should retry later. Both apply to a seeder's lock too. The CLI's --lock-timeout and --non-blocking flags set them. A lock that cannot be taken is reported as a *migrate.LockAcquisitionError, whose Lock field says whether it was the migration or the seed lock.

The table lock holds a row lock inside an open transaction for the whole run, so the migrations themselves run on a second connection. Give the pool at least two connections when you use it.

The lock transaction sits idle while the migrations run, and a server can end a transaction that idles. The PostgreSQL table lock (the default of migrate_backend_postgres.NewPgBouncer() and migrate_backend_cockroachdb.New(), and WithLock(LockTable) on migrate_backend_postgres.New() and migrate_backend_timescaledb.New()) therefore runs SET LOCAL idle_in_transaction_session_timeout = 0 before it locks the row, which lifts that timeout for the lock transaction only. Before each migration and each seed, the executor runs SELECT 1 in the transaction of any table lock, a custom one included, waiting at most five seconds. When the transaction has ended, the run stops before that migration or seed with an error wrapping migrate.ErrLockLost, since another process may hold the lock by then. The migrations before it stay applied, and the next Up carries on from there.

Two timeouts are out of the lock session's reach:

  • PgBouncer's own idle_transaction_timeout closes a client that idles in a transaction, whatever the server allows. Behind PgBouncer in transaction pooling mode, leave it at 0 (its default) or set it above your longest run.
  • PostgreSQL 17's transaction_timeout, which the lock does not change, ends any transaction that lasts longer than it, the lock transaction included. Where the server, database or role sets it, set it to 0 for the role that migrates, or above your longest run.

Either one ending the lock transaction stops the run with migrate.ErrLockLost.

Up creates the history table after acquisition. With advisory or table locking, this coordinates replicas booting against an empty database. LockNone does not provide that guarantee. The lock tables of the table lock are created before the lock exists; a replica that loses that race retries. Status and Validate take no lock and run no DDL: against a database without a history table they report every migration as pending, so they work on a read replica or with a read-only role. A seeder's Status behaves the same way.

Within one process, the calls on a Migrator (or a Seeder) run one at a time, so a Status from a health check waits for a running Up rather than reading through its locked session. An executor shared by several migrators admits one lock holder at a time, and WithNonBlockingLock() reports a holder in the same process as migrate.ErrLockNotAcquired too. A call whose context has already ended fails before it starts, without waiting for another call or taking the lock.

Releasing the lock is bounded at five seconds, even after the run's context has ended. An advisory lock's unlock statement runs on a context of its own with that bound, and a table lock's commit that takes longer is abandoned by cancelling the context its transaction began on. A driver that watches that context, such as pgx, then gives the commit up and the session is discarded, which frees the lock; lib/pq only asks the server to cancel, so on a connection that no longer answers, its commit can still wait until the network times out. If releasing the lock fails, the session that held it is closed rather than returned to the connection pool, so it cannot keep the lock, and Up or Down returns the release error joined to its own result, alongside the number of migrations it ran.

DuckDB

DuckDB has neither advisory locks nor row locks (it rejects SELECT ... FOR UPDATE), so migrate_backend_duckdb.New() takes no lock. A DuckDB database file is opened for writing by one process at a time, so no other process can migrate it while yours has it open. Within one process, run a database's migrations through one Migrator, whose calls take turns. Migrators built on separate executors are not serialised and concurrent transactions can fail with a conflict. Use one migrator to coordinate runs and their retries.

Each DuckDB migration runs in a transaction with its history row, since DuckDB's DDL is transactional. Seed files run whole, because the driver runs every statement of one call.

MySQL and MariaDB

MySQL DDL commonly commits implicitly, so a transaction cannot make a migration atomic: a failure part-way would keep the earlier statements and lose the history row. migrate_backend_mysql.New() therefore runs each statement on its own and records its progress, as for a file with the no_transaction directive: a migration that fails part-way stays dirty, and the next Up resumes at the statement that failed (see Interrupted runs).

go-sql-driver/mysql runs one statement per call unless the DSN sets multiStatements=true, so seed files are split into statements. When the DSN does set it, migrate_backend_mysql.New(migrate.WithDriverMultiStatements(true)) runs each seed file in one call. Migrations are split either way, so that their progress can be recorded.

A server whose sql_mode holds ANSI_QUOTES reads "..." as a quoted identifier rather than a string. migrate_backend_mysql.New(migrate_backend_mysql.WithANSIQuotes(true)) splits migration and seed files under the rules the generator reads them with when dialect_engine_mysql.WithANSIQuotes(true) or dialect_options.ansi_quotes: true is set, so each statement the generator sees is a statement the runtime runs, and a problem such as an unterminated " run is reported as the quoted identifier the server would read. sgl migrate and sgl seed pass it whenever sgl.yaml sets dialect_options.ansi_quotes. Other backends ignore the option.

CockroachDB

migrate_backend_cockroachdb.New() uses the table lock, made safe for the two ways CockroachDB differs from PostgreSQL.

Under SERIALIZABLE, CockroachDB keeps a SELECT ... FOR UPDATE lock only in memory, and drops it when the range that holds the row splits or moves. A schema change that backfills, such as ADD COLUMN ... NOT NULL DEFAULT or CREATE INDEX, splits ranges, so a plain row lock could be lost halfway through a run and let a second replica in to apply the same migration again. The lock transaction therefore runs SET LOCAL enable_durable_locking_for_serializable = true before it locks the row, which replicates the lock so that it survives. That setting needs CockroachDB v23.2 or later; on an older server the lock fails with migrate.ErrUnsupportedLock, and the alternative is WithLock(LockNone) with migrations serialised outside the migrator. The lock transaction then turns off the idle-in-transaction timeout, as the PostgreSQL table lock does (see Locking).

Migrations do not run in transactions. From v25.1, CockroachDB commits an open transaction before each schema change (its autocommit_before_ddl setting). With that setting off, a schema change that fails its validation at COMMIT, such as a unique index over duplicate rows or a CHECK that existing rows break, reports SQLSTATE XXA00 after the transaction's other writes, the history row among them, have committed. So each statement runs on its own and the history row records progress, as for a file with the no_transaction directive: a migration that fails part-way stays dirty, and the next Up resumes at the statement that failed (see Interrupted runs). A seed file runs as one batch in a transaction with its history row, since CockroachDB commits data atomically; a seed that also changes the schema loses that atomicity, as autocommit_before_ddl commits what came before the schema change, but the seed is still recorded once it has run.

Sessions and the search path

On PostgreSQL, TimescaleDB and CockroachDB every run (an Up, Down or seeder Apply) works on one database session: the lock's own session, the second connection a table lock pins, or, under LockNone, a connection the run pins for itself. A SET in one migration, SET search_path included, therefore lasts for the later migrations of the same run, which is what sgl generate and sgl vet assume when they replay the migrations. SET LOCAL ends with its migration's transaction, or, where a migration runs without one (always on CockroachDB), with its statement.

-- 001_tenant.up.sql
CREATE SCHEMA tenant;
SET search_path TO tenant;
CREATE TABLE widgets (id INT PRIMARY KEY);

-- 002_gadgets.up.sql, applied in the same run: gadgets is created in tenant too
CREATE TABLE gadgets (id INT PRIMARY KEY);

Before anything runs, the runner reads the session's current_schema() and qualifies the history and seed tables with it for the rest of the run ("public".sgl_migrations by default), so a migration that changes the path cannot move them or make the runner lose them. Where the tables live does not change: the schema the connection starts in, as before. Table-lock operations use the separate lock session, and advisory lock keys are fixed before migrations begin. Neither follows a migration's later search-path changes. Status and Validate run on the connection pool and name the tables as the backend does.

When the run ends its session is closed rather than returned to the connection pool, so a setting a migration made reaches neither the application's queries on the same *sql.DB nor a later run. Each run starts from the search path the connection gives (the DSN's search_path, or the role's or the database's default), so a migration applied by a later run, after a deployment, does not see a path that an earlier run's migration set, although the generator replays every migration as one run. A migration that depends on a search path should set it itself, or name its schemas.

Behind PgBouncer in transaction mode a session-level SET is not tied to the client's connection, so use SET LOCAL or schema-qualified names there.

MySQL and MariaDB runs work the same way, with the database in place of the schema: a run works on one session, reads DATABASE() before anything runs and qualifies the history and seed tables with it (`app`.sgl_migrations), so a migration's USE other moves the statements after it, in that migration and the later ones of the run, but not the history, and a later run still finds every migration it applied. The session is closed when the run ends, so the USE reaches neither the connection pool nor a later run. A DSN that names no database leaves the tables unqualified, as before. The generator does not follow a USE when it replays the migrations: it reads a table a later statement creates without naming its database as one in the database the queries run against, so name the database instead (CREATE TABLE other.widgets ...), which the generator and the server read alike. The other engines run as before.

Interrupted runs

An up migration that runs without a transaction is recorded as dirty before its first statement, except on an append-only history backend such as ClickHouse. Up resumes after the last successfully recorded statement checkpoint; committed statements without a checkpoint can run again. Progress writes after successful SQL are best-effort: a failed update logs a warning and execution continues, so recorded progress can lag the database effects.

migrate.WithProgressBatchSize(n) records progress every n completed statements. With successful periodic writes, a crash after the nth statement commits but before its checkpoint can leave up to n statements to replay. Failed checkpoint writes can leave more. Replay requires inspection of the actual effects and idempotent statements where possible.

When a statement fails, or the call's context ends part-way through such a migration, the runner records the last statement that completed before it returns, so the next Up does not run the completed ones again. It writes that checkpoint on a context detached from the cancellation and bounded at five seconds, so it is recorded after a deadline or an interrupt too, and a failure to write it is joined to the returned error. The statement that was running when the context ended is not recorded as completed and runs again on resume, so check whether it took effect. A migration that runs in a transaction is rolled back instead and leaves nothing of itself behind.

A run whose context ends between migrations stops before the next one, and its error says how far it got, such as migration run stopped after 2 of 5 migrations, followed by the context's error and the cause given to context.WithCancelCause or context.WithTimeoutCause; it matches context.Canceled or context.DeadlineExceeded and that cause with errors.Is. The migrations it applied stay applied, and the after-run hooks do not run. Rollbacks and seed runs stop the same way.

The retried migration is the first of the run, participates in its hooks and is counted in the result. If the dirty record no longer matches the next pending file, the run stops with migrate.DirtyMigrationError until the record is resolved.

A migration that fails part-way is left dirty in the same way. Fix what made the statement fail without editing the file, for example by removing duplicate rows a new unique index rejected, and run Up again. It starts after recorded progress, which can precede the failure if completed statements were not checkpointed. When the statement itself was wrong, fix it in the file and run sgl migrate repair, which records the edited file's checksum and asks how many of its statements ran, so the next Up resumes with the fixed one; without it, Up stops with migrate.ChecksumMismatchError. ClickHouse is the exception: it records a migration only once every statement has run, so a failed one is simply run again from the start. A non-transactional down migration also has no forward-resume checkpoint; earlier committed effects can remain when it fails.

With the index, the history row records the version of the statement splitter and a SHA-256 digest of the statements the run completed. A resume skips those statements only when the file still splits so that its leading statements match the digest. When it does not, because the splitter changed between the runs or the row was recorded by a release that kept no digest, Up runs nothing, leaves the migration dirty and returns migrate.ResumeMismatchError, whose message asks you to check which of the file's statements ran and then repair its history. sgl adds the command to run as a hint: resolve the row with sgl migrate repair VERSION, which records the splitter version and digest of the statements you confirm. A program using the library resolves it with Migrator.PlanRepair and Migrator.Repair (see recovering with force and repair). A dirty migration that had completed no statement always resumes from the start.

Releases before this one created the history table without the splitter_version and statements_hash columns. The first Up adds them while it holds the lock; Status and Validate read such a table without them, and earlier releases ignore them.

Down and DownTo refuse to roll back while any migration is dirty. A down file assumes its up file ran in full: run over a partly applied migration, it could fail after dropping objects, and the dirty record would then let the next Up resume as if they were still there. They read the history under the lock and, before any hook or down file runs, release the lock and return an error wrapping migrate.ErrDirtyRollback and a *migrate.DirtyMigrationError naming the migration. Finish the migration with Up, or record how many of its statements ran with sgl migrate repair (Migrator.PlanRepair and Migrator.Repair), then roll back. A rollback with nothing to roll back still returns 0 and no error.

On ClickHouse a rollback removes the history row with a mutation that finishes before Down returns, so an Up or Status straight afterwards sees the version as pending.

Checksum mismatches

Editing a file after it ran makes Up and UpTo fail with migrate.ChecksumMismatchError, and deleting one makes them fail with migrate.MissingMigrationFileError, even when nothing is pending. They check before taking the lock and again once they hold it. Revert the edit and add a new migration for the change. Only when the edit changes nothing the migration did, such as a comment or a reformatted statement, and has been reviewed, record the new checksum with sgl migrate repair. A deleted file must be restored; repair cannot change a migration that has no file.

sgl fmt never rewrites migrations unless asked with --include-migrations, for this reason: see fmt.

Recovering with force and repair

Both commands change only the history table, never run a migration's SQL or hooks, and work under the migration lock, so they wait for (or, with --non-blocking or --lock-timeout, give up on) a run in progress. They describe what they will change and ask for confirmation on a terminal; --yes confirms without asking, and without a terminal they refuse to run unless --yes is given.

sgl migrate force VERSION records that migration as applied and clean, with its files' current checksums, as if it had run: the recovery for a migration someone finished by hand, or for a database whose schema was created outside the migrations. It clears a dirty flag and any recorded progress, and inserts the row when there is none. It never touches other rows: earlier pending migrations stay pending and run with the next up, and it refuses, changing nothing, when a later migration is recorded (roll that back first, or force the later one), so the forced migration is always the last applied one. Unlike golang-migrate's force, which sets one version for the whole database, Seagull keeps a row per migration, so forcing never marks other migrations applied or pending.

sgl migrate repair [VERSION] first lists what it can change, then changes it:

  • every applied migration whose up file, or recorded down file, no longer matches its checksum gets the files' current checksums (the migration is not run again);
  • each dirty migration is shown with its statements, as the file splits now, and the progress the history records; you say how many of its statements ran, as the database shows, and repair records exactly that: 0 makes the migration pending again, so up runs it from its first statement; all of them records it as applied; a number in between records it as dirty after those statements, with the current splitter version and digest, so up resumes with the next statement. An empty answer leaves it dirty. Without a terminal, --completed N answers for the one dirty migration.

VERSION limits the repair to one migration. Repair plans under no lock, then applies the plan under the lock only if the history still matches it; when another run changed the history meanwhile, it changes nothing and exits 1. An applied migration without a file is reported but cannot be repaired.

On ClickHouse, whose history is append-only, force and repair write a migration's new history row before anything is removed. The new row carries the old row's applied time, which is what makes it replace the old one, so a failed write leaves the old row in place. Only when the old row's applied time cannot be read is that row deleted first.

From Go, Migrator.Force(ctx, version), Migrator.PlanRepair(ctx, versions...) and Migrator.Repair(ctx, migrate.RepairOptions{Plan: plan, Completed: map[int64]int{...}}) do the same; they need an executor that implements migrate.HistoryEditorPort, as the executors migrate.NewExecutor builds do.

Options

Every migrate.Option works with migrate.NewMigrator, migrate.NewSeeder, migrate.NewMigratorFor and migrate.NewSeederFor; a seeder ignores the migration-only ones.

OptionPurpose
WithNonBlockingLock()Fail at once with ErrLockNotAcquired instead of waiting for the migration or seed lock. A seeder needs a seed executor that implements migrate.SeedLockTrierPort, as the one migrate.NewSeeder builds does.
WithLockTimeout(time.Duration)Give up on the migration or seed lock after this long, with an error wrapping ErrLockTimeout; zero, the default, waits until the context ends.
WithBeforeMigration, WithAfterMigration, WithBeforeRun, WithAfterRunHooks around each migration or each run, for logging, metrics or environment checks. A nil hook is refused with an error matching seagull.ErrInvalidConfig. A hook that returns an error stops the run; a hook that panics stops it with a *migrate.HookPanicError rather than crashing the process, and its stack is logged once at warning level.
WithDirectivePrefixes(canonical, aliases...)The prefixes recognised in -- sgl.migration(no_transaction: true); the default is sgl and seagull.
WithLogger(*slog.Logger), WithObserver(observe.ObserverPort)Where logs and spans go; by default the ones carried by the context.

A migrator or seeder reads at most 10,000 entries from its directory, 4 MiB from each file and 1 GiB across every file in one call. A larger file fails with an error matching seagull.ErrFileTooLarge. A directory with more entries fails with one matching seagull.ErrLimit before any file is read, and so does a call that reads more bytes. Each call, such as Up, Status or Apply, reads the files afresh within its own cap.

Backend options (migrate.WithNamespace, WithHistoryTable, WithLock and WithDriverMultiStatements, and migrate_backend_mysql.WithServerWideLockKey and WithANSIQuotes) configure the backend, and the executor option WithProgressBatchSize the executor that migrate.NewExecutor builds for migrate.NewMigratorFor. NewExecutor refuses a WithProgressBatchSize below one with an error matching seagull.ErrInvalidConfig. Migration and seed durations are measured with the system clock.

Other databases and drivers

The backends cover database/sql. An engine without a built-in backend uses a migrate.BackendSpec passed to migrate.CustomBackend(spec, options...). The spec provides the history and seed SQL, lexical rules, lock operations and separate transaction capabilities for migrations and seeds. The migrate API reference lists its fields; the implementation guide shows construction and verification against the actual engine.

AppendOnlyHistory records completed files without resumable progress updates, as on ClickHouse. HistoryReadModifier supplies a suffix when reading that history, and MultiStatementSeeds allows a seed file to run in one call. An incomplete spec is reported by the executor constructor with an error matching seagull.ErrInvalidConfig.

CustomBackend calls each of the spec's renderers once, for the names and bind-parameter markers the backend uses, and the executors reuse the SQL they returned. A renderer that panics is recorded as an incomplete spec is: the constructors return an error matching seagull.ErrInvalidConfig and seagull.ErrPanic, and the stack is logged once at warning level. Split and the Rules callbacks run on each file instead. A panic in Split fails that call with an error matching seagull.ErrPanic, its stack logged once at warning level, and a panic in a Rules callback fails it with an error naming the callback and the panic value, which does not match seagull.ErrMalformedSQL, its stack also logged once; either way the migration or seed runs nothing.

A table lock must accept SELECT 1 in its transaction, which the executor runs before each migration and seed to confirm the lock is still held (see Locking). An advisory lock's Wait query should wait a bounded time on the server, as MySQL's GET_LOCK(name, 5) does. When it returns without the lock, the next try starts at least half a second after the one before, so a Wait that returns at once does not poll in a tight loop.

For anything else (a native driver, another store for the history), implement migrate.ExecutorPort and migrate.SeedExecutorPort and pass them to migrate.NewMigratorFor(executor, files, directory, options...) and migrate.NewSeederFor. The migrator calls EnsureMigrationTable only while it holds the lock, and never from Status or Validate, so AppliedVersions and AppliedSeeds must return no rows, not an error, when their table does not exist yet. An executor splits files under the engine's lexical rules, as the backends do, records a dirty migration's progress with LastStatement, SplitterVersion and StatementsHash, and resumes a retried one only when those still identify the file's leading statements; the migrate.ExecutorPort documentation spells out the contract, which is Evolving (see Compatibility). The useTransaction argument reflects only the no_transaction directive, so an executor also runs outside a transaction a statement its engine refuses inside one. The database/sql migration executor is available on its own as migrate.NewExecutor; migrate.NewSeeder builds the database/sql seed executor itself.

Two interfaces are optional. An executor that also implements migrate.HistoryEditorPort (SplitStatements, RecordApplied and RecordProgress) supports Force, PlanRepair and Repair; without it they return an error matching seagull.ErrInvalidConfig that says the executor cannot rewrite the history. A seed executor that implements migrate.SeedLockTrierPort supports WithNonBlockingLock on a seeder. With WithLockTimeout, the context AcquireLock receives ends once the lock is held, so an executor whose lock lives in a transaction begins that transaction on context.WithoutCancel(ctx) and ends it only in ReleaseLock.

Errors

ErrorWhen
migrate.ErrLockNotAcquiredThe lock is held and WithNonBlockingLock is set.
migrate.ErrLockTimeoutThe lock was still held after WithLockTimeout's timeout.
*migrate.UnknownMigrationErrorForce or PlanRepair named a version no file has.
*migrate.LaterMigrationErrorForce was asked for a migration with a later one recorded.
migrate.ErrRepairOutOfDateThe history changed between PlanRepair and Repair; nothing was changed.
seagull.ErrInvalidConfigForce or repair with an executor that is not a migrate.HistoryEditorPort; the error says the executor cannot rewrite the history.
*migrate.NoDownMigrationErrorA rollback reached a version without a down file; Version names it.
*migrate.ChecksumMismatchError, *migrate.DownChecksumMismatchErrorAn applied file changed on disk.
*migrate.DirtyMigrationErrorAn interrupted migration no longer lines up with the files, or, with migrate.ErrDirtyRollback, names the dirty migration that blocks a rollback.
migrate.ErrDirtyRollbackDown or DownTo was asked for while a migration is dirty; nothing ran. Finish it with Up, or repair it, then roll back.
*migrate.ResumeMismatchErrorAn interrupted migration's recorded progress no longer matches how its file splits, so it is not resumed; resolve it with Migrator.Repair (sgl migrate repair VERSION).
*migrate.MissingMigrationFileErrorAn applied version has no file any more.
*migrate.MigrationExecutionError, *migrate.SeedExecutionErrorA statement failed; the error names the file and statement.
*migrate.LockAcquisitionErrorThe migration lock, or from a seeder the seed lock, could not be taken; Lock names which.
migrate.ErrLockLostA table lock's transaction ended during the run, so another process may hold the lock; the run stopped before the migration or seed the error names.
*migrate.DuplicateMigrationVersionError, *migrate.DuplicateSeedVersionErrorTwo files share a version.
*migrate.HookPanicErrorA hook panicked; the error carries the panic value, and the stack is logged once at warning level through the migrator's logger (WithLogger's, or the context's).
seagull.ErrPanicA panic recovered from code the migrator called, such as a hook or a backend's callback; *migrate.HookPanicError wraps it.
seagull.ErrInvalidConfig, seagull.ErrFileTooLarge, seagull.ErrLimitUnusable configuration, a file over the size limit, or more files or bytes than the migrator reads in one call.

Two refusals have no type of their own, and nothing has run when either is returned. A migration about to run whose sgl.migration directive cannot be read for certain, such as no_transacton: true, or whose directive has an unknown role, such as sgl.migraton, stops the run with an error naming the file, the line and the problem. A migration about to be applied that holds another tool's down section, after a marker such as -- +goose Down, stops it with an error naming the file, the marker's line and the tool.

Seeds

Seeds hold reference data and fixtures. They live in their own directory, use the same numbered names (001_countries.sql). Apply skips versions already recorded as successfully applied:

sgl seed apply
sgl seed status
sgl seed reseed     # clear the seed history and apply every seed again

From Go, migrate.NewSeeder(database, migrate_backend_postgres.New(), seeds, "db/seeds") returns a *migrate.Seeder with the same three operations: Apply, Status and Reseed. Write seeds to be idempotent (INSERT ... ON CONFLICT DO NOTHING), since reseed runs them again.

Examples for this page