Errors and intentional skips

A Migration Item Error is a normalized failure record stored in Migration Item State. It explains why one item failed after the execution has ended. It is distinct from the live Effect error that interrupted processing and from diagnostic context recorded alongside that failure.

The failure boundary determines what the runtime can preserve and whether execution can continue. An item-level failure can become a durable outcome; a failure to persist that outcome cannot safely be treated the same way.

Live typed failures

Effect represents expected failures in the error parameter of Effect<Success, Error, Requirements>. A tagged error allows process code to distinguish failure categories and inspect their properties before choosing recovery.

The SDK exposes several boundary-specific error types:

Planning, runtime, and rollback preflight have additional typed errors. Their purpose is to identify an invalid operation or unavailable capability before it can safely proceed.

User-authored helpers can expose more specific tagged errors. Preserve the fields needed for decisions such as retry eligibility, HTTP status handling, or a recoverable conflict. Map those errors to an SDK boundary error when required, retaining their cause for live inspection.

Programming defects and interruptions are separate from expected typed failures. Ordinary Effect.retry does not retry them. Do not convert every unexpected defect into a transient provider failure.

Durable Migration Item Errors

The runtime normalizes an item failure into these fields:

Raw causes, thrown objects, and complete provider responses are not copied into this record. Put essential failure information in the message; save extra context explicitly as a journal diagnostic. Arbitrary properties on a custom error are not automatically preserved as error details.

The runtime boundary and error class are separate. For example, a DestinationError returned through a process is currently saved with kind: "process" and errorTag: "DestinationError".

Failure lifecycle

A failed item can retain destination evidence from partial work. Failure does not undo writes. Read Tracking for preservation rules and Rollback for compensation.

A completed source pass may contain failed items. Completion describes coverage, while item state describes processing outcomes.

Destination Journal Diagnostics

A Destination Journal Diagnostic records explicitly marked context inside the Pipeline Execution Scope. It contains severity (info, warning, or error), a message, and optional JSON-object details. These details differ from validation-oriented Migration Item Error Details.

Tracking.logDiagnostic appends that entry and emits an Effect log. The journal append is not suppressed by the configured minimum log level. Ordinary Effect.log calls remain observability output and do not become durable item evidence.

Diagnostics can accompany success, failure, or skip. They explain conditions but do not establish that a destination change occurred. Operator-facing Migration Messages project saved item errors, state reasons, and journal diagnostics for later inspection.

Skip an item on purpose

Skip Item is a successful Process Pipeline result with a reason. skipItem(reason) constructs it; returning it records skipped item state without requiring a newly staged tracking record.

import { skipItem } from "migrate-sdk";

export const excludedByPolicy = skipItem("Excluded by migration policy");

The enclosing process must explicitly return the skip. A nested helper's result does not automatically exit its caller. Returning a skip before destination work prevents that work; returning it afterward does not compensate completed writes. Actual failures use the error channel.

Use a skip for an intentional policy outcome, an error for work that failed, and a diagnostic for context. These distinctions drive retry selection and operational reporting.

Give API failures a useful type

The API cookbook applies these boundaries to transport code. The following optional helper shows a more detailed classification: transient request failure, invalid JSON, and schema-invalid payload remain different error types.

import { Data, Effect, Schema } from "effect";

export class ApiRequestError extends Data.TaggedError("ApiRequestError")<{
  readonly message: string;
  readonly transient: boolean;
  readonly status?: number;
  readonly retryAfterMs?: number;
}> {}

export class InvalidApiResponse extends Data.TaggedError("InvalidApiResponse")<{
  readonly message: string;
}> {}

const retryAfterMilliseconds = (value: string | null): number | undefined => {
  if (value === null) {
    return;
  }
  const seconds = Number(value);
  if (Number.isFinite(seconds) && seconds >= 0) {
    return seconds * 1000;
  }
  const date = Date.parse(value);
  return Number.isFinite(date) ? Math.max(0, date - Date.now()) : undefined;
};

const transientTransportFailure = (cause: unknown): boolean => {
  if (!(cause instanceof Error)) {
    return false;
  }
  if (cause.name === "TimeoutError") {
    return true;
  }
  const detail = cause.cause;
  return (
    typeof detail === "object" &&
    detail !== null &&
    "code" in detail &&
    ["ECONNRESET", "ETIMEDOUT", "EAI_AGAIN"].includes(String(detail.code))
  );
};

// Contract: GET {baseUrl}/authors/{id} returns { id: string, name: string }.
// Supply your own API URL. This module makes no requests when imported.
const requestAuthor = (baseUrl: string, id: string) =>
  Effect.gen(function* () {
    const response = yield* Effect.tryPromise({
      try: (signal) =>
        fetch(`${baseUrl}/authors/${encodeURIComponent(id)}`, {
          signal: AbortSignal.any([signal, AbortSignal.timeout(10_000)]),
        }),
      catch: (cause) =>
        new ApiRequestError({
          message: "Author request did not complete",
          transient: transientTransportFailure(cause),
        }),
    });

    if (!response.ok) {
      const retryAfterMs = retryAfterMilliseconds(
        response.headers.get("retry-after")
      );
      yield* Effect.tryPromise(
        () => response.body?.cancel() ?? Promise.resolve()
      ).pipe(Effect.ignore);
      return yield* Effect.fail(
        new ApiRequestError({
          message: `Author request returned HTTP ${response.status}`,
          status: response.status,
          transient: [408, 429, 502, 503, 504].includes(response.status),
          ...(retryAfterMs === undefined ? {} : { retryAfterMs }),
        })
      );
    }

    const body = yield* Effect.tryPromise({
      try: () => response.text(),
      catch: (cause) =>
        new ApiRequestError({
          message: "Author response body did not complete",
          transient: transientTransportFailure(cause),
        }),
    });

    return yield* Effect.try({
      try: (): unknown => JSON.parse(body),
      catch: () =>
        new InvalidApiResponse({
          message: "Author response was not valid JSON",
        }),
    });
  });

export const ApiAuthor = Schema.Struct({
  id: Schema.NonEmptyString,
  name: Schema.NonEmptyString,
});

export const getAuthor = (baseUrl: string, id: string) =>
  requestAuthor(baseUrl, id).pipe(
    Effect.flatMap(Schema.decodeUnknownEffect(ApiAuthor))
  );