Effect v4Module 6 of 8: Selective retries
< PrevNext >
Module 6 of 8Time policysrc/course/examples/retry.ts

Retry the outage, not the bad request

Attach a bounded time policy to the one failure that can actually recover.

First principle

Retry is a decision about time. It only makes sense when waiting can change the outcome.

06 / 08module position
01

The concrete friction

Indiscriminate retry loops amplify damage and waste compute

External network calls fail for different physical reasons. A 503 Service Unavailable or network reset is transient: waiting allows upstream load balancers to recover or connections to re-establish. A 401 Unauthorized or 400 Bad Request is deterministic: re-sending the same invalid token or query will never succeed. Retrying deterministic failures wastes network bandwidth, burns API quotas, and delays returning an actionable error to the user.

First-principles consequence

Retrying without a filter is a category error: it treats permanent logic errors as if they were transient environmental conditions. Retrying without exponential backoff and jitter causes thundering herd problems, knocking recovering services back offline.

baseline-friction.tsbaseline problem
type SearchError =
  | { readonly _tag: "Unauthorized" }
  | { readonly _tag: "IssueServiceUnavailable" };

const searchIssues = async (query: string): Promise<string> => {
  if (query === "unauthorized") {
    throw ({ _tag: "Unauthorized" } satisfies SearchError);
  }

  throw ({ _tag: "IssueServiceUnavailable" } satisfies SearchError);
};

async function searchWithRetry(query: string): Promise<string> {
  for (let attempt = 0; attempt < 3; attempt += 1) {
    try {
      return await searchIssues(query);
    } catch (_error: unknown) {
      // Blindly retries unauthorized requests too.
    }
  }

  throw new Error("search failed");
}
02

The mental model

A retry policy combines an error predicate and a time schedule

Effect.retry decouples the retry policy into two orthogonal decisions: which errors are eligible (the while predicate inspecting tagged errors), and what schedule to apply (exponential backoff with finite upper bounds). Schedule.exponential("20 millis", 2).pipe(Schedule.upTo({ times: 2 })) guarantees backoff and bounds total attempts to two retries.

Attempt issue provider
   ├── Success ────────────────────────────> Return ISSUE-101
   └── Failure:
         ├── InvalidQuery ─────────────────> Fail immediately (no retry)
         └── IssueServiceUnavailable (503)
               │
               ▼ Check schedule (retries < 2)
               ├── Yes ──> Wait 20ms, then retry
               └── No  ──> Exhausted ──────> Fail with 503
03

Minimal working example

Executable source

This is the checked source file used by the course. Read it before you run it.

src/course/examples/retry.tsTime policy
import { Console, Effect, Ref, Schedule } from "effect"

export type IssueServiceUnavailable = {
  readonly _tag: "IssueServiceUnavailable"
  readonly statusCode: number
}

export const IssueServiceUnavailable = (
  fields: Omit<IssueServiceUnavailable, "_tag">
): IssueServiceUnavailable => ({
  _tag: "IssueServiceUnavailable",
  ...fields
})

const retryPolicy = Schedule.exponential("20 millis", 2).pipe(
  Schedule.upTo({ times: 2 })
)

const searchIssueProvider = (attemptsRef: Ref.Ref<number>) =>
  Effect.gen(function* () {
    const attempt = yield* Ref.getAndUpdate(attemptsRef, (count) => count + 1)
    if (attempt < 2) {
      return yield* Effect.fail(IssueServiceUnavailable({ statusCode: 503 }))
    }
    return "ISSUE-101"
  })

export const retryTransientIssueSearch = Effect.gen(function* () {
  const attemptsRef = yield* Ref.make(0)
  const issueId = yield* Effect.retry(
    searchIssueProvider(attemptsRef),
    {
      schedule: retryPolicy,
      while: (error) => error._tag === "IssueServiceUnavailable"
    }
  )
  const attempts = yield* Ref.get(attemptsRef)
  return { issueId, attempts }
})

await Effect.runPromise(
  retryTransientIssueSearch.pipe(
    Effect.flatMap((recovered) =>
      Console.log(
        `recovered ${recovered.issueId} after ${recovered.attempts} attempts`
      )
    )
  )
)
04

Execute and verify

Predict the result, run the command, and compare the output with the model.

Run this command
pnpm exec tsx src/course/examples/retry.ts
Before you run it

Why are two retries reported as three total attempts? Which predicate prevents a permanent search failure from retrying?

Reveal expected output
recovered ISSUE-101 after 3 attempts
05

Error anatomy and edge cases

Retrying indefinitely or without error filtering

Using Schedule.forever without a while predicate on external network calls.

Compiler or runtime diagnostic

If an invalid query or expired credential is submitted, the fiber retries continuously in an infinite loop, pegging CPU usage, spamming the auth provider, and never returning an error to the caller.

Remedy

Bound the schedule with Schedule.upTo({ times: N }) and retry only the tagged failure variants that represent temporary conditions.

06

Active lab exercise

src/exercises/exercise6/starter.ts

Exercise: selective issue-search retries

Retry temporary provider failures twice, but never retry an unauthorized request.

ObjectiveFix starter.ts so only IssueServiceUnavailable retries. UnauthorizedError must fail immediately.
pnpm exec tsx src/exercises/exercise6/exercise.test.ts starter

The harness should fail against the starter. Inspect the assertion, change the starter implementation, and run it again.

Inspect starter code
Broken starter implementation
src/exercises/exercise6/starter.tsstarter
import { Context, Effect, Schedule, pipe } from "effect"

export type IssueServiceUnavailable = {
  readonly _tag: "IssueServiceUnavailable"
  readonly statusCode: number
}

export const IssueServiceUnavailable = (
  fields: Omit<IssueServiceUnavailable, "_tag">
): IssueServiceUnavailable => ({
  _tag: "IssueServiceUnavailable",
  ...fields
})

export type UnauthorizedError = {
  readonly _tag: "UnauthorizedError"
  readonly reason: string
}

export const UnauthorizedError = (
  fields: Omit<UnauthorizedError, "_tag">
): UnauthorizedError => ({
  _tag: "UnauthorizedError",
  ...fields
})

export interface IssueProvider {
  readonly search: (
    query: string
  ) => Effect.Effect<string, IssueServiceUnavailable | UnauthorizedError>
}

export const IssueProvider = Context.Service<IssueProvider>("IssueProvider")

export const searchIssue = (
  query: string
): Effect.Effect<
  string,
  IssueServiceUnavailable | UnauthorizedError,
  IssueProvider
> =>
  Effect.gen(function* () {
    const provider = yield* IssueProvider

    // TODO: retry only IssueServiceUnavailable and stop after two retries.
    // The starter incorrectly retries every provider failure.
    return yield* pipe(provider.search(query), Effect.retry(Schedule.recurs(2)))
  })
Reveal reference solution
Target reference solution
solution.tsverified solution
import { Context, Effect, Schedule, pipe } from "effect"

export type IssueServiceUnavailable = {
  readonly _tag: "IssueServiceUnavailable"
  readonly statusCode: number
}

export const IssueServiceUnavailable = (
  fields: Omit<IssueServiceUnavailable, "_tag">
): IssueServiceUnavailable => ({
  _tag: "IssueServiceUnavailable",
  ...fields
})

export type UnauthorizedError = {
  readonly _tag: "UnauthorizedError"
  readonly reason: string
}

export const UnauthorizedError = (
  fields: Omit<UnauthorizedError, "_tag">
): UnauthorizedError => ({
  _tag: "UnauthorizedError",
  ...fields
})

export interface IssueProvider {
  readonly search: (
    query: string
  ) => Effect.Effect<string, IssueServiceUnavailable | UnauthorizedError>
}

export const IssueProvider = Context.Service<IssueProvider>("IssueProvider")

const retryPolicy = Schedule.exponential("20 millis", 2).pipe(
  Schedule.upTo({ times: 2 })
)

export const searchIssue = (
  query: string
): Effect.Effect<
  string,
  IssueServiceUnavailable | UnauthorizedError,
  IssueProvider
> =>
  Effect.gen(function* () {
    const provider = yield* IssueProvider

    return yield* pipe(
      provider.search(query),
      Effect.retry({
        schedule: retryPolicy,
        while: (error) => error._tag === "IssueServiceUnavailable"
      })
    )
  })
Why this solution works

Use Schedule.exponential("20 millis", 2).pipe(Schedule.upTo({ times: 2 })) and check error._tag === "IssueServiceUnavailable" in the retry predicate. UnauthorizedError bypasses the retry policy and fails immediately.