Browse this guideTurn Extract on
DevelopmentUpdated 2026-10-17

Turn Extract on

A new Site's worker runs Capture only: it copies each message and its attachments into the Deposit and stops at captured. Turning Extract on starts model calls, which cost money and classify customer text. Do the steps in this order, and do not turn it on until every earlier step has passed its check.

Contextualize, Govern, and the roster backfill run only when Extract is on. Until then, backfill requests wait. There is no throttle and no cutoff date: once Extract is on, every waiting captured row is processed.

1. Everything is deployed

2. The customer has entered projects and rosters, and the typology is seeded

The customer's administrator enters the projects and the people on each roster in the web app. The model classifies every text against the Site's categories, so the Site needs a vocabulary before Extract starts.

  1. Seed the starter typology, once, before any administrator adds a category. From site/db, with the Site's operator connection string set for this run only:

    $env:SITE_DATABASE_URL = "<operator connection string>"
    uv run python scripts/seed_typology.py --file seeds/typology/remodeler.yaml
    

    The script validates the file before it connects, then in one transaction creates the typology version, the categories, an operator service principal that cannot sign in, and one typology.seeded Daybook event.

    • Check: it prints one summary line and exits 0. The categories appear in the web app's Typology screen.
    • If not: it refuses a Site that already has a live category, so it cannot run twice. A malformed file is refused before any connection. Exit 2 is a missing connection string and exit 1 any other failure.
  2. To undo or change the vocabulary, retire categories in the Typology screen. Never run the seed again.

    • Check: an administrator can add a category, edit its description, and retire it there.

Script and seed: seed_typology.py and remodeler.yaml. Run the local Site database shows the same run against a local database.

3. Size the cost

  1. Count the captured rows waiting. Count by state only; do not read message content.
  2. Estimate by attachments as well as rows. The configuration's estimates are about $0.02 for an SMS, $0.04 for a message with three photos, and $0.20 for a 20-page PDF. One call is capped at an estimated $0.25.
    • Check: you have a monthly figure you are willing to spend, written down.

These are estimates, not measurements. Real usage is what they are measured against.

4. Record the start of the calibration period

The calibration record is what a prompt, model, schema, or threshold change is checked against.

  1. A reviewed pull request gives the current entry in docs/calibration/beta-1.md its Started date and sets its Status to active.
  2. After that the record is append-only. Any later change to a prompt, the model, the output schema, or a threshold adds a new entry in a reviewed pull request. A prompt file the record names is never edited.
    • Check: the last entry in the record is the one in force and matches the worker's configuration in site/backend/worker/config.py, as the record's own rules require.

5. Create the key and switch it on

  1. Create an Anthropic API key for this Site's worker alone.
  2. Set a monthly spend limit on it that matches step 3.
    • Check: the limit shows on the key's page.
  3. Confirm the model's published price matches the prices in site/backend/worker/config.py. If it does not, stop and raise it before switching Extract on.
  4. Set ANTHROPIC_API_KEY on the Railway worker service. Then set WORKER_EXTRACT_ENABLED to true and redeploy the worker. Set the key before the switch.
    • Check: the worker stays up. A captured row becomes extracted, then contextualized, then governed, and its intake ends complete. Open the web app and see the extracted artifacts and any Review Queue items.
    • If not: the worker's log names the setting. Any WORKER_EXTRACT_ENABLED value other than true, false, 0, or unset or empty stops startup rather than guessing.

To turn it off again

Unset WORKER_EXTRACT_ENABLED (or set it to false) and redeploy the worker. Rows already extracted stay extracted. Waiting captured rows stay captured, ready for the next Extract-enabled run.

Reprocessing a project

An administrator can rerun a project's earlier messages from the web app (Reprocessing a project). Nothing needs configuring; the worker does the work.

What a refusal looks like

If the model refuses to process a text, the text goes straight to extraction-failed: there is no retry and no fallback model. The administrator sees it in the Review Queue and decides what to do with it. A pile of refusals is the signal to read the Incident runbooks, not to retry.

An Anthropic refusal of the account is different: the worker logs Anthropic refused the account with its HTTP status, error type, and request id, then stops and Railway sends a crash email. The held row is released untouched rather than becoming an extraction-failed item. Raise the Anthropic limit or wait for the first of the month at 00:00 UTC; if the key was refused, fix it. Then restart or redeploy the Railway worker. See Incident runbooks.