Enrichment passes

What fraud-enricher-v2 actually runs when it pops an envelope off fraud.rows.

LLM backend routing: where each LLM pass actually runs (NIM vs LAN GPUs) and why the NIMs carry only a trickle of the traffic is documented in llm-routing.

DIAGRAM TODO (desktop): _enrichment-passes.svg predates the per-defendant disposition passes (disposition-extract, sentence-extract, sanctions-extract, added 2026-05-27) and the 2026-05-21 regex passes. The SVG is intentionally left unedited — regenerate it at the desktop to add the new passes feeding the defendant_dispositions table.

The dispatcher (services/fraud-enricher-v2/index.ts) maps envelope.workflow → WorkflowDef import. The live list (as of 2026-05-27):

const WORKFLOW_NAMES = [
  "titles", "entities", "ocr-recover", "summarize", "refetch-pdf",
  "case-facts", "format-title", "title-llm", "link-references", "scheme-classify",
  // 2026-05-20/21 regex + statistical passes
  "statute-refs", "court-jurisdiction", "keywords-extract", "outcome-detect",
  "mentions-populate", "money-classify", "scheme-detail", "case-facts-llm",
  "anomaly-classify",
  // 2026-05-27 per-defendant disposition passes → defendant_dispositions
  "disposition-extract", "sentence-extract", "sanctions-extract",
];

For each: the worker loads corpus.db (read-only), fetches the freshly-current row, calls workflow.process(doc), and either:

• {patch} → POST to broker /update (whitelisted columns) or wf.applyEndpoint (/apply/case-facts etc.).
• {skip} → write a workflow_runs row with status='skipped'.

Then writes a workflow_runs row, clears documents.queued_at/queued_workflow/envelope_id, ACKs the envelope.

Pass-by-pass

1. case-facts (regex, no LLM)

• Input: documents.{id, body, kind, published_at}.
• Output: new row in case_facts table with defendants_json, first_act_date, plea_date, sentence_date, monetary_amount.
• Apply endpoint: /apply/case-facts.
• When it runs: selectSql matches docs that don't yet have a case_facts row and whose kind is criminal-disposition flavour.
• GPU pool: none.
• Why first: zero LLM cost, perfect canary for the wave-6 cutover.

2. entities

• Input: documents.body.
• Output: rows in entities + mentions. Canonicalised via shared/canonicalize.ts (normalizeTicker, normalizeCik, nameSignature — added 2026-05-18 commit bf97502).
• Apply endpoint: /apply/entities.
• GPU pool: none.

3. triples

• Input: documents.body.
• Output: (subject, predicate, object) rows in triples table.
• LLM: gpumon-ingress pool-postcrime-triples (qwen3:8b, JSON-constrained).
• Used by: D3 Sankey on /stats and the cross-network analysis in ciks-banana.

4. scheme-classify

• Input: documents.body, optional case_facts.
• Output: documents.{scheme, scheme_confidence, scheme_classified_at, scheme_slug}.
• LLM: gpumon-ingress pool-postcrime-triples (qwen3:8b, enum-constrained output).
• Apply endpoint: /apply/scheme.
• Taxonomy: see heuristics.md for the full list.
• GPU call shape: POST /v1/chat/completions with headers from shared/gpumon-headers.ts (x-gpumon-project: fraud-heuristics, x-gpumon-service: fraud-enricher-v2, x-gpumon-pool: pool-postcrime-triples).

5. format-title

• Input: documents.{title, body, page_texts_json, url, kind} (joins sources).
• Output: documents.{title_formatted, title_format_source} and (re-using same migration) documents.reference_filing_json.
• LLM: none.
• Selection: title IS NOT NULL AND (title_formatted IS NULL OR title LIKE '%pdf #%' OR title GLOB '*[A-Z][A-Z][A-Z]*').
• Quality:
  • regex 22% — excellent on press_release, terrible on pdf (grabs first body sentence).
  • composite 32% — OK on complaint (caption + case-num + date), noisy trailing junk on some.
  • fallback 45% — does not enhance, leaves the raw filename-like title. This is what title-llm cleans up.

6. title-llm

• Input: documents.{title, body, kind, case_facts.*, scheme} for docs where title_format_source IN ('fallback') or composite output contains junk markers (Pro hac|Attorneys for|:body).
• Output: same columns as format-title, title_format_source='llm'.
• LLM: gpumon-ingress pool-postcrime-triples (qwen3:8b).
• Synthesises: Bluebook-style caption + short narrative (e.g. "SEC v. Harsh V. Patel, No. 1:26-cv-03203 (S.D.N.Y. Apr. 20, 2026), alleged to have ...").
• Status: smoke ran 2026-05-19 — 3/5 high-confidence wins on litigation_release + complaint + judgment. Excellent on those kinds. See processor/scripts/title-llm-smoke.ts.

7. link-references

• Input: documents.body.
• Output: documents.reference_filing_json — JSON array of {kind, value} references (case nums, accession numbers, citations).
• LLM: none. Pure regex.
• Apply endpoint: /update (whitelisted column).
• Used by: DocumentPreview.svelte to render "Related" anchors (commit 815b460).

8. summarize

• Input: documents.body (chunked).
• Output: documents.{summary_one_sentence, summary_paragraph, summary_llm1_gemma, summary_llm2_qwen, summary_combined, summary_llm3_qwen14b}.
• LLM: gemma + qwen3:8b + qwen3:14b via gpumon-ingress (3-model consensus from the docs-skills pattern). Quorum-skip 3rd pass when 1+2 agree — see federation-queue throughput notes.
• Apply endpoint: /apply/summarize.
• Status: CHANGELOG Phase A marks this "pending — zero-rows investigation". Either selectSql is wrong or the column isn't being read back.

9. refetch-pdf

• Input: documents.{id, url, pdf_path} where pdf is missing.
• Output: writes pdf to body_path / pdf_path.
• LLM: none.

10. ocr-recover

• Input: documents.body LIKE '[OCR_UNRECOVERABLE%'.
• Output: new body + ocr_text + ocr_method + ocr_confidence.
• OCR engine: fraud-tesseract-sidecar (CPU) by default; tested against pool-vision for hard cases.

11–13. Per-defendant disposition passes (2026-05-27)

Three workflows that decorate each named defendant of a disposition-bearing document, writing to the per-defendant defendant_dispositions table via POST /apply/defendant-disposition. All three share the same candidate selection: docs of kind IN (litigation_release, complaint, judgment, indictment, press_release) that have a case_facts.defendants_json list, with no terminal workflow_runs row for that (doc, workflow).

• Candidate list: reuses case_facts.defendants_json (does not re-discover defendants).
• Attribution: regex matches each remedy to the nearest named defendant's text window. Multi-defendant docs fall back to an LLM (pool-postcrime-triples, JSON-constrained, per-defendant output) — see processor/src/extract/defendant-attribution.ts. Rows are tagged source_method = regex | llm.
• Entity link: the broker resolves entity_id from defendant_name via nameSignature(canon(name)) against entities.norm, so per-defendant rows join back to the canonical actor.
• Apply endpoint: /apply/defendant-disposition — UPSERTs per (document_id, defendant_name), COALESCEing columns so the three passes compose without clobbering each other.
• Skips: docs with no defendants or no detected remedy write a workflow_runs skip (no table row). SEC-civil docs legitimately skip sentence-extract (no prison terms).
• Downstream: two derived stats-miner materialisers read this table — actor-recidivism (same entity_id across ≥2 matters) and parallel-proceedings (same entity_id in both an SEC and a DOJ doc on one scheme). They power the dispositions and sentence-length charts on /stats (previously synthetic). See also the config-loop harness in enrichment-configs.md.

Not yet documented pass-by-pass: the 2026-05-20/21 passes statute-refs, court-jurisdiction, keywords-extract, outcome-detect, mentions-populate, money-classify, scheme-detail, case-facts-llm, anomaly-classify are live in WORKFLOW_NAMES and the publisher allowlist; their WorkflowDefs live in processor/src/extract/.

Retry + DLQ

• On retryable failure: republish to fraud.rows with x-attempt+1 (header survives broker restarts), ACK the original.
• After WORKER_MAX_ATTEMPTS=5: NACK with requeue=false → DLQ via x-dead-letter-exchange: federation.dlx → fraud.dlq.
• Unknown workflow → straight to DLQ.

Open enrichment blockers (from _overnight-questions.md)

TODO: Q12 cleanup pass — strip Pro hac vice|Attorneys for|^\d+\.|^Defendant after the trailing date in composite output.

When each pass runs

The fraud-publisher PUBLISHER_WORKFLOWS env gates which workflows the sweeper publishes (code registration in ALL_WORKFLOWS is necessary but not sufficient — the env is the live allowlist). As of 2026-05-27 it lists all 21 workflows, including the three per-defendant passes:

case-facts, format-title, title-llm, link-references, refetch-pdf, scheme-classify,
summarize, ocr-recover, statute-refs, court-jurisdiction, keywords-extract,
outcome-detect, mentions-populate, money-classify, scheme-detail, case-facts-llm,
titles, entities, disposition-extract, sentence-extract, sanctions-extract

NOTE: it was set as an explicit env var (not the spec default) via docker service update --env-add PUBLISHER_WORKFLOWS=.... When adding a new workflow, append it to this env on the running fraud_fraud-publisher service — otherwise the publisher will never sweep it even though the consumer can run it.