Design a system that extracts supplier, invoice number, dates, line items and totals from about 100,000 invoice PDFs a day, in many layouts and languages, and feeds them into the finance system.

Run each invoice through a model that reads the PDF and returns JSON matching a fixed schema, validate the result with business rules in code, post the invoices that pass, and send the rest to a human review queue whose corrections become test cases. Process in bulk asynchronously because nobody waits for each invoice, and measure accuracy per field against a labelled set. The model does the reading; code does the checking.

1. Clarify

• Deadline: same day is fine? Then asynchronous batch processing.
• Accuracy target per field: a wrong total or bank account costs far more than a wrong description.
• Volume and size: 100,000 a day at 1 to 3 pages is 3 to 10 million pages a month, which drives cost.
• What exists: supplier master data, purchase orders, past invoices to label.

2. Pipeline

inbox / upload ──▶ store original (immutable) ──▶ dedupe (file hash, supplier + number)
   ──▶ extraction (PDF + schema, batched) ──▶ validation rules
         ├─ pass ──▶ finance system (idempotent by invoice ID)
         └─ fail or low confidence ──▶ review queue ──▶ corrections ──▶ labelled set

3. Extraction

• PDF as input: each page is seen as an image plus its extracted text, so tables, stamps and layout are read together. No separate OCR step for most invoices.
• A JSON schema for the output (supplier, tax IDs, dates, currency, line items, totals), enforced with structured outputs so every response parses. Handle the 2 exceptions: refusals and responses cut off by max_tokens.
• Batch API: half the price of real-time calls, up to 100,000 requests per batch, most batches done within an hour. Real-time calls only for urgent invoices.
• Ask for evidence: the page and the text each value came from, which makes review much faster.
• Smaller model first, escalate to a larger one when validation fails. Measure whether it pays.

4. Validation in code

• Line items add up to the subtotal; subtotal plus tax equals the total.
• Supplier exists in the master data; the bank account matches the one on file (a change is a classic fraud signal, always reviewed).
• Dates are plausible; currency matches the supplier; invoice number not already posted.
• Purchase order match, where there is one.

A schema guarantees the shape, not the truth. These rules catch wrong values.

5. Human review

• A queue sorted by amount and age, showing the PDF beside the extracted fields and the failed rule.
• Every correction is stored with the original output. Corrections become the evaluation set and show which layouts or fields fail most.

6. Measure and operate

• Accuracy per field and straight-through rate (share posted with no human touch) per supplier and language.
• Cost per invoice, from tokens per page. Track it as layouts change.
• Rerun the labelled set on every prompt, schema or model change before release.

What interviewers want: clarifying accuracy and latency, an immutable original and deduplication, schema-enforced extraction with its exceptions handled, batch processing for cost, business-rule validation in code, a review queue that feeds evaluation, and per-field metrics.