** The Problem**
Processing large documents through an AI model takes 30-60 seconds. A synchronous API makes the client wait the entire time. Browsers timeout. Users think it crashed. The experience feels broken.
Yesterday I learned why this happens. Today I built the fix.
## What I Built
A document processing pipeline with FastAPI and PostgreSQL that handles long-running AI tasks in the background. Three task types. Immediate response. Full audit trail.
Client submits a document and gets a job ID in milliseconds. Claude processes it in the background. Client polls for the result when ready. Everything logged to PostgreSQL.
## Three Task Types
Summarise: submit a document and receive a structured summary with executive overview and key points. Focus area is configurable.
Extract: specify which fields you want pulled out. Company name, net profit, revenue, key risks. Claude reads the document and returns exactly what you asked for as structured JSON.
Evaluate: provide a list of criteria and Claude checks whether the document meets each one. Returns a pass or fail with reasoning for every criterion.
*## How the Pipeline Works
*
POST /jobs/summarise
↓
Job ID returned in milliseconds
status: pending
↓
Claude processes in background
status: running
↓
GET /jobs/{job_id}
status: completed
full result ready
Optional webhook support means the server can call your endpoint when the job completes and no polling needed.
## The Database Behind It
Every job is stored in PostgreSQL with full audit trail:
Job ID, status at every stage, task type, input data as JSONB, result as JSONB, error message if failed, webhook URL, created timestamp, completed timestamp.
If a job fails the error is recorded for debugging. Nothing disappears silently.
## Real Use Cases
Contract review for law firms. Loan application screening for SACCOs. Insurance claims processing. Report summarisation for enterprises.
The same pipeline. Different documents. Different criteria. One system.
## What I Learned
FastAPI's BackgroundTasks is the simplest way to separate submission from processing. The endpoint returns immediately. The heavy work happens after the response is sent.
JSONB in PostgreSQL stores the full input and result as structured data — not a string you have to parse later. Query it, index it, use it.
Webhooks flip the polling model entirely. Instead of the client asking if the job is done, the server tells the client when it is ready.
🔗 Full project on GitHub → https://github.com/mbuguacessy-glitch
38 more to go.