GPU-accelerated OCR server. 50x faster than PaddleOCR Python.
C++ / CUDA / TensorRT / PP-OCRv5 — Linux + NVIDIA GPU

Quick Start · API · Benchmarks · Monitoring · Configuration · Build · Roadmap

Highlights

• 🚀 270 img/s on FUNSD A4 forms (c=16) — 1,200+ img/s on sparse images
• ⚡ 11 ms p50 latency, single request
• 🎯 F1 = 90.2% on FUNSD — higher accuracy than PaddleOCR Python with the same weights
• 🖨️ Prints & handwriting — PP-OCRv5 handles both out of the box
• 📄 PDF native — pages rendered and OCR'd in parallel
• 🔒 4 PDF modes — pure OCR, native text layer, auto-dispatch, detection-verified hybrid
• 🧩 Layout detection — PP-DocLayoutV3 with 25 region classes, per-request ?layout=1 toggle
• 🌐 HTTP + gRPC from a single binary, sharing the same GPU pipeline pool
• 🐳 One-line Docker deploy — docker run with auto TRT engine build on first start
• 📊 Prometheus metrics — request counters, latency histograms, VRAM usage on /metrics

RTX 5090, PP-OCRv5 mobile latin, TensorRT FP16, pool=5. Prints, handwriting, layout detection. This is the fast lane.

🗺️ Roadmap

• 🌍 Configurable languages
• 🔍 Structured extraction
• 📝 Markdown output
• 📊 Table parsing

Quick Start

Requirements: Linux, NVIDIA driver 595+, Turing or newer GPU (RTX 20-series / GTX 16-series+).

docker run --gpus all -p 8000:8000 -p 50051:50051 \
  -v trt-cache:/home/ocr/.cache/turbo-ocr \
  ghcr.io/aiptimizer/turboocr:v2.0.0

First startup builds TensorRT engines from ONNX (~90s). The volume caches them for instant restarts. nginx (port 8000) reverse-proxies to Drogon (port 8080) for connection buffering — both start automatically.

curl -X POST http://localhost:8000/ocr/raw \
  --data-binary @document.png -H "Content-Type: image/png"

{
  "results": [
    {"text": "Invoice Total", "confidence": 0.97, "bounding_box": [[42,10],[210,10],[210,38],[42,38]]}
  ]
}

API

HTTP on port 8000, gRPC on port 50051 — single binary, shared GPU pipeline pool.

Important: Use persistent connections (HTTP keep-alive). Sending many short-lived connections (e.g. one curl per request in a loop) can overwhelm the server and cause it to stall. All standard HTTP client libraries (requests.Session, aiohttp, Go http.Client, etc.) reuse connections by default.

Endpoints

Query Parameters

Examples

# Image — raw bytes (fastest)
curl -X POST http://localhost:8000/ocr/raw \
  --data-binary @doc.png -H "Content-Type: image/png"

# Image — base64 JSON
curl -X POST http://localhost:8000/ocr \
  -H "Content-Type: application/json" \
  -d '{"image":"'$(base64 -w0 doc.png)'"}'

# PDF — raw bytes
curl -X POST http://localhost:8000/ocr/pdf \
  --data-binary @document.pdf

# PDF — multipart (works from any client, including browsers)
curl -X POST http://localhost:8000/ocr/pdf \
  -F "file=@document.pdf"

# PDF — with layout + auto mode
curl -X POST "http://localhost:8000/ocr/pdf?layout=1&mode=auto" \
  --data-binary @document.pdf

# gRPC (grpcurl uses base64 for CLI; real clients send raw bytes)
grpcurl -plaintext -d '{"image":"'$(base64 -w0 doc.png)'"}' \
  localhost:50051 ocr.OCRService/Recognize

Response Format

Image endpoints return:

{"results": [{"text": "Invoice Total", "confidence": 0.97, "bounding_box": [[42,10],[210,10],[210,38],[42,38]]}]}

With ?layout=1, a layout array is added. Each OCR result gets a layout_id linking it to the containing layout region:

{
  "results": [{"text": "...", "confidence": 0.97, "id": 0, "layout_id": 2, "bounding_box": [...]}],
  "layout": [{"id": 0, "class": "header", "confidence": 0.91, "bounding_box": [...]},
             {"id": 2, "class": "table", "confidence": 0.95, "bounding_box": [...]}]
}

PDF endpoint wraps results per page:

{
  "pages": [{
    "page": 1, "page_index": 0, "dpi": 100, "width": 1047, "height": 1389,
    "mode": "ocr", "results": [...]
  }]
}

Coordinate conversion: x_pdf = x_px * 72 / dpi.

PDF Extraction Modes

Layout Detection

All endpoints accept ?layout=1 to detect document regions using PP-DocLayoutV3 (25 classes):

abstract · algorithm · aside_text · chart · content · display_formula · doc_title · figure_title · footer · footer_image · footnote · formula_number · header · header_image · image · inline_formula · number · paragraph_title · reference · reference_content · seal · table · text · vertical_text · vision_footnote

_{Layout detection overlay — color-coded regions: paragraph_title, text, chart, figure_title, header, footer, number}

Benchmarks

FUNSD form-understanding dataset (50 pages, ~170 words/page). Same word-level F1 metric for all engines. Single RTX 5090.

Configuration

Layout detection is enabled by default. The model is loaded at startup but only runs when a request includes ?layout=1. Requests without ?layout=1 have zero overhead. Requests with ?layout=1 reduce throughput by ~20%. Set DISABLE_LAYOUT=1 to skip loading the model entirely and save ~300-500 MB VRAM.

docker run --gpus all -p 8000:8000 \
  -v trt-cache:/home/ocr/.cache/turbo-ocr \
  -e PIPELINE_POOL_SIZE=3 \
  turboocr

Add MAX_PDF_PAGES (default 2000) to limit the number of pages processed per PDF request. LOG_LEVEL (debug/info/warn/error) and LOG_FORMAT (json/text) control structured logging output.

Monitoring

Prometheus Metrics

Scrape GET /metrics for Prometheus-compatible metrics:

turbo_ocr_requests_total{route="/ocr/raw",status="2xx"} 1042
turbo_ocr_request_duration_seconds_bucket{route="/ocr/raw",le="0.025"} 980
turbo_ocr_request_duration_seconds_sum{route="/ocr/raw"} 12.345
turbo_ocr_request_duration_seconds_count{route="/ocr/raw"} 1042
turbo_ocr_gpu_vram_used_bytes 9052815360
turbo_ocr_gpu_vram_total_bytes 33661911040
turbo_ocr_pipeline_pool_size 5
turbo_ocr_pool_exhaustions_total 0
turbo_ocr_request_bytes_total 49493243
turbo_ocr_request_body_avg_bytes 9407

Response Headers

Every response includes:

Health Endpoints

Structured Errors

All error responses return JSON with Content-Type: application/json:

{"error": {"code": "EMPTY_BODY", "message": "Empty body"}}

Error codes: EMPTY_BODY, INVALID_JSON, MISSING_IMAGE, BASE64_DECODE_FAILED, IMAGE_DECODE_FAILED, INVALID_PARAMETER, UNSUPPORTED_PARAMETER, INVALID_DPI, INVALID_DIMENSIONS, DIMENSIONS_TOO_LARGE, BODY_SIZE_MISMATCH, MISSING_HEADER, INVALID_HEADER, EMPTY_BATCH, MISSING_FILE, MISSING_PDF, INVALID_MULTIPART, PDF_RENDER_FAILED, PDF_TOO_LARGE, EMPTY_PDF, SERVER_BUSY, NOT_READY, INFERENCE_ERROR.

Building from Source

Wuffs, Clipper, PDFium vendored in third_party/.

# Docker (recommended)
docker build -f docker/Dockerfile.gpu -t turboocr .
docker run --gpus all -p 8000:8000 -p 50051:50051 \
  -v trt-cache:/home/ocr/.cache/turbo-ocr turboocr

# CPU only (Docker) — ~2-3 img/s, mainly for testing
docker build -f docker/Dockerfile.cpu -t turboocr-cpu .
docker run -p 8000:8000 turboocr-cpu

# Native build
cmake -B build -DTENSORRT_DIR=/usr/local/tensorrt && cmake --build build -j$(nproc)

Supported Languages

Latin script (English, German, French, Italian, Polish, Czech, and more) plus Greek. 836 characters total.

Acknowledgements

This project builds on the work of several open-source projects:

• PaddleOCR (Baidu) — PP-OCRv5 detection, recognition, and classification models. PP-DocLayoutV3 layout detection model. This project would not exist without their research and pre-trained weights.
• Drogon — high-performance async C++ HTTP framework
• Wuffs — fast PNG decoder by Google (vendored)
• PDFium — PDF rendering and text extraction (vendored)
• Clipper — polygon clipping for text detection post-processing (vendored)

License

MIT. See LICENSE.

_{Main Sponsor: Miruiq — AI-powered data extraction from PDFs and documents.}