Every LLM app I audited had the same problem.
Users type real data into AI features. Names, emails, social security numbers, credit card numbers, medical details. The app takes that input, wraps it in a prompt, and sends it straight to OpenAI or Anthropic. No filtering. No redaction. Nothing.
The developer didn't plan for it. The product manager didn't think about it. The compliance team doesn't even know AI features exist yet.
I built AI Security Gateway to fix this. It's an open-source proxy that sits between your app and any LLM provider. Every prompt passes through a security layer before it reaches the model.
What It Does
The proxy inspects every request in real-time and applies four layers of governance:
1. PII Redaction
Before your prompt reaches OpenAI, Anthropic, Google, or anyone else, the proxy detects and redacts 28+ PII entity types:
- Personal identifiers — names, emails, phone numbers, dates of birth
- Financial data — credit card numbers, IBANs, bank accounts
- Government IDs — SSNs, passport numbers, driver's licenses
- Medical identifiers — medical record numbers, NPI numbers
- Locations — physical addresses, IP addresses
- Custom patterns — your own regex for internal codes, customer IDs, etc.
It also handles images. If a user uploads a screenshot to a vision model (GPT-4o, Claude, Gemini), our OCR pipeline extracts text from the image and scans it for PII before the image reaches the provider.
2. Prompt Injection Blocking
Heuristic detection catches jailbreak attempts, role override attacks, and instruction extraction — combined with custom regex rules for your specific application patterns.
3. Budget Enforcement
Set hard spend caps per API key. When a key hits its limit, the proxy returns HTTP 402. Not a warning — a hard stop.
This exists because I watched an agent loop burn through $3,000 in a single night during testing.
4. Smart Cost Routing
Configure multiple providers and the proxy automatically routes each request to the cheapest available model. We track live pricing across 600+ models and 8+ providers. Teams typically see 30-60% cost reduction from routing alone.
The Architecture Decision That Matters Most
AISG is fully stateless. This isn't a feature toggle — it's the architecture.
Prompts pass through memory and are discarded. Only metadata survives: cost, latency, token counts, PII entity counts, policy violations. The proxy physically cannot retain prompt content. There's no database to store it, no log to write it to, no queue to buffer it.
I made this decision early because the alternative — a proxy that logs everything "for observability" — creates exactly the problem it claims to solve. You're trying to prevent data leaking to third parties, so you route it through a proxy that... stores all the data? That never made sense to me.
This matters for compliance:
| Standard | What it means with AISG |
|---|---|
| HIPAA | Patient data in prompts never persists outside your app |
| PCI DSS | Credit card numbers redacted before any third-party API call |
| GDPR | No personal data stored by the proxy layer |
| SOC 2 | Audit logs capture what happened without capturing what was said |
The Tech Stack
For anyone interested in what's under the hood:
- Python + FastAPI — async proxy layer, handles streaming responses
- Presidio + custom NER — multi-layered PII detection pipeline
- Database — metadata only (costs, violations, never prompts)
- Docker Compose — single command self-hosting
- AWS — managed cloud version
Integration
If you're using the OpenAI SDK, it's two lines:
from openai import OpenAI
client = OpenAI(
base_url="https://api.aisecuritygateway.ai/v1",
api_key="your-aisg-key"
)
# Your existing code stays exactly the same
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Summarize this contract..."}]
)
No new SDK.
No wrapper library.
Your existing OpenAI calls now go through:
- PII redaction
- Injection blocking
- Budget enforcement
- Smart routing
All transparent to your application.
What I Learned Building This
1. PII Detection Is Harder Than You Think
"John Smith" is a name. "Smith & Wesson" is not. "Call me at 555-1234" contains a phone number. "Error code 555-1234" does not. Context matters enormously. Regex alone gets you maybe 60% accuracy. You need NER models layered on top.
2. Latency Budgets Are Brutal
Every millisecond of proxy overhead is overhead users feel.We got text inspection down to ~50ms. Image OCR still costs ~0.5–1 second. That's the trade-off — and for images containing PII, it's worth it.
3. Budget Enforcement Became the Killer Feature
I originally built this for PII redaction. But the feature people ask about most is budget caps. Turns out, "My agent loop burned $2,000 overnight" is a more common pain point than, "My prompts contain SSNs."
4. Self-Hosting Is a Trust Multiplier
Making the entire stack open-source under Apache 2.0 was the best decision I made. Enterprise security teams don't trust a proxy they can't inspect. Open source removes that objection immediately.
Try It
Managed Cloud
- Website: https://aisecuritygateway.ai
- Free credits: 1M credits
- Credit card required: No
Self-Host
docker compose up
Documentation
- https://aisecuritygateway.ai/docs
- The project is Apache 2.0 licensed. Stars, issues, and PRs are all welcome.
Final Thought
I'd love to hear from anyone dealing with PII in LLM prompts.
What's your current approach?
- Filtering at the application layer?
- Using a proxy?
- Ignoring it and hoping for the best?