The problem no one talks about
Large language models are great at pattern matching.
Show them enough “vulnerable” examples, and they learn the words – not the reason.
That’s why they struggle with logical vulnerabilities like IDOR, privilege escalation, and workflow bypasses.
These attacks depend on who the actor is, who owns the resource, and where the trust boundary sits – not just on the presence of keywords like “authorization”.
Enter AMAS
AMAS (AI Multi‑Agent Security Analysis System) is a security reasoning substrate.
It doesn’t teach the model to memorise patterns.
It teaches it to understand causality.
Instead of “this is an IDOR”, AMAS models:
- Identity & Session – who is the actor? who is the owner?
- Enforcement failures – which security control was missing or broken?
- Temporal transitions – what changed before and after the request?
- Causal graphs – how does a chain of events lead to a breach?
How it works (in a nutshell)
- Ingest real‑world CVEs – streaming NVD feeds, filter by relevance (access control, auth, business logic).
- Convert each CVE into a causal graph – extract actors, resources, trust boundaries, and missing controls.
- Mix with high‑quality synthetic data – generated by a semantic mutation engine that changes mechanics (infrastructure, auth model, async behaviour), not just wording.
- Quality control – deduplication, domain coherence checks, repetition validation.
-
Export –
train.jsonlready for fine‑tuning, plus detailed manifests.
Everything is deterministic (same seed → same dataset) and scalable (handles 200k+ CVEs).
Results (so far)
After fine‑tuning a Mistral‑7B model on AMAS‑generated data:
- ✅ Generalisation – same vulnerability, different wording → correctly identified.
- ✅ Negative tests – secure login requests → no false alarms.
- ✅ Domain shift – secure scenarios in fintech, e‑commerce, healthcare → all classified correctly.
Duplication dropped from 52% to <10%, and domain corruption was eliminated in the latest pipeline.
Next steps
- Open‑source release (GitHub) – complete pipeline, docs, examples.
- Research paper (causal reasoning + identity‑aware training).
- Pre‑trained model weights.
Why share this now?
Because the industry keeps building pattern‑matching scanners.
We need reasoning engines.
If you’re into AI security, causal ML, or synthetic data – let’s connect.
I’ll drop the repo link here as soon as it’s public.
Comments and questions are very welcome. Let’s make AI actually understand security.