A deterministic corporate simulator for generating ground-truth ecosystems and evaluating enterprise AI agents

OrgForge simulates weeks of realistic enterprise activity — Confluence pages, JIRA tickets, Slack threads, Git PRs, Zoom transcripts, Zendesk tickets, Salesforce records, emails, and server telemetry — grounded in an event-driven state machine so LLMs can't hallucinate facts out of sequence.

The dataset is the exhaust of a living simulation. Engineers leave mid-sprint, forcing deterministic incident handoffs, ticket reassignments, and CRM ownership lapses. Knowledge gaps surface when under-documented systems break. New hires build their internal network through simulated collaboration. Stress propagates through a live, weighted social graph. Every artifact reflects the exact state of the org at the moment it was written.

Table of Contents

• Why Does This Exist?
• What the Output Looks Like
• What Gets Generated
• Architecture & Mechanics
• The Departure Cascade
• Insider Threat Simulation
• Quickstart
  • Setup Options
  • Option 1 — Everything in Docker
  • Option 2 — Local Ollama, Docker for MongoDB Only
  • Option 3 — Cloud Preset
  • Running on AWS EC2
• Configuration
  • Quality Presets
  • Key Config Fields
  • Dynamic Org Lifecycle
• How the Event Bus Works
• Memory Requirements
• Project Structure
• Roadmap
• Adding a New Artifact Type
• Contributing
• Citation
• License

Why Does This Exist?

When building AI agents that reason over institutional knowledge, you need a realistic corpus to test against. The only widely-used corporate dataset is the Enron email corpus — 25 years old, legally sensitive, and covering one company in crisis.

OrgForge generates that corpus from scratch, parameterized to any company, industry, or org structure. LLMs write the prose, but the facts — who was on-call, which ticket was open, when the incident resolved, who just left the team, and which customer SLA was breached — are strictly controlled by the state machine.

The central design bet: grounding LLM output in a deterministic event log makes the dataset actually useful for evaluating retrieval systems. You have ground truth about what happened, when, who was involved, and what the org's state was — so you can measure whether an agent surfaces the right context, not just plausible-sounding context.

What the Output Looks Like

Here's what a slice of a real simulation produces. An incident fires on Day 8:

slack/channels/engineering-incidents.json — the alert arrives first, timestamped to the millisecond the on-call pager fired:

{
  "ts": "2026-03-10T14:23:07",
  "user": "pagerduty-bot",
  "text": "🔴 P1 ALERT: TitanDB latency spike — connection pool exhaustion under load. On-call: Jax."
}

jira/IT-108.json — opened seconds later, facts pulled from the same SimEvent:

{
  "id": "IT-108",
  "type": "incident",
  "priority": "P1",
  "title": "TitanDB: latency spike",
  "root_cause": "connection pool exhaustion under load",
  "assignee": "Jax",
  "reporter": "system",
  "opened": "2026-03-10T14:23:19"
}

confluence/postmortems/IT-108.md — written the next day, linking the same root cause and PR:

This incident was triggered by connection pool exhaustion under sustained load, first surfaced in IT-108. The fix landed in PR #47 (merged by Sarah). A prior knowledge gap in TitanDB connection management — stemming from Jordan's departure on Day 12 — contributed to the delayed diagnosis.

Meanwhile, the datadog/metrics.jsonl time-series data reflects the exact latency spike, Zendesk support tickets from affected customers are automatically escalated to 'Urgent', Salesforce opportunities are flagged as 'at-risk', and end-of-month customer invoices (invoices/) automatically apply SLA credits based on the incident's duration.

None of this is coincidence — it all traces back to one SimEvent that every downstream artifact reads from.

What Gets Generated

A default 22-day simulation produces:

Architecture & Mechanics

OrgForge is not an LLM wrapper. Four interlocking systems enforce correctness.

👉 Read the full Architecture Deep-Dive here.

The Departure Cascade

The most complex behaviour in the simulation. When an engineer departs mid-sprint, the following fires in order before that day's planning runs:

1. Incident handoff — active incidents assigned to the departing engineer are rerouted via Dijkstra escalation routing (while the node is still in the graph) to the next available person in the chain.
2. Ticket & CRM reassignment — orphaned JIRA tickets go to the dept lead. Salesforce accounts and open opportunities owned by the departed employee are flagged for reassignment, maintaining cross-domain ground truth.
3. Graph recompute — betweenness centrality is recalculated on the smaller graph. Engineers absorbing the departed node's bridging load receive a proportional stress hit.
4. Knowledge gap propagation — if the departed engineer owned undocumented domains (configured via documented_pct), those gaps are registered in the SimEvent log and surface in subsequent incidents as contributing factors.
5. employee_departed SimEvent — emitted with edge snapshot, centrality at departure, reassigned tickets, and incident handoffs. Full ground truth for retrieval evaluation.

So when Jordan leaves on Day 12, the postmortem on Day 9's incident doesn't mention her. But the postmortem on Day 15 might: "A prior knowledge gap in auth-service, stemming from a recent departure, contributed to the delayed diagnosis." That sentence is grounded in a real SimEvent, not LLM inference.

Insider Threat Simulation

OrgForge includes an optional insider threat module that layers adversarial behavior on top of the normal simulation — without touching any of the clean simulation paths. When disabled (the default), it is completely inert: no overhead, no additional output, no altered code paths.

When enabled, designated employees exhibit configurable threat behaviors across multiple surfaces: anomalous git activity, off-hours access, sentiment drift in Slack, data staging on their workstation, and IDP authentication anomalies. All threat telemetry is written to a separate security_telemetry/ directory in industry-standard log formats (JSONL, CEF, ECS, LEEF), keeping it cleanly isolated from the normal simulation output so detection agents must work to find it.

The module is designed for building and evaluating insider threat detection systems. Ground truth is always preserved in JSONL regardless of output format.

👉 Read the full Insider Threat reference here.

Quickstart

flow.py is the main simulation entry point. config/config.yaml is the single source of truth for org structure, personas, and quality presets.

Setup Options

Option 1 — Everything in Docker (Recommended)

git clone https://github.com/aeriesec/orgforge
cd orgforge
docker compose up

First run pulls models automatically (~5–8 min depending on your connection). Subsequent runs start in seconds — models are cached in a named volume.

When the simulation finishes, run the post-processing artifact generators:

python email_gen.py
python post_sim_artifacts.py

Output lands in ./export/.

Option 2 — Local Ollama, Docker for MongoDB Only

Create a .env file:

OLLAMA_BASE_URL=http://host.docker.internal:11434

Then:

docker compose up mongodb orgforge

Linux note: host.docker.internal requires Docker Desktop, or the extra_hosts: host-gateway entry in docker-compose.yaml (already included).

Option 3 — Cloud Preset (AWS Bedrock + OpenAI)

Best output quality. Uses Claude Sonnet for document generation, Llama 3.1 8B on Bedrock for high-volume worker calls, and OpenAI text-embedding-3-large for embeddings.

Set quality_preset: "cloud" in config.yaml, then:

# .env
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_DEFAULT_REGION=us-east-1
OPENAI_API_KEY=...

pip install boto3 langchain-aws openai
docker compose up mongodb orgforge

Running on AWS EC2

Cheap EC2 + Bedrock/OpenAI (no GPU required)

A t3.small works fine — the cloud APIs do all the heavy lifting.

1. Launch an EC2 instance (Ubuntu or Amazon Linux) and install Docker
2. git clone https://github.com/aeriesec/orgforge.git && cd orgforge
3. cp .env.example .env and fill in your credentials
4. Set quality_preset: "cloud" in config/config.yaml
5. docker compose up --build -d mongodb orgforge

GPU Instance + 70B Local Models

For Llama 3.3 70B entirely locally, use a g5.2xlarge or g5.12xlarge with the Deep Learning AMI. Uncomment the GPU deploy block under the ollama service in docker-compose.yaml, set quality_preset: "local_gpu", then docker compose up -d.

Configuration

config/config.yaml is the single source of truth. No Python changes are needed for most customizations.

Quality Presets

quality_preset: "local_gpu" # local_gpu | cloud

Key Config Fields

Dynamic Org Lifecycle

Engineers can join and leave during the simulation. Departures and hires are scheduled in config and execute before the day's planning runs, so every downstream artifact that day reflects the new roster.

org_lifecycle:
  scheduled_departures:
    - name: "Jordan"
      day: 12
      reason: "voluntary" # voluntary | layoff | performance
      role: "Senior Backend Engineer"
      knowledge_domains:
        - "auth-service"
        - "redis-cache"
      documented_pct: 0.25 # fraction written down — drives gap severity

  scheduled_hires:
    - name: "Taylor"
      day: 15
      dept: "Engineering"
      role: "Backend Engineer"
      expertise: ["Python", "FastAPI"]
      style: "methodical, asks lots of questions before writing code"
      tenure: "new"

  enable_random_attrition: false
  random_attrition_daily_prob: 0.005

On hire, the new engineer enters the graph with cold-start edges below the warmup_threshold, so the day planner naturally proposes warmup_1on1 and onboarding_session events until real collaboration warms the edges.

How the Event Bus Works

Every significant action emits a SimEvent:

SimEvent(
    type="incident_opened",
    day=8,
    date="2026-03-10",
    actors=["Jax", "Sarah"],
    artifact_ids={"jira": "IT-108"},
    facts={
        "title": "TitanDB: latency spike",
        "root_cause": "connection pool exhaustion under load",
        "involves_gap": True
    },
    summary="P1 incident IT-108: connection pool exhaustion",
    tags=["incident", "P1"]
)

Every downstream artifact pulls its facts from the event log rather than asking an LLM to invent them. This prevents temporal drift and hallucination across a multi-week simulation.

The end-of-day day_summary SimEvent captures a structured snapshot of everything that happened:

facts={
    "active_actors":        ["Jax", "Sarah", "Morgan"],
    "dominant_event":       "incident_opened",
    "event_type_counts":    {"incident_opened": 1, "pr_review": 2, "standup": 1},
    "departments_involved": ["Engineering"],
    "open_incidents":       ["IT-108"],
    "stress_snapshot":      {"Jax": 72, "Sarah": 55, "Morgan": 41},
    "health_trend":         "degraded",
    "morale_trend":         "moderate",
}

This is what makes the dataset useful for RAG evaluation: you have ground truth about what happened, when, who was involved, and what the org's state was — so you can measure whether a retrieval system actually surfaces the right context.

Memory Requirements

For local_gpu on AWS, a g5.2xlarge (A10G 24GB) runs 70B at q4 quantization. At ~$0.50/hour spot pricing a full 22-day simulation costs roughly $3–5.

Project Structure

orgforge/
├── .github/workflows/    # CI/CD pipelines
├── src/
│   ├── flow.py           # State machine and simulation engine
│   ├── day_planner.py    # LLM-driven per-department daily planning
│   ├── normal_day.py     # Agenda dispatcher — produces typed artifacts per activity
│   ├── crm_system.py     # Salesforce & Zendesk integration and propagation rules
│   ├── planner_models.py # Dataclasses for plans, events, and validation results
│   ├── plan_validator.py # Integrity boundary between LLM proposals and execution
│   ├── org_lifecycle.py  # Dynamic hiring, firing, and knowledge gap propagation
│   ├── graph_dynamics.py # Social graph: stress propagation, edge decay, escalation
│   ├── memory.py         # Vector DB and SimEvent bus
│   ├── email_gen.py      # Reflective post-processing email artifacts
│   └── post_sim_artifacts.py # Deterministic post-processing (NPS, invoices, Datadog)
├── config/               # YAML configurations
├── tests/                # Pytest suite
├── scripts/              # Entrypoint and helper scripts
├── export/               # Output directory for generated dataset
├── README.md
├── ARCHITECTURE.md
└── CONTRIBUTING.md

Roadmap

• Native integrations for Zoom, Zendesk, and Salesforce CRM
• Plugin architecture for additional integrations (PagerDuty, Workday, etc.)
• Domain packs — pre-configured config.yaml templates for healthcare, fintech, legal
• Export to HuggingFace dataset format
• Evaluation harness — benchmark RAG retrieval against SimEvent ground truth

Adding a New Artifact Type

1. Add an event emission in flow.py when the triggering condition occurs
2. Write a handler that reads from the SimEvent log and generates the artifact

A formal plugin architecture is on the roadmap. Open an issue before starting so we can align on the interface.

Contributing

Contributions are welcome. Please read CONTRIBUTING.md before opening a PR. For new domain configs or artifact types, open an Issue first.

Citation

If you use this work, please cite:

@article{Flynt2026,
  author = {Jeffrey Flynt},
  title  = {OrgForge: A Multi-Agent Simulation Framework for Verifiable Synthetic Corporate Corpora},
  year   = {2026},
  url    = {https://arxiv.org/abs/2603.14997}
}

License

MIT — see LICENSE.