One API for every AI — cloud, on-device, and Apple Intelligence.

Arbiter is a unified AI runtime for Swift that lets you call any AI provider through a single, consistent interface. Write your AI code once, then swap providers — or run them all simultaneously with intelligent routing.

Quick Start

import Arbiter

let ai = try Arbiter {
    try $0.cloud(.anthropic(from: .keychain))
}

// Simple generation
let response = try await ai.generate("Explain quantum computing")

// Or drop in a full chat UI
ArbiterChatView(ai: ai)

Multi-Provider Setup

let ai = try Arbiter {
    try $0.cloud(.anthropic(from: .keychain))
    try $0.cloud(.openAI(from: .keychain))
    try $0.cloud(.gemini(from: .keychain))
    $0.local(OllamaProvider())
    $0.local(MLXProvider(.auto))
    $0.system(AppleFoundationProvider())
    $0.routing(.smart)
    $0.spendingLimit(5.00)
    $0.privacy(.strict)
}

// Arbiter picks the best available provider
let response = try await ai.generate("Hello!")

// Tag sensitive requests — forces on-device routing
let options = RequestOptions(tags: [.health])
let privateResponse = try await ai.generate("Summarize my lab results", options: options)

Streaming

let stream = ai.stream("Write a haiku about Swift.")
for try await chunk in stream {
    // chunk.delta contains the incremental text
}

Conversations

@State private var session = ConversationSession(systemPrompt: "You are a helpful assistant.")

// In your SwiftUI view:
try await session.send("What is SwiftUI?", using: ai)
// session.messages is @Observable — your UI updates automatically

Structured Output

Generate typed Swift values directly — no manual JSON parsing:

struct Recipe: Codable {
    let name: String
    let ingredients: [String]
}

// Simple — works for most types
let recipe: Recipe = try await ai.generate("Pasta recipe", as: Recipe.self)

// With example — most reliable for complex types
let recipe: Recipe = try await ai.generate(
    "Pasta recipe",
    as: Recipe.self,
    example: Recipe(name: "", ingredients: [])
)

Works with conversations too:

let analysis: SentimentResult = try await session.send(
    "Analyse this review: 'Great product!'",
    as: SentimentResult.self,
    example: SentimentResult(sentiment: "", score: 0),
    using: ai
)

Intelligent Routing

Arbiter's router doesn't just match capabilities — it analyses the actual request to determine complexity, intent, and optimal routing. No other library does this.

// Arbiter analyses your request and routes intelligently:

// Simple classification → Apple FM (free, fast, sufficient)
let sentiment = try await ai.generate("Is this positive? 'Great product!'")

// Complex reasoning → Claude (best quality for hard tasks)
let analysis = try await ai.generate("Compare microservices vs monolith...")

// Code generation → Cloud provider with best code capability
let code = try await ai.generate("Write a binary search in Swift")

// All automatic. No manual routing. The router learns and improves.

How It Works

The RequestAnalyser examines every prompt before routing:

1. Complexity classification — trivial, simple, moderate, complex, or expert
2. Task detection — classification, code generation, reasoning, translation, etc.
3. Output estimation — predicts response size based on task type
4. Cost estimation — calculates expected cost per provider

This analysis feeds into the Smart Router's scoring engine:

Each factor produces a score, and the routing strategy applies different weights:

.smart               // Balanced across all factors (default)
.costOptimized       // Heavily weights cost — prefers free/cheap providers
.privacyFirst        // Heavily weights privacy — prefers on-device
.qualityFirst        // Heavily weights quality — prefers the most capable
.latencyOptimized    // Heavily weights speed — prefers the fastest
.fixed(.anthropic)   // Always use a specific provider
.priority([.ollama, .anthropic])  // Try in order, fail over to next

Adaptive Routing

The router gets smarter the more you use it. The ProviderPerformanceTracker records real-world metrics for every request:

• Success rate per provider per task type
• Latency compared to the global average
• Token throughput for cost efficiency

After 10+ requests, the tracker starts adjusting routing scores:

• High success rate (>95%) → +10 score bonus
• Low success rate (<70%) → -20 score penalty
• Faster than average → +5 latency bonus
• Much slower (>2x average) → -10 penalty

Performance data persists across app launches via UserDefaults.

Cost Estimation

Know what a request will cost before sending it:

let estimates = await ai.estimateCost("Write a detailed essay about AI")
for estimate in estimates {
    print("\(estimate.provider): $\(estimate.estimatedCost)")
}
// Anthropic: $0.0031
// OpenAI: $0.0024
// Gemini: $0.0012
// MLX: $0.0000

Three-Tier Architecture

┌─────────────────────────────────────────────────────────┐
│                  Intelligent Router                      │
│  ┌─────────┐ ┌──────────┐ ┌──────────┐ ┌────────────┐  │
│  │ Request  │ │Capability│ │ Provider │ │Environment │  │
│  │ Analyser │ │ Matcher  │ │ Tracker  │ │  Checks    │  │
│  └─────────┘ └──────────┘ └──────────┘ └────────────┘  │
└──────┬──────────────┬──────────────┬───────────────┬─────┘
       │              │              │               │
┌──────▼──────┐┌──────▼──────┐┌──────▼──────┐┌──────▼──────┐
│   Tier 1    ││   Tier 2    ││   Tier 3    ││   Tier 4    │
│  Apple FM   ││    MLX      ││   Ollama    ││   Cloud     │
│ Free, Fast  ││Free, Medium ││Free, Local  ││ Paid, Best  │
│  Limited    ││   Good      ││   Good      ││   Quality   │
└─────────────┘└─────────────┘└─────────────┘└─────────────┘

Environment-Aware Adjustments

After scoring, the router adjusts for real-time conditions:

• Offline? Cloud providers are automatically removed
• Thermal pressure? Local model scores are halved (prefers cloud to avoid overheating)
• Budget exhausted? Cloud provider scores drop to zero

Privacy Routing

Tag requests with privacy classifications to enforce routing rules:

// These tags force on-device routing automatically
let options = RequestOptions(tags: [.health])
let response = try await ai.generate("Analyze my blood pressure trends", options: options)

// Built-in tags: .private, .health, .financial, .personal
// Or define your own: RequestTag("legal")

The PrivacyGuard can also detect PII automatically:

let ai = Arbiter {
    $0.privacy(.strict)  // Enables PII detection (email, phone, SSN, credit card)
}
// Requests containing PII are automatically routed on-device

Fallback Chain

When the top-scored provider fails, the router automatically tries alternatives:

let ai = Arbiter {
    $0.cloud(anthropicProvider)
    $0.cloud(openAIProvider)
    $0.local(OllamaProvider())
    $0.local(MLXProvider(.auto))
    $0.system(AppleFoundationProvider())
    $0.routing(.smart)  // fallbackEnabled is true by default
}
// If Anthropic is down → tries OpenAI → Ollama → MLX → Apple FM

Cost Tracking

Arbiter tracks spend per provider and enforces budgets:

let ai = Arbiter {
    $0.spendingLimit(5.00, action: .fallbackToCheaper)
}
// When budget runs low, automatically switches to cheaper/free providers

Per-Request Timeout

Override the default 30-second timeout for individual requests:

let options = RequestOptions(timeout: .seconds(60))
let response = try await ai.generate("Write a long essay", options: options)

Retry Configuration

Configure automatic retries for single-provider setups:

let ai = Arbiter {
    $0.cloud(anthropicProvider)
    $0.retry(maxAttempts: 3, baseDelay: .milliseconds(500), maxDelay: .seconds(30))
}

Provider Health Monitoring

Enable periodic availability checks to avoid routing to unhealthy providers:

let ai = Arbiter {
    $0.cloud(anthropicProvider)
    $0.local(OllamaProvider())
    $0.healthCheck(.enabled(interval: .minutes(5)))
}

Supported Providers

On-Device Providers

MLX (Apple Silicon)

Runs open-source models locally via mlx-swift. Zero network, zero cost, complete privacy.

// Auto-select best model for this device
$0.local(MLXProvider(.auto))

// Or pick a specific model
$0.local(MLXProvider(.model("mlx-community/Qwen2.5-7B-Instruct-4bit")))

The MLX model registry automatically recommends models based on device RAM:

Apple Foundation Models

Uses Apple's built-in on-device model via the FoundationModels framework. Requires iOS 26+ / macOS 26+ with Apple Intelligence enabled.

$0.system(AppleFoundationProvider())

Check availability in SwiftUI:

Text("AI Feature")
    .appleFoundationAvailable {
        Text("Requires Apple Intelligence")
    }

SwiftUI Components

Drop-in UI components that work with any Arbiter configuration.

Chat Interface

import Arbiter

struct ContentView: View {
    let ai: Arbiter

    var body: some View {
        ArbiterChatView(ai: ai)
        // Or with a system prompt:
        // ArbiterChatView(ai: ai, systemPrompt: "You are a helpful assistant.")
    }
}

Features: message bubbles, streaming animation, provider badge on each response, error handling with retry button, dark mode support.

Provider Picker

ProviderPicker(ai: ai) { selectedProvider in
    // Override routing for this session
}

Lists configured providers with real-time availability status and tier badges (Cloud/Local/On-Device/System).

Usage Dashboard

UsageDashboard(analytics: analytics)

Shows total requests, tokens used, estimated cost, per-provider breakdown with bar charts, and month-over-month comparison.

Routing Debug View

RoutingDebugView(router: ai.smartRouter)

Live feed of routing decisions — shows timestamp, selected provider, reason, fallbacks, contributing factors, detected complexity, detected task type, and estimated costs per provider.

Lifecycle Management

ContentView()
    .swiftAILifecycle(ai)

Automatically unloads on-device models when the system reports memory pressure, freeing RAM for your app.

Middleware

Process requests and responses through a configurable pipeline:

let ai = Arbiter {
    $0.cloud(anthropicProvider)
    $0.middleware(LoggingMiddleware(logLevel: .standard))
    $0.middleware(RequestSanitiserMiddleware(requestsPerMinute: 30))
}

Request Sanitiser

Protects against prompt injection and abuse:

• Blocks prompts exceeding maximum length
• Detects known injection patterns ("ignore previous instructions", etc.)
• Rate limiting per minute
• Rejects empty or whitespace-only prompts

Note: MLXProvider conforms to UnloadableProvider — on memory warnings, LifecycleManager automatically unloads cached models to free RAM. Implement UnloadableProvider on your own providers for the same behaviour.

Logging Middleware

Structured logging with automatic credential redaction:

• API keys: sk-ant-api03-... → sk-ant-***REDACTED***
• Bearer tokens: Bearer eyJ... → Bearer ***REDACTED***
• Optional prompt text redaction for privacy-sensitive apps
• Configurable log levels: .none, .minimal, .standard, .verbose
• Output to os.Logger or custom destination

Response Cache

In-memory or disk-backed cache to reduce API costs:

let cache = ResponseCache(maxEntries: 500, ttl: .seconds(300))
let diskCache = ResponseCache(maxEntries: 1000, ttl: .seconds(600), persistence: .disk)

Usage Analytics

Cross-session usage tracking with SwiftUI binding:

let analytics = UsageAnalytics()
let snapshot = await analytics.snapshot() // UsageSnapshot is @Observable

Why Arbiter?

The Three-Tier Problem

Modern apps need AI from three different places:

1. Cloud APIs (Anthropic, OpenAI, Gemini) — most capable, but require network and cost money
2. Local servers (Ollama) — good for development and privacy, but need setup
3. On-device models (MLX, Apple Foundation Models) — instant, private, free, but less capable

Each has a different SDK, different data types, different error handling. Arbiter unifies all three behind a single protocol, so your app code stays clean regardless of which tier you're using.

Installation

Add Arbiter to your project via Swift Package Manager:

dependencies: [
    .package(url: "https://github.com/sabby3861/Arbiter.git", from: "0.1.0")
]

MLX support is included as an optional dependency — it compiles only on macOS and iOS with Apple Silicon. If mlx-swift is not resolved, the MLX provider gracefully reports as unavailable.

Requirements

• Swift 6.0+
• iOS 17+ / macOS 14+ / visionOS 1+
• Xcode 16+
• MLX provider: Apple Silicon (M1+) with 4GB+ RAM
• Apple Foundation Models: iOS 26+ / macOS 26+ with Apple Intelligence enabled

Security

Arbiter is designed with security defaults, not security afterthoughts.

API key protection: Keys are stored in the iOS Keychain by default. Hardcoded key strings trigger a deprecation warning at compile time.

Privacy routing: Tag requests as .private, .health, or .financial to ensure they never leave the device. The smart router enforces this.

Spending limits: Set monthly and per-request budget caps. When limits are reached, Arbiter falls back to free on-device providers automatically.

PII detection: Optional prompt scanning catches email addresses, phone numbers, and other patterns before they reach cloud APIs.

Redacted logging: API keys and sensitive headers are automatically redacted in all log output.

Request sanitization: Built-in middleware catches prompt injection attempts and enforces rate limits.

For production apps, we strongly recommend:

1. Use SecureKeyStorage (Keychain) instead of hardcoded API keys
2. Set up a server-side proxy for API calls (your key stays on your server)
3. Enable .privacy(.strict) for any app handling personal data
4. Set spending limits with SpendingGuard
5. Add RequestSanitiserMiddleware to catch injection attempts

See our Security Guide for detailed best practices.

Examples

Ready-to-run example projects in the Examples/ directory:

Each example has its own Package.swift — clone, add your API key, and run.

API Documentation

API docs are available via DocC:

swift package generate-documentation

Roadmap

• Core protocol layer
• Anthropic Claude provider
• OpenAI provider (including compatible APIs: Groq, Together, Perplexity)
• Google Gemini provider
• Ollama local provider
• Streaming (SSE + NDJSON)
• Tool calling / function calling
• Conversation session management
• Spending guards with budget enforcement
• Keychain-based secure key storage
• Smart Router with multi-factor scoring
• Privacy Guard with PII detection
• Cost tracking per provider
• Fallback chain with automatic retry
• Environment-aware routing (connectivity, thermal, budget)
• MLX on-device provider
• Apple Foundation Models provider
• SwiftUI components (ChatView, ProviderPicker, UsageDashboard, RoutingDebugView)
• Middleware pipeline (logging, sanitization, caching)
• Usage analytics with cross-session persistence
• Lifecycle management for on-device providers
• Security documentation and proxy architecture guide
• Structured output (typed Codable responses)
• Request intelligence engine (complexity, task detection, cost estimation)
• Adaptive routing (learns from usage patterns)
• Pre-request cost estimation API
• Per-request timeout configuration
• Configurable retry engine
• Disk-backed response cache
• Provider health monitoring
• Tool calling documentation
• Response quality validation
• Token budget planning
• v0.2 — MCP client support
• v0.2 — Certificate pinning for cloud providers
• v0.3 — Conversation persistence
• v0.3 — Function calling abstraction

Contributing

Contributions are welcome! Whether it's a bug fix, new feature, documentation improvement, or test coverage — every contribution helps.

Getting Started

1. Fork the repository
2. Create your feature branch (git checkout -b feat/your-feature)
3. Make your changes and add tests
4. Run the test suite (swift test)
5. Commit your changes and push to your fork
6. Open a Pull Request against main

Ways to Contribute

• Good First Issues — Check out issues labelled good first issue for beginner-friendly tasks
• Feature Requests — Have an idea? Open a feature request
• Bug Reports — Found a bug? Open a bug report
• Documentation — Improvements to guides, docstrings, or examples
• New Providers — Add support for additional AI providers

See CONTRIBUTING.md for detailed guidelines.

License

MIT — see LICENSE for details.

Built by Sanjay Kumar — Lead iOS Engineer, London | Blog