Website  •  Docs  •  Support  •  Discord

QVAC is an open-source, cross-platform ecosystem for building local-first, peer-to-peer AI applications and systems. With QVAC, you can run AI tasks like LLMs, speech, RAG, and more locally across Linux, macOS, Windows, Android, and iOS — or delegate inference to peers using its built-in P2P capabilities.

Key features

• Local-first: load AI models and perform inference on your own machine. No third-party APIs, SaaS, or cloud involved.
• P2P: build unstoppable internet systems — like BitTorrent, IPFS, and blockchain networks, but for AI.
• Cross-platform: consistent developer experience across hardware, operating systems, and JS runtime environments — write code once, run it everywhere.
• OpenAI-compatible API: integrate with the broader AI ecosystem.
• Open source: 100% free to use and modify — build on top, contribute back, be part of our community.

Usage

QVAC is composed of JavaScript libraries and tools that converge in the JS SDK. The SDK is the main entry point for using QVAC. It is type-safe and exposes all QVAC capabilities through a unified interface. It runs on Node.js, Bare runtime, and Expo.

Additionally, QVAC provides a CLI with tools and an HTTP server that exposes an OpenAI-compatible API. By implementing the OpenAI API format, QVAC can integrate with the broader AI ecosystem.

Install the @qvac/sdk npm package in your project. Then load models and run AI inference locally, or delegate inference to peers using the built-in P2P features.

Quickstart

1. Create the examples workspace:

mkdir qvac-examples
cd qvac-examples
npm init -y && npm pkg set type=module

2. Install the SDK:

3. Create the quickstart script:

import { loadModel, LLAMA_3_2_1B_INST_Q4_0, completion, unloadModel, } from "@qvac/sdk";
try {
    // Load a model into memory
    const modelId = await loadModel({
        modelSrc: LLAMA_3_2_1B_INST_Q4_0,
        modelType: "llm",
        onProgress: (progress) => {
            console.log(progress);
        },
    });
    // You can use the loaded model multiple times
    const history = [
        {
            role: "user",
            content: "Explain quantum computing in one sentence",
        },
    ];
    const result = completion({ modelId, history, stream: true });
    for await (const token of result.tokenStream) {
        process.stdout.write(token);
    }
    // Unload model to free up system resources
    await unloadModel({ modelId });
}
catch (error) {
    console.error("❌ Error:", error);
    process.exit(1);
}

4. Run the quickstart script:

Functionalities

AI capabilities

• Completion: LLM inference for text generation and chat via qvac-fabric-llm.cpp.
• Text embeddings: vector embedding generation for semantic search, clustering, and retrieval, via qvac-fabric-llm.cpp.
• Translation: text-to-text neural machine translation (NMT), via qvac-fabric-llm.cpp and Bergamot.
• Transcription: automatic speech recognition (ASR) for speech-to-text via qvac-ext-lib-whisper.cpp or NVIDIA Parakeet.
• Text-to-Speech: speech synthesis for text-to-speech (TTS) via ONNX Runtime.
• OCR: optical character recognition (OCR) for extracting text from images via ONNX runtime.
• Image generation: text-to-image generation via qvac-ext-stable-diffusion.cpp.
• Fine-tuning: adapting LLMs to domain-specific tasks via LoRA.
• Multimodal: LLM inference over text, images, and other media within a single conversation context.
• RAG: out-of-the-box retrieval-augmented generation workflow.

P2P capabilities

• Delegated inference: delegate inference to peers via the Holepunch stack, enabling resource sharing.
• Fetch models: download AI models from peers via the distributed model registry.
• Blind relays: connect peers across NATs/firewalls by routing traffic through relay nodes.

Utilities

• Plugin system: build lean apps by including only required AI capabilities, and extend the SDK by plugging in custom capabilities.
• Logging: visibility into what's happening during loading, inference, and other operations.
• Download Lifecycle: pause and resume model downloads.
• Sharded models: download a model that is sharded into multiple parts.

Complete user docs

Contributing

Repository layout

Monorepo structure overview. All QVAC components live under /packages, including the SDK, libraries, and tooling. Not every component is published to npm.

Legend:

• Core: foundational building blocks shared across the ecosystem.
• Addon: capability packages — each QVAC capability is implemented by one or more addons.
• SDK: primary entry point for consumers.
• Tool: user-facing tools and services that support the ecosystem.

Development

• For the standard development workflow used in this monorepo, see /docs/gitflow.md.
• For development specifics of each QVAC component, refer to the documentation in the respective subdirectory under /packages.
• For the QVAC architecture as a whole, see /docs/architecture.

Banners and badges

Built something with QVAC? Add a badge or banner to your README, website, or app. It is a simple way to highlight your project, help others discover QVAC, and strengthen our community.

By using these badges and banners, you help foster the QVAC ecosystem!

Choose a banner or badge below and copy its Markdown snippet, or copy its image URL and use the hosted SVG asset directly.

Banners

Large format badges (240x60) for prominent placement in your README header.

Dark with monochrome glow

Dark with colorful flow

Dark with stars pattern

Light with colorful flow

Banner usage

[![Built with QVAC](https://raw.githubusercontent.com/tetherto/qvac/refs/heads/main/docs/branding/qvac-banner-dark-glow.svg)](https://github.com/tetherto/qvac)

Badges

Compact badges for use alongside other shields/badges in your README.

Compact

Variant	Dark bg	Light bg
Green logo
Monochrome

Inline

Variant	Dark bg	Light bg
Green logo
Monochrome

Badge usage

[![Built with QVAC](https://raw.githubusercontent.com/tetherto/qvac/refs/heads/main/docs/branding/qvac-badge-green-dark.svg)](https://github.com/tetherto/qvac)