How I Built a Blazingly Fast, Privacy-First Batch Image C...

The Problem with Modern Web Tools

Most online image converters follow a flawed pattern: you upload your files to a cloud server, their backend processes them, and you download them back. If you are handling hundreds of images, this layout introduces massive network bottlenecks. More importantly, it completely breaks user data privacy.

I wanted to build a batch image conveyor that processes hundreds of files instantly, supports next-gen formats (WebP, AVIF, QOI), and ensures that zero bytes of user data ever leave the local machine.

But doing this purely on the client side brings two massive boss-level challenges:

UI Freezing: Image compression is CPU-intensive. Running it on the main thread makes the browser completely unresponsive.
Out of Memory (OOM) Crashes: Keeping raw pixel arrays for 100+ high-res images in the browser's JavaScript heap will instantly crash the tab.

Here is how I solved this architecture using modern Web APIs

The Architecture: Continuous Streaming Pipeline

To bypass the browser's RAM and CPU limitations, I built a decoupled streaming architecture. Dev.to supports Mermaid.js rendering natively, so here is exactly how the data flows from your desktop back to your download folder:

graph TD
    Input[User Input: Drag & Drop / Files] -->|Stream Raw Bytes| OPFS[(OPFS Sandbox Disk)]
    OPFS -->|Sequential Read| Pool[WorkerPool Manager]

    subgraph Multi-Threaded Core (Backpressure Capped < 200MB)
        Pool -->|Payload 1| W1[Worker 1: WebP/Lanczos3]
        Pool -->|Payload 2| W2[Worker 2: Nearest/QOI]
        Pool -->|Payload N| WN[Worker N: AVIF WASM]
    end

    W1 -->|Compressed Blobs| Zip[fflate ZIP Archiver]
    W2 -->|Compressed Blobs| Zip
    WN -->|Compressed Blobs| Zip

    Zip -->|Continuous Binary Stream| Download[Instant Local Download]

    style Multi-Threaded Core fill:#121214,stroke:#39ff14,stroke-width:2px
    style OPFS fill:#1f2937,stroke:#58a6ff,stroke-width:1px
    style Download fill:#065f46,stroke:#10b981,stroke-width:2px

Eliminating RAM Bloat with OPFS (Origin Private File System)
Instead of loading dropped files directly into memory, my pipeline instantly intercepts the stream and writes the raw binary data into the Origin Private File System (OPFS) sandbox.
OPFS acts as a fast, isolated virtual disk inside the browser. This allows the application to accept a folder with 500+ items without consuming more than a few megabytes of actual RAM.
Multi-Threading with a Custom Web Worker Pool
To keep the frame rate at a buttery-smooth 60fps, all conversion and scaling tasks are delegated to a pool of background Web Workers. The number of active workers scales dynamically based on the user's CPU thread count (navigator.hardwareConcurrency).
Each worker loads target codecs and executes compression in total isolation from the UI.
Implementing Strict Backpressure
If you feed 100 workers at once, the browser will still crash due to rapid memory allocation. To counter this, I implemented a custom Backpressure mechanism inside the WorkerPool.
The pool tracks total active byte allocation. If the memory footprint of "in-flight" images approaches 200MB, the pipeline pauses reading from OPFS. As soon as a worker finishes compressing an image and releases its buffer, the pipeline pushes the next asset forward.
Streaming ZIP Archive Compilation
Once compressed, storing finished assets back to memory to create a ZIP file would defeat the whole purpose. Instead, the architecture streams individual compressed files into fflate on the fly, packaging them into a continuous Blob stream that triggers an instant local download.

Why not a full WebAssembly monolith?

A common question for this kind of heavy-lifting utility is: "Why didn't you just compile a native C++ or Rust image processing library directly into a single WebAssembly (WASM) binary?"

While WASM is incredibly fast, using it as a monolithic backend inside the browser has critical trade-offs for this specific architecture:

Native Browser Strengths: Modern browsers already have hyper-optimized, native, hardware-accelerated pipelines for rendering and encoding formats like WebP. Wrapping JS APIs inside a Worker Pool lets us use these native engines for free without the penalty of huge WASM binary overhead.
OPFS Threading Sync: Working closely with the Origin Private File System, generating local sandboxed URLs, and handling dynamic runtime cancellations is significantly easier and safer through asynchronous JavaScript/TypeScript Workers.
Hybrid Approach: Instead of a full-WASM monolith, I chose a hybrid ecosystem. JavaScript handles orchestration, pipeline state, and native codecs, while WebAssembly is injected strictly where JS is too slow — specifically inside background workers for heavy Lanczos3 image resampling (via Pica) and advanced AVIF encoding.

The Tech Stack Inside

Vite + TypeScript: Fast building and type-safe core pipelines.
Pica: Industrial-grade Lanczos3 resampling filter (WASM accelerated).
@jsquash/avif: Multi-threaded, industrial-grade AVIF encoding algorithms compiled to WASM.
fflate: High-speed, memory-efficient binary ZIP compression.
StreamSaver: Low-overhead client-side streaming downloads.

Performance Benchmarks

Testing the pipeline on an 8-core machine yielded impressive results, proving that browser storage can compete with native desktop tools:

100 high-res images (10MB each): ~45 seconds
500 mobile assets (2MB each): ~2 minutes
Max RAM Usage: Stays under 200MB at all times
Disk Usage: Streamed directly to OPFS, zero memory leak

Check it out!

The project is completely free, open-source, and has no trackers, cookies, or ads.

I would love to hear your thoughts on this architecture! How are you handling heavy asset manipulation on the client side? Let's discuss in the comments!

推荐订阅源

DEV Community