惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

月光博客
月光博客
Microsoft Security Blog
Microsoft Security Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
博客园 - 叶小钗
博客园 - 三生石上(FineUI控件)
爱范儿
爱范儿
T
Tailwind CSS Blog
F
Fortinet All Blogs
Blog — PlanetScale
Blog — PlanetScale
WordPress大学
WordPress大学
H
Help Net Security
Apple Machine Learning Research
Apple Machine Learning Research
Y
Y Combinator Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
GbyAI
GbyAI
Martin Fowler
Martin Fowler
有赞技术团队
有赞技术团队
腾讯CDC
V
V2EX
Jina AI
Jina AI
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
B
Blog
M
MIT News - Artificial intelligence
宝玉的分享
宝玉的分享

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
400 ad accounts 3 subrequests = silent data loss in Cloud...
강해수 · 2026-06-22 · via DEV Community

강해수

Adding a single KV cache write broke my Worker — and it didn't throw a single error to the caller.

I run a Korean D2C ad-ops pipeline where one Worker fans out performance checks across ~400 ad accounts per request. The math looked fine: 400 accounts × 2 API calls = 800 subrequests, safely under Cloudflare's hard 1000-per-invocation ceiling. Then I added KV writes to cache each result. 400 × 3 = 1200. The Worker silently stopped processing the last ~200 accounts. No 5xx. No exception propagated. Just missing data. It took two days on wrangler tail to surface this line:

workers/runtime/fetch: subrequest count exceeded limit of 1000

The thing that makes this limit especially tricky for fan-out patterns: it resets per invocation, not per batch or queue message. And the count includes more than you'd expect — every fetch(), every KV get/put, every R2 operation, every D1 query, every Durable Object stub call. The one useful exception: Queue.sendBatch() does not count against it. That's the escape hatch the fix depends on.

The architectural answer is splitting the fan-out trigger from the fan-out work. The entry Worker receives the request and immediately calls env.QUEUE.sendBatch() with one message per account — zero upstream fetches, zero KV writes, well under any limit. Each queue consumer invocation then handles a small batch of messages (I run max_batch_size = 10), doing its 3 subrequests per account in isolation. Ten messages × 3 subrequests = 30 per invocation. The ceiling becomes irrelevant.

The tradeoff worth knowing before you commit to this pattern: Queues are fire-and-forget. If your use case requires returning all 400 account summaries in a single synchronous HTTP response — like a live dashboard — this approach alone won't get you there. That's a different problem requiring a Durable Object as an aggregation point, which adds its own subrequest considerations.

I wrote up the full breakdown — including the Durable Object aggregation pattern for synchronous fan-out, the exact wrangler.toml consumer config, and the complete subrequest reference table — over on dailymanuallab.com.

Full post →