惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

小众软件
小众软件
博客园_首页
M
MIT News - Artificial intelligence
雷峰网
雷峰网
GbyAI
GbyAI
博客园 - 叶小钗
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
V
V2EX
S
SegmentFault 最新的问题
H
Help Net Security
Apple Machine Learning Research
Apple Machine Learning Research
H
Hackread – Cybersecurity News, Data Breaches, AI and More
博客园 - 【当耐特】
V
Visual Studio Blog
月光博客
月光博客
G
Google Developers Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
腾讯CDC
云风的 BLOG
云风的 BLOG
美团技术团队
Microsoft Azure Blog
Microsoft Azure Blog
A
About on SuperTechFans
有赞技术团队
有赞技术团队

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
Rate Limiting and API Key Management for Video Data APIs
ahmet gedik · 2026-05-01 · via DEV Community

The YouTube Data API v3 gives you 10,000 quota units per day per key. That sounds generous until you realize a single search request costs 100 units and a video details call costs 1 unit per video. Do the math for a platform that fetches trending content from 8 regions every 2 hours, and you hit the wall fast. Here's how DailyWatch handles it.

Understanding YouTube API Quotas

Not all API calls are equal. The quota cost varies by endpoint:

Endpoint Cost per call
search.list 100 units
videos.list 1 unit
channels.list 1 unit
playlistItems.list 1 unit

A single search.list call returning 50 results costs the same 100 units as one returning 5. Always request maxResults=50 to maximize value per call.

Multi-Key Rotation

One key gives 10,000 units. Three keys give 30,000. The rotation logic is straightforward:

class ApiKeyManager {
    private array $keys;
    private PDO $db;

    public function __construct(PDO $db) {
        $this->db = $db;
        $this->keys = $this->loadKeys();
    }

    private function loadKeys(): array {
        $stmt = $this->db->query(
            "SELECT api_key, daily_usage, last_reset 
             FROM api_keys 
             WHERE active = 1 
             ORDER BY daily_usage ASC"
        );
        return $stmt->fetchAll(PDO::FETCH_ASSOC);
    }

    public function getNextKey(): ?string {
        foreach ($this->keys as $key) {
            // Reset counter if new day (Pacific Time, matching Google's reset)
            if ($this->isNewDay($key['last_reset'])) {
                $this->resetUsage($key['api_key']);
                return $key['api_key'];
            }
            // Skip exhausted keys
            if ($key['daily_usage'] < 9500) { // 500 unit buffer
                return $key['api_key'];
            }
        }
        return null; // All keys exhausted
    }

    public function recordUsage(string $apiKey, int $units): void {
        $stmt = $this->db->prepare(
            "UPDATE api_keys 
             SET daily_usage = daily_usage + ?, last_used = datetime('now') 
             WHERE api_key = ?"
        );
        $stmt->execute([$units, $apiKey]);
    }
}

Enter fullscreen mode Exit fullscreen mode

Keys are sorted by usage ascending, so the least-used key always gets picked first. The 500-unit buffer prevents accidentally exceeding the quota on the last call of the day.

Rate Limiting the Fetch Process

Even with multiple keys, you don't want to fire 50 API calls in a tight loop. Google enforces per-second rate limits too:

class RateLimiter {
    private float $lastCallTime = 0;
    private float $minInterval;

    public function __construct(float $callsPerSecond = 5.0) {
        $this->minInterval = 1.0 / $callsPerSecond;
    }

    public function wait(): void {
        $elapsed = microtime(true) - $this->lastCallTime;
        if ($elapsed < $this->minInterval) {
            usleep((int)(($this->minInterval - $elapsed) * 1_000_000));
        }
        $this->lastCallTime = microtime(true);
    }
}

// Usage in fetch loop
$limiter = new RateLimiter(3.0); // 3 calls per second
$keyManager = new ApiKeyManager($db);

foreach ($regions as $region) {
    $limiter->wait();
    $apiKey = $keyManager->getNextKey();
    if (!$apiKey) {
        log_message('All API keys exhausted, stopping fetch');
        break;
    }
    $results = fetchTrending($apiKey, $region);
    $keyManager->recordUsage($apiKey, 100); // search cost
}

Enter fullscreen mode Exit fullscreen mode

Quota-Efficient Fetching

The biggest optimization is reducing search calls. Instead of searching per category per region, batch smartly:

// BAD: 8 regions x 15 categories = 120 search calls = 12,000 units
foreach ($regions as $region) {
    foreach ($categories as $category) {
        searchVideos($region, $category); // 100 units each
    }
}

// GOOD: Use chart=mostPopular (1 unit) + videoCategoryId filter
foreach ($regions as $region) {
    // 1 unit instead of 100!
    $popular = getPopularVideos($apiKey, $region, 50);
    foreach ($categories as $category) {
        $catVideos = getPopularVideos($apiKey, $region, 50, $category);
    }
}

Enter fullscreen mode Exit fullscreen mode

The videos.list endpoint with chart=mostPopular costs 1 unit versus 100 for search.list. For 8 regions with 15 categories, that's 128 units instead of 12,000.

Handling 403 Quota Exceeded

When a key hits its limit, Google returns HTTP 403 with a specific error reason. Handle it gracefully:

function makeApiCall(ApiKeyManager $km, string $url): ?array {
    $maxRetries = count($km->getKeys());
    for ($i = 0; $i < $maxRetries; $i++) {
        $key = $km->getNextKey();
        if (!$key) return null;

        $response = httpGet($url . '&key=' . $key);

        if ($response['status'] === 200) {
            return json_decode($response['body'], true);
        }

        if ($response['status'] === 403) {
            $error = json_decode($response['body'], true);
            $reason = $error['error']['errors'][0]['reason'] ?? '';

            if ($reason === 'quotaExceeded' || $reason === 'dailyLimitExceeded') {
                $km->markExhausted($key);
                continue; // Try next key
            }
        }
    }
    return null;
}

Enter fullscreen mode Exit fullscreen mode

The loop automatically rotates to the next available key on quota errors. If all keys are exhausted, it returns null and the caller decides whether to retry later.

Monitoring and Alerts

Track daily usage in the database and alert when approaching limits:

function checkQuotaHealth(PDO $db): array {
    $stmt = $db->query(
        "SELECT api_key, daily_usage,
                ROUND(daily_usage * 100.0 / 10000, 1) AS usage_pct
         FROM api_keys WHERE active = 1"
    );
    $keys = $stmt->fetchAll(PDO::FETCH_ASSOC);
    $totalUsed = array_sum(array_column($keys, 'daily_usage'));
    $totalAvailable = count($keys) * 10000;

    return [
        'keys_active'    => count($keys),
        'total_used'     => $totalUsed,
        'total_available' => $totalAvailable,
        'usage_percent'  => round($totalUsed / $totalAvailable * 100, 1),
        'per_key'        => $keys,
    ];
}

Enter fullscreen mode Exit fullscreen mode

On DailyWatch, this data feeds into a simple admin dashboard. When combined usage crosses 80%, the system automatically reduces fetch frequency for lower-priority regions.

The key insight: treat API quotas as a finite resource to be budgeted, not a limit to be worked around. Design your fetch strategy around the quota, not the other way around.


This article is part of the Building DailyWatch series.