惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

小众软件
小众软件
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
WordPress大学
WordPress大学
月光博客
月光博客
Hugging Face - Blog
Hugging Face - Blog
博客园 - 聂微东
博客园 - 【当耐特】
博客园_首页
The Cloudflare Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Apple Machine Learning Research
Apple Machine Learning Research
Last Week in AI
Last Week in AI
酷 壳 – CoolShell
酷 壳 – CoolShell
大猫的无限游戏
大猫的无限游戏
雷峰网
雷峰网
量子位
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
T
Tailwind CSS Blog
IT之家
IT之家
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
GbyAI
GbyAI
V
Visual Studio Blog
F
Fortinet All Blogs
Martin Fowler
Martin Fowler

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
Your bandit's exploration floor probably violates its own...
Whatsonyourmind · 2026-06-18 · via DEV Community

Most multi-armed bandit / A-B allocation systems add a minimum exploration weight: every arm should get at least, say, 5% of traffic, so no variant is ever fully starved and you keep collecting data on all of them. The guarantee sounds simple — p_i >= f for every arm — and the implementation looks even simpler:

def clip_renorm(w, f):
    p = np.maximum(w, f)   # raise anything below the floor up to it
    return p / p.sum()     # renormalize so probabilities sum to 1

This is wrong, and it fails silently. The renormalize step pushes the floored arms back below the floor.

Why clip-then-renormalize breaks

Clipping raises the small weights up to f, which makes the total exceed 1. Dividing by that total then scales everything down — including the arms you just clipped to f. So they land below f again, and the floor you advertised is not the floor you enforce.

Concrete case — 4 arms, a confident winner, floor f = 0.10:

w   = [0.94, 0.02, 0.02, 0.02]   floor = 0.10
clip-renorm -> [0.7581, 0.0806, 0.0806, 0.0806]   min = 0.0806  ❌ (< 0.10)

The three starved arms each get 8.06%, not the 10% you promised. And it isn't an edge case. Over 100,000 random peaky weight vectors (Dirichlet, α=0.3, n=4, f=0.10):

clip-and-renormalize violated the floor 97.2% of the time — worst arm seen: 7.69% against a 10% floor.

Whenever one arm dominates (exactly when a bandit is exploiting), the floor leaks.

The fix: one affine map onto the simplex

Instead of clipping, mix the learned weights with the uniform floor. Put the weights on the simplex (sum(w) = 1), then:

def additive_simplex(w, f):
    w = w / w.sum()
    return f + (1.0 - len(w) * f) * w

Each output is f + (non-negative), so p_i >= f holds exactly, and the total is n*f + (1 - n*f)*1 = 1 by construction — no renormalization needed, so nothing gets dragged back under the floor. It also preserves the ordering and relative spacing of w (it's affine), so you don't distort the policy you learned. Same run:

additive-simplex -> [0.664, 0.112, 0.112, 0.112]   min = 0.112  ✅

Over the same 100,000 vectors it violated the floor 0.00% of the time.

The one guard you do need

The map needs n * f <= 1 — you can't promise four arms a 30% floor each (that's 120%). Handle it explicitly instead of producing negative weights:

def exploration_floor(w, f):
    n = len(w)
    if f < 0:
        raise ValueError("floor must be non-negative")
    if n * f >= 1.0:
        return np.full(n, 1.0 / n)          # floor is infeasible -> uniform
    w = np.asarray(w, dtype=float)
    w = w / w.sum()
    return f + (1.0 - n * f) * w

That's the whole correct primitive: a non-negativity check, an infeasible-floor fallback to uniform, and the affine mix.

Why it actually matters

The exploration floor isn't cosmetic. It's what bounds worst-case regret and guarantees you keep collecting data on every arm — the property a lot of bandit regret arguments lean on, and often a fairness/SLA requirement too ("no variant ever drops below X%"). A floor that's silently 7.7% instead of 10% means the guarantee you reported to stakeholders, and any bound that depends on it, doesn't hold. The bug is invisible because the output still sums to 1 and still looks floored — the smallest number is just quietly too small.

import numpy as np
rng = np.random.default_rng(0)
f, n, viol = 0.10, 4, 0
for _ in range(100_000):
    w = rng.dirichlet(np.ones(n) * 0.3)
    p = np.maximum(w, f); p = p / p.sum()       # clip-renorm
    if p.min() < f - 1e-12: viol += 1
print(f"clip-renorm floor violations: {viol/100_000:.1%}")   # ~97%

I ran into this reviewing a Thompson-sampling weighting routine and proposed the additive-simplex version (plus the two guards) as a fix upstream. If your bandit or weighted-experiment layer clips-then-renormalizes to enforce a minimum, it's worth a one-line check: does the smallest probability it emits actually clear the floor?