惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

B
Blog
C
Cybersecurity and Infrastructure Security Agency CISA
Microsoft Security Blog
Microsoft Security Blog
B
Blog RSS Feed
云风的 BLOG
云风的 BLOG
G
Google Developers Blog
Recent Announcements
Recent Announcements
A
About on SuperTechFans
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
Google Online Security Blog
Google Online Security Blog
Google DeepMind News
Google DeepMind News
S
Schneier on Security
S
Secure Thoughts
T
The Exploit Database - CXSecurity.com
Martin Fowler
Martin Fowler
P
Proofpoint News Feed
Security Latest
Security Latest
Jina AI
Jina AI
D
Darknet – Hacking Tools, Hacker News & Cyber Security
Recorded Future
Recorded Future
T
Tor Project blog
有赞技术团队
有赞技术团队
H
Hackread – Cybersecurity News, Data Breaches, AI and More
N
News | PayPal Newsroom
博客园 - 三生石上(FineUI控件)
MyScale Blog
MyScale Blog
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
Last Week in AI
Last Week in AI
F
Full Disclosure
Hacker News: Ask HN
Hacker News: Ask HN
Forbes - Security
Forbes - Security
D
DataBreaches.Net
人人都是产品经理
人人都是产品经理
NISL@THU
NISL@THU
C
Cisco Blogs
Recent Commits to openclaw:main
Recent Commits to openclaw:main
Google DeepMind News
Google DeepMind News
Project Zero
Project Zero
IT之家
IT之家
T
Threatpost
Cyberwarzone
Cyberwarzone
O
OpenAI News
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
J
Java Code Geeks
P
Proofpoint News Feed
The Last Watchdog
The Last Watchdog
月光博客
月光博客
Latest news
Latest news
MongoDB | Blog
MongoDB | Blog
Apple Machine Learning Research
Apple Machine Learning Research

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
I Let 12 AI Models Predict the World Cup. The First 169 Picks Already Show a Pattern.
tokenmixai · 2026-06-18 · via DEV Community

I put 12 AI models into a public World Cup prediction arena.

Not because I think anyone should use LLMs for betting. They should not. The page says entertainment only for a reason.

I did it because sports prediction is a surprisingly clean stress test for models:

  • structured facts
  • stale priors
  • uncertainty
  • calibration
  • price-performance
  • and the most painful thing for LLMs: admitting a favorite might draw

After 169 predictions and 21 settled scoring entries, the leaderboard is technically tied.

But the misses are already more useful than the winners.

TL;DR

  • No, there is no "best World Cup AI model" yet. The sample is too small.
  • 12 models are currently tied on 3 points.
  • Qwen3.5 Flash, Claude Opus 4.7, and Claude Sonnet 4.6 show 100% winner accuracy, but only on one settled pre-match prediction each.
  • All 12 models got Colombia over Uzbekistan directionally right.
  • Nine valid pre-match models all missed Portugal 1-1 Congo DR because they picked Portugal.
  • The early lesson is not "flagship models win." It is "favorite bias is real, and cheap models are good enough to poll at scale."

Full live scoreboard: WorldCup AI Arena

What I actually tracked

The public dashboard tracks model forecasts, match results, team context, and prediction accuracy.

Snapshot used here: 2026-06-18 05:53 UTC.

Metric Value
Models tracked 12
Total predictions 169
Settled scoring entries 21
Total leaderboard points 36
Exact score hits 0
Correct-winner hits 12
Average winner accuracy 62.5%

The model list includes Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, and Grok variants.

Important caveat: I count pre-match predictions only for accuracy. Post-match reviews are useful for explanation, but they know the result. They are not forecasts.

The current leaderboard

Every model has 3 points right now.

That sounds boring until you look at the sample size.

Model Tier Predictions Settled Winner hits Points Accuracy
Qwen3.5 Flash wildcard 13 1 1 3 100%
Claude Opus 4.7 flagship 14 1 1 3 100%
Claude Sonnet 4.6 flagship 14 1 1 3 100%
GPT-5.4 flagship 15 2 1 3 50%
Gemini 3.1 Pro flagship 15 2 1 3 50%
DeepSeek V4 Pro value 15 2 1 3 50%
Qwen 3.7 Plus value 14 2 1 3 50%
Kimi K2.6 value 14 2 1 3 50%
Gemini 2.5 Flash value 14 2 1 3 50%
Grok 4.1 Fast Reasoning wildcard 14 2 1 3 50%
DeepSeek V4 Flash wildcard 14 2 1 3 50%
GPT-5 Nano wildcard 13 2 1 3 50%

My read: the leaderboard is not mature enough to crown a winner.

The first useful signal is elsewhere.

The obvious match: everyone got Colombia right

Uzbekistan vs Colombia ended 1-3.

All 12 models picked Colombia.

None got the exact score.

Model Prediction Final Winner hit
Claude Opus 4.7 0-2 Colombia 1-3 Colombia Yes
Claude Sonnet 4.6 1-2 Colombia 1-3 Colombia Yes
GPT-5.4 1-2 Colombia 1-3 Colombia Yes
Gemini 3.1 Pro 0-2 Colombia 1-3 Colombia Yes
DeepSeek V4 Pro 0-2 Colombia 1-3 Colombia Yes
Qwen 3.7 Plus 0-2 Colombia 1-3 Colombia Yes
Kimi K2.6 0-2 Colombia 1-3 Colombia Yes
Gemini 2.5 Flash 0-2 Colombia 1-3 Colombia Yes
Grok 4.1 Fast Reasoning 0-2 Colombia 1-3 Colombia Yes
DeepSeek V4 Flash 0-2 Colombia 1-3 Colombia Yes
GPT-5 Nano 0-1 Colombia 1-3 Colombia Yes
Qwen3.5 Flash 0-1 Colombia 1-3 Colombia Yes

This is the kind of match where a cheap model can be enough.

If all you need is "which side is more likely," then polling cheap models may beat paying a flagship model for every pick.

The useful miss: every valid model missed Portugal-Congo DR

Portugal vs Congo DR ended 1-1.

Every valid pre-match model picked Portugal.

Model Prediction Final Outcome
GPT-5.4 2-0 Portugal 1-1 Miss
Gemini 3.1 Pro 2-0 Portugal 1-1 Miss
DeepSeek V4 Pro 2-0 Portugal 1-1 Miss
Qwen 3.7 Plus 2-0 Portugal 1-1 Miss
Kimi K2.6 2-0 Portugal 1-1 Miss
Gemini 2.5 Flash 2-0 Portugal 1-1 Miss
Grok 4.1 Fast Reasoning 3-0 Portugal 1-1 Miss
DeepSeek V4 Flash 2-0 Portugal 1-1 Miss
GPT-5 Nano 2-1 Portugal 1-1 Miss

That is the part I care about.

The models did not just get unlucky independently. They shared the same prior: Portugal strong, Congo DR weaker, therefore Portugal win.

That is a classic LLM failure mode.

It shows up outside sports too:

  • "OpenAI usually ships X, so the next release will be X"
  • "Claude is the premium model, so it must win this task"
  • "The famous team/vendor/person is probably the right answer"
  • "Historical quality beats current uncertainty"

In other words, the World Cup is a cute interface for a serious eval problem: models are often too willing to convert reputation into certainty.

The cost angle

The dashboard includes listed price tiers for each model.

Here is the funny part: the cheapest model currently has the cleanest-looking row.

Model Listed input / output price Current result
Qwen3.5 Flash $0.026 / $0.263 per 1M 1/1 winner hit
GPT-5 Nano $0.049 / $0.388 per 1M 1/2 winner hit
Claude Opus 4.7 $5 / $25 per 1M 1/1 winner hit
GPT-5.4 $2.45 / $14.7 per 1M 1/2 winner hit

Do not overread that. One match is not proof.

But the unit economics are hard to ignore.

Suppose a prediction prompt uses 10K input tokens and 1K output tokens.

Approximate cost:

Qwen3.5 Flash:
10K * $0.026 / 1M + 1K * $0.263 / 1M = $0.000526

Claude Opus 4.7:
10K * $5 / 1M + 1K * $25 / 1M = $0.075

That is roughly a 143x spread for one prediction-shaped call.

If I were building a prediction system, I would not send every match to the most expensive model. I would route it.

def pick_prediction_route(match_uncertainty, model_disagreement, budget_mode):
    if budget_mode == "cheap_poll":
        return ["qwen3.5-flash", "gpt-5-nano", "deepseek-v4-flash"]

    if match_uncertainty == "low" and model_disagreement == "low":
        return ["qwen3.5-flash"]

    if match_uncertainty == "high" or model_disagreement == "high":
        return [
            "qwen3.5-flash",
            "deepseek-v4-pro",
            "gemini-3.1-pro",
            "claude-sonnet-4.6",
        ]

    return ["qwen3.5-flash", "claude-sonnet-4.6"]

Cheap models for breadth. Expensive models for disagreement.

That is the same routing logic I use for normal API workloads.

What I would measure next

Winner accuracy is not enough.

I want these metrics:

Metric Why it matters
Winner accuracy Basic direction
Exact score Hard mode
Goal difference More informative than exact score alone
Brier score Calibration
Confidence bucket accuracy Overconfidence detection
Cost per correct winner Production routing
Draw recall Favorite-bias detector
Disagreement value Whether ensembles help

The biggest one is draw recall.

Portugal-Congo DR already suggests the models may underpredict draws when a prestigious team is involved.

If that pattern holds, it is more important than the leaderboard.

What I'd do if I were tracking this live

I would not declare a winner until at least 30-50 settled pre-match predictions per model.

For now:

  • Track every match.
  • Exclude post-match reviews from accuracy.
  • Compare cheap vs flagship models by cost per correct winner.
  • Watch draw prediction rate.
  • Add a baseline from betting markets or Elo.
  • Update after each matchday.

If you want the full data-cited writeup and live links, I wrote the original breakdown here: AI World Cup Predictions 2026: 12 Models, Early Leaderboard.

Disclosure: I work on the research side at TokenMix, which is why I can wire this kind of multi-model scoreboard quickly.

Bottom line

The early World Cup AI leaderboard does not tell us which model is best yet.

It does tell us something useful: cheap models can match flagship consensus on obvious favorites, and all models can share the same bad prior on a draw.

That is a model-evaluation lesson, not betting advice.

If you were scoring this, would you reward exact score heavily, or focus on calibrated probabilities instead?