惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

L
LINUX DO - 最新话题
小众软件
小众软件
C
Check Point Blog
酷 壳 – CoolShell
酷 壳 – CoolShell
V
Visual Studio Blog
Last Week in AI
Last Week in AI
P
Proofpoint News Feed
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
The GitHub Blog
The GitHub Blog
T
Tailwind CSS Blog
Recorded Future
Recorded Future
雷峰网
雷峰网
WordPress大学
WordPress大学
A
Arctic Wolf
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
Project Zero
Project Zero
T
Tor Project blog
T
Threat Research - Cisco Blogs
Scott Helme
Scott Helme
Spread Privacy
Spread Privacy
G
Google Developers Blog
Security Latest
Security Latest
MongoDB | Blog
MongoDB | Blog
T
Threatpost
I
InfoQ
T
Tenable Blog
T
The Exploit Database - CXSecurity.com
S
Security Affairs
H
Hackread – Cybersecurity News, Data Breaches, AI and More
人人都是产品经理
人人都是产品经理
C
Cisco Blogs
博客园 - 三生石上(FineUI控件)
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
aimingoo的专栏
aimingoo的专栏
C
CXSECURITY Database RSS Feed - CXSecurity.com
美团技术团队
Google DeepMind News
Google DeepMind News
The Hacker News
The Hacker News
D
Docker
博客园 - 【当耐特】
大猫的无限游戏
大猫的无限游戏
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
Vercel News
Vercel News
PCI Perspectives
PCI Perspectives
Microsoft Azure Blog
Microsoft Azure Blog
C
CERT Recently Published Vulnerability Notes
月光博客
月光博客
Cloudbric
Cloudbric
A
About on SuperTechFans
F
Fortinet All Blogs

Hacker News

Introducing Claude Opus 4.7 Qwen Studio The Future of Everything is Lies, I Guess: Where Do We Go From Here? GitHub - SeanFDZ/macmind: Single-layer transformer in HyperTalk for the classic Macintosh Show HN: Agent-cache – Multi-tier LLM/tool/session caching for Valkey and Redis Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus GitHub - Nightmare-Eclipse/RedSun: The Red Sun vulnerability repository GitHub - SethPyle376/hiraeth: Local AWS emulator focused on fast integration testing, with SQS support, SQLite-backed state, and a debug-friendly web UI. GitHub - macOS26/Agent: Any AI, replaces Claude Code, Cursor, OpenClaw. Over 18 LLM providers (Claude, OpenAI, Gemini, Ollama, Zai, HF, Qwen) wired into a native Mac app that writes code, builds Xcode projects, bumps versions, manages git, automates Safari, use AppleScript, JS or Accessibility, extend Agent! w/ MCP Servers, run tasks from your iPhone via Messages. YouTube now lets you turn off Shorts I Made a Terminal Pager Burgers | マクドナルド公式 Commands — HackerNews CLI documentation ChatGPT for Excel PiCore - Raspberry Pi Port of Tiny Core Linux Live Nation illegally monopolized ticketing market, jury finds Google Broke Its Promise to Me. Now ICE Has My Data. Founding Engineer at Adaptional | Y Combinator CRISPR takes important step toward silencing Down syndrome’s extra chromosome GitHub - saffron-health/libretto: The AI toolkit for building reliable browser automations US v. Heppner (S.D.N.Y. 2026) no attorney-client privilege for AI chats [pdf] Unexpected €54k billing spike in 13 hours: Firebase browser key without API restrictions used for Gemini requests Retrofitting JIT Compilers into C Interpreters IPv6 – Google The Accursèd Alphabetical Clock Cybersecurity Looks Like Proof of Work Now Fragments: April 14 Cal.com Goes Closed Source: Why AI Security Is Forcing Our Decision | Cal.com - Scheduling Software for Online Bookings Laravel raised money and now injects ads directly into your agent When moving fast, talking is the first thing to break Too much Discussion of the XOR swap trick – Heather Cafe Introduction to Spherical Harmonics for Graphics Programmers The Grand Line Building a Z-Machine in the worst possible language High-Level Rust: Getting 80% of the Benefits with 20% of the Pain GitHub - duguyue100/midnight-captain: Inspired by Midnight Commander, tailored to my taste. How to build a `git diff` driver · Jamie Tanna | Software Engineer Center for Responsible, Decentralized Intelligence at Berkeley The Local Universe’s Expansion Rate Is Clearer Than Ever, but Still Doesn’t Add Up - A new synthesis of astronomical measurements confirms a persistent mismatch that could point to physics beyond current models The air throughout our homes is infused with microplastics. But there are things you can do to breathe less of them The disturbing white paper Red Hat is trying to erase from the internet – OSnews The Future of Everything is Lies, I Guess: Annoyances ‘Abhorrent’: the inside story of the Polymarket gamblers betting millions on war Productive procrastination — Max van IJsselmuiden maps, territory and LMs 447 Terabytes per Square Centimetre at Zero Retention Energy: Non-Volatile Memory at the Atomic Scale on Fluorographane Show HN: Pardonned.com – A searchable database of US Pardons 20 Years on AWS and Never Not My Job The Seasons are Wrong Artemis II crew splashes down near San Diego after historic moon mission We gave an AI a 3 year retail lease in SF and asked it to make a profit | Andon Labs How a dancer with ALS used brainwaves to perform live On filing the corners off my MacBooks Installing every* Firefox extension OpenClaw’s memory is unreliable, and you don’t know when it will break Steve Blank Nowhere Is Safe Chimpanzees in Uganda locked in vicious 'civil war', say researchers watgo - a WebAssembly Toolkit for Go linux/Documentation/process/coding-assistants.rst at master · torvalds/linux GitHub - callumlocke/json-formatter: Makes JSON easy to read. Founding Product Engineer at Bild AI | Y Combinator A compelling title that is cryptic enough to get you to take action on it GitHub - Keychron/Keychron-Keyboards-Hardware-Design: Industrial design files for Keychron keyboards and mice. 100+ models with CAD assets in STEP, DXF, DWG, and PDF. Source-available, with commercial use allowed for original compatible accessories within the license terms. [ANNOUNCE] WireGuardNT v0.11 and WireGuard for Windows v0.6 Released 1D-Chess Helium Is Hard to Replace Cooperative Vectors Introduction | Evolve Keeping a Postgres queue healthy — PlanetScale Our response to the Axios developer tool compromise Do Americans read print books, e-books or audiobooks more? The Zettelkasten Method in Obsidian: A Practical Setup Guide Artemis II Is Competency Porn and We Are Starving For It WeakC4 Flight Viz — Cockpit View A Mexican surveillance giant you’ve never heard of is now watching the U.S. border Surelock: Deadlock-Free Mutexes for Rust RISC-V 101 – what is it and what does it mean for Canonical? | Ubuntu The Problem That Built an Industry How Much Linear Memory Access Is Enough? | Solidean Investigating Split Locks on x86-64 Simplest hash functions Sybilproof reputation mechanisms (2005) [pdf] What is a property? How Complex is my Code? Static code analysis in Kotlin — tools overview Toffoli gates are all you need PGLite evangelism dcmake: a new CMake debugger UI Clojure on Fennel part one: Persistent Data Structures Fragments: April 2 Python Release Python install manager 26.1 The Life and Death of the Book Review - Liberties Introducing Database Traffic Control — PlanetScale Bitcoin miners are losing $19,000 on every BTC produced as difficulty drops 7.8% God sleeps in the minerals Building slogbox Apple Silicon and Virtual Machines: Beating the 2 VM Limit Who was “Not Even Wrong” first? Pokemon Evolution Vs Darwinian Evolution The APL Programming Language Source Code
Benchmarking a Bug Scanner | Detail
2026-05-01 · via Hacker News

We’re all coding with agents now, but delivering high quality software at 10x velocity remains an open problem. Code review bots are an important start, but a lot of bugs are still landing in production. Even top products are accumulating a layer of low-grade brokenness.1 We need new ways to make products secure and high quality.

We built a new kind of bug scanner to solve this problem.

The hard part about building a bug scanner is that any meaningfully complicated codebase has many thousands of bugs, and the vast majority don’t matter. You want to reserve human attention (and your tokens) for the bugs that matter. So, one of the most important ways we benchmark ourselves is that we want the bugs we generate to be significantly more important than the typical finding from a code review bot.

How can we quantify importance?

“Important” is a codebase-relative term. A crash in OpenBSD is a headliner finding for a frontier model launch, but a crash in an experimental personal project might not be worth thinking about.

One way to evaluate ourselves is against review bots that operate against individual PRs in a repo. We find PR bots to be generally high value (we have 3 installed right now!) so they’re a useful baseline for signal-to-noise ratio.

To quantify our interestingness, we compared one week of review bot comments on OpenClaw and vLLM against bug reports from Detail. We chose OpenClaw and vLLM because the two are extremely successful products with wildly different development practices and two very different definitions of “importance”.

We also put Detail in “recent changes mode”, which limits Detail to the same week of changes that the review bots saw. This also means any bugs that Detail found were missed by code review and merged into the repo. I.e., code review bots get the first pass at the code.

We ran a tournament with Sonnet 4.6 as a judge between pairs of findings to decide which was more “important”. Then all the pairwise outcomes were fed into a Bradley-Terry model to produce a global ranking. You can think of this as similar to an ELO score for bug reports. The tournament code is here.

We gave Sonnet 4.6 this prompt:

You are comparing two bug findings from code review tools on the same codebase.

An engineer only has time to investigate one. Pick the one that is more important for them to see.

Finding A: {finding_a}

Finding B: {finding_b}

Use the select_winner tool to pick a or b.

After our first run, we realized that the depth of evidence in the Detail findings was strongly biasing the model in favor of Detail.2 So we took both the review bot comments and the Detail findings and put them through a summarization prompt:

Summarize this code review finding in one sentence.

{finding}

Once we had the importance ranks, we converted them into percentiles and plotted the distribution of findings. The higher the percentile, the more important the finding.

Loading rank data…

We find that Detail heavily skews towards the right, which means that the bugs are very high signal-to-noise compared to the baseline of a CR bot. This is in spite of review bots running on unfinished PRs and Detail running on merged code, which should be harder to find bugs in.

To put a number on the asymmetry, here’s how far each tool’s average bug sits from the average bug across the whole dataset. The numbers are standard errors relative to the combined dataset. Detail at +5σ reflects that a random Detail bug ranks more important than a random code review bot finding ~91% of the time.

Loading rank data…

Detail bugs average 88th percentile importance vs CR bots.

Note that this eval only measures importance of the bugs, which is separate from correctness. If bugs from Detail seem important but are wrong, we won’t have a useful product. We separately measure the acceptance rate for our findings. In the last two months, 82.9% of Detail findings were marked as correct by humans or agents. 3

Show me an example!

Here is an example bug Detail found in PostHog/posthog:

Summary

  • Context: The get_sandbox_for_repository Temporal activity provisions sandbox environments for task runs, including fetching and applying user-defined environment variables from SandboxEnvironment records.
  • Bug: The "agent hardening" refactor (commit 48e7429e29) replaced ctx.get_sandbox_environment() with a direct database query that bypasses the private environment access control check.
  • Actual vs. expected: Any team member can access another user's private sandbox environment secrets; private environments should only be accessible to their creator.
  • Impact: A malicious team member can exfiltrate secrets stored in private sandbox environments (API keys, tokens) by creating a task run that references the victim's private environment ID.

Code with Bug

# get_sandbox_for_repository.py lines 112-125
sandbox_environment = None
if ctx.sandbox_environment_id:
    sandbox_environment = SandboxEnvironment.objects.filter(
        id=ctx.sandbox_environment_id, team=task.team
    ).first()  # <-- BUG 🔴 No privacy check; private envs become accessible to any team member
    if sandbox_environment and sandbox_environment.environment_variables:
        skipped_keys: list[str] = []
        added_keys = 0
        for key, value in sandbox_environment.environment_variables.items():
            if key in RESERVED_SANDBOX_ENVIRONMENT_VARIABLE_KEYS:
                skipped_keys.append(key)
                continue
            environment_variables[key] = value  # <-- BUG 🔴 Injects secrets from another user's private env
            added_keys += 1

Explanation

  • Private sandbox environment access control is implemented in TaskRun.get_sandbox_environment() (via ctx.get_sandbox_environment()), which returns None when env.private and the environment creator differs from task.created_by.
  • The refactor replaced this helper with a direct SandboxEnvironment query filtered only by id and team, dropping the private/created_by check.
  • Because get_sandbox_for_repository reads and injects sandbox_environment.environment_variables, this becomes an authorization bypass that exposes stored secrets to other team members.
  • The API layer also accepts sandbox_environment_id without enforcing privacy, so an attacker can supply a victim’s private environment ID and have its secrets injected into the attacker’s sandbox.

Codebase Inconsistency

  • Another activity still uses the privacy-aware helper:
    # create_sandbox_from_snapshot.py
    sandbox_env = ctx.get_sandbox_environment()  # <-- Privacy check applied
    while get_sandbox_for_repository.py performs a direct query, indicating a regression/oversight rather than intended behavior.
  • Existing model test codifies intended privacy behavior:
    self.assertIsNone(run.get_sandbox_environment())  # <-- Privacy enforced

Failing Test

Add this test to products/tasks/backend/tests/test_api.py:

def test_run_endpoint_rejects_private_environment_from_other_user(self):
    """
    Private sandbox environments should NOT be usable by other team members.
    This test proves the vulnerability exists at the API layer.
    """
    # Create User A with a private sandbox environment
    other_user = User.objects.create_user(email="victim@example.com", first_name="Victim", password="password")
    self.organization.members.add(other_user)
    private_env = SandboxEnvironment.objects.create(
        team=self.team,
        name="Victim's Private",
        private=True,
        created_by=other_user,
        environment_variables={"SECRET_KEY": "secret_value"}
    )
    task = self.create_task()
    # Current user (User B, same team) tries to run with User A's private environment
    response = self.client.post(
        f"/api/projects/@current/tasks/{task.id}/run/",
        {"sandbox_environment_id": str(private_env.id)},
        format="json",
    )
    # SHOULD return 403 Forbidden - user cannot use another's private environment
    # ACTUALLY returns 200 OK - the vulnerability allows this
    self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)

Expected result: 403 returned (privacy enforced) Actual result: 200 OK (privacy bypass)

Exploit Scenario

  1. User A creates a private sandbox environment with secrets in environment_variables.
  2. User B (same team) discovers sandbox_environment_id from task run state.
  3. User B starts a run using the victim’s private sandbox_environment_id.
  4. The activity injects User A’s secret env vars into User B’s sandbox, allowing exfiltration from within the run.

Recommended Fix

Restore the privacy-aware lookup in the activity:

sandbox_environment = ctx.get_sandbox_environment()

and only read/apply environment_variables from that result.

History

This bug was introduced in commit 48e7429e294. The "agent hardening" commit refactored environment variable handling by replacing ctx.get_sandbox_environment() (which delegated to the privacy-aware TaskRun.get_sandbox_environment() method) with a direct database query that only filters by team. The original code path included a check that denied access to private environments unless task.created_by_id == env.created_by_id; the new inline query silently dropped this check, allowing any team member to read secrets from private sandbox environments belonging to other users.

PostHog Tasks lets users run an agent inside of a sandbox. Users can attach env bundles known as “Sandbox Environments” to these tasks which have an optional “private” flag that gates access to the creator. That “private” flag is not respected, and if a teammate knows the UUID of a private Sandbox Environment they can start a task that references it.

This is probably a bug an engineer at PostHog would want to be alerted to. Detail provides a convincing failing test and the codebase context to bootstrap an agent for a fix (including a damning docstring - “Private sandboxes environments should NOT be usable by other team members”).

Despite multiple review bots (Greptile, Codex, Hex Security, Copilot) the bug was merged. The issue was waiting to be exploited by a sinister user.

Claude Code baseline

Daily driver coding agents (e.g. Claude Code, Codex, Amp, Cursor, Devin) are pretty fantastic. Can you get similar performance by just asking them to find the most important bugs in a codebase?

Attempt #1

We gave Claude Code this prompt:

Find the 3 most important bugs in this repo. Use subagents.

With this prompt, Claude Code just searched for existing issues (e.g. grep for FIXME/BUG/HACK, query GitHub Issues). The bugs looked severe, but all of them were already reported.

Attempt #2

We tried again and asked Claude Code not to report existing issues:

Find the 3 most important bugs in this repo. Use subagents. Analyze the code, don’t just search for existing bugs.

After this, Claude Code started producing new issues, but didn’t seem to find anything we deemed noteworthy (Claude Code even acknowledged “[it] cannot confidently produce 3 most important bugs from this pass” in the vLLM experiment).

The Claude Code sessions are here.

Can I try it?

Yes of course. Head to app.detail.dev to get a free scan and see what we find in your codebase.


  1. claude uptime, last 90 days claude.ai status page showing frequent downtime bars over the last 90 days, with 98.84% uptime  ↩︎

  2. This is downstream of a product form factor difference that gave Detail an unfair advantage in our first attempt at comparing bug reports: most people don’t want page-long code review comments, so code review bots don’t generate them, but engineers do prefer bug reports to be thorough, so the reports from Detail are more complete. ↩︎

  3. Fix rate is another option, and we track that as well. This measurement includes other factors beyond intrinsic importance, e.g. easier bugs are more likely to be fixed. ↩︎