A compromised package can steal credentials the moment you install it, and until recently, GitHub could only flag those in npm. Not anymore. This is the story of how the supply chain engineering team behind Dependabot expanded malware advisories to eight ecosystems by building on OpenSSF’s shared malicious packages data.
Here’s where things stand: earlier this year, Dependabot started flagging malware in your npm dependencies. Great news if you write JavaScript. Now we’re bringing that same functionality to PyPI.
We’ve enhanced the GitHub Advisory Database to ingest malware reports from OpenSSF’s malicious-packages repository, which means malware advisories and the Dependabot alerts they power cover all eight major package ecosystems: npm, PyPI, Maven, RubyGems, NuGet, Go, crates.io, and PHP Composer. I lead the Dependabot team in GitHub’s supply chain security organization, and in this post, I’ll show you how this pipeline works.
The Advisory Database has imported vulnerability data from external sources for years. RubySec for gems, RustSec for crates, PyPA for Python. Each one is an importer that reads a public advisory repo and maps records into our database. Malware was the odd one out: it flowed through a separate, internal, npm-only path, built around GitHub’s own detection of malicious npm packages.
Expanding the existing detection from one to eight supported ecosystems would have taken us years. Meanwhile, OpenSSF has already solved the aggregation problem for everybody. Their malicious-packages repo launched in 2023, with over 15,000 reports in OSV format. Since then, it has grown every day, fed by community submissions and automated detection sources across the industry: typosquats, dependency-confusion packages, account takeovers, malicious prebuilt binaries. It’s public, it’s structured, and it covers any ecosystem the OSV schema supports.
So, the design nearly wrote itself. Rather than building eight unique detection systems, we built one importer.
We reused the same pattern our repo-based importers already followed to walk the source repository’s file tree, pick up files changed since the last run, and process each one. The new OpenSSF importer reads every OSV record and validates the required fields, types, and format against the schema before anything touches the database. A record that fails this validation gets rejected and logged. It’s never quietly patched up and waved through, because a “mostly valid” malware advisory is exactly the kind of thing that bites you six months later.
Valid records get normalized into feed entries: the source, an identifier, a CVE ID when one exists, the complete upstream record preserved as a snapshot, and the mapped subset from our publishing pipeline consumes.
Normalizing sounds boring until you meet the data. Upstream ecosystem strings don’t always match ours (the repo says PyPI, our database says pip). OSV records list affected versions as discrete values where we think in ranges, and some records name no usable version at all. The details field is frequently empty, and when several sources report the same package, their write-ups get appended into one blob. Reports also get retracted: the repo keeps a whole osv/withdrawn folder for advisories that turned out to be wrong, so the importer must cope with a package being flagged on Monday and disavowed on Wednesday.
Then there’s the dedup problem, and it’s a fun one. GitHub is itself a contributor to the OpenSSF repo; our own npm malware advisories flow upstream into it. Import the repo naively, and we’d be re-importing our own data in a loop. The fix rides on OSV’s origin metadata: every entry in malicious-packages records where the report came from, and anything tagged ghsa-malware began with us. The importer drops those before a feed entry is ever created.
When we validated against live data, more than half of the new npm reports flowing into the repo each month traced back to our own advisories and were skipped as round-trips, so what the importer picks up is the stuff we genuinely didn’t know about.
One question dominated our security review: what happens if the upstream data goes bad?
Malware advisories auto-publish. No human reads each one before it goes out, and that’s deliberate. When a package is stealing credentials right now, a review queue measured in days is a gift to the attacker. The deliberate departure is that these auto-published advisories can now generate Dependabot alerts. Our unreviewed advisories were already published automatically, but this is the first time an auto-published advisory can trigger an alert, and it’s worth being clear about why.
My colleague Madison Ficorilli recently wrote about what “reviewed” actually means for advisories about vulnerabilities: human curators verifying package mappings, version ranges, and severity before anything ships. That rigor earns its delay when the question is which versions of a library are vulnerable. Malware is a different beast. The report is close to binary (this package is hostile), and hours matter more than nuance. The design assumes the upstream feed could one day carry bad data: a false report flagging a legitimate, widely used package as malware, a record with the wrong package name, or a whole batch published from a compromised source.
So, we built a resilient ingestion pipeline with three layers of protection for exactly that day. Here’s how they work.
Dependabot and GitHub will now alert you if you use a malicious dependency across most package ecosystems.
Malware alerts are opt-in: enable them in your repository, organization, or enterprise security settings. Dependabot will match your dependencies against malware advisories in the Advisory Database, including a backfill against existing advisories, starting the moment you turn it on.
Ankit is a Senior Engineering Manager at GitHub, where he leads the Dependabot team in the Supply Chain Security organization. Dependabot watches over 30M+ repositories across 34+ package ecosystems, which keeps him appropriately paranoid about supply chain attacks.
Build what’s next on GitHub, the place for anyone from anywhere to build anything.
Join us October 28-29 in San Francisco or online for GitHub Universe, our flagship developer event uniting people, agents, and the world’s code.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。
