惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

N
Netflix TechBlog - Medium
I
InfoQ
Engineering at Meta
Engineering at Meta
Jina AI
Jina AI
Recent Announcements
Recent Announcements
T
The Blog of Author Tim Ferriss
P
Proofpoint News Feed
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
D
Docker
Microsoft Security Blog
Microsoft Security Blog
宝玉的分享
宝玉的分享
Last Week in AI
Last Week in AI
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
GbyAI
GbyAI
博客园 - Franky
博客园 - 聂微东
Microsoft Azure Blog
Microsoft Azure Blog
博客园 - 叶小钗
酷 壳 – CoolShell
酷 壳 – CoolShell
B
Blog RSS Feed
WordPress大学
WordPress大学
MyScale Blog
MyScale Blog
月光博客
月光博客
罗磊的独立博客

Common Crawl

Common Crawl - Blog - Introducing the AI Visibility Audit Common Crawl - Blog - Host- and Domain-Level Web Graphs March, April, and May 2026 Common Crawl - Blog - May 2026 Crawl Archive Now Available Common Crawl - Blog - April 2026 Crawl Archive Now Available in a Hugging Face Storage Bucket Common Crawl - Blog - You can now build directly on Common Crawl from the browser Common Crawl - Blog - Host- and Domain-Level Web Graphs February, March, and April 2026 Common Crawl - Blog - April 2026 Crawl Archive Now Available Common Crawl - Blog - Announcing a Change to Common Crawl Dataset Size Reporting Common Crawl - Blog - Host- and Domain-Level Web Graphs January, February, and March 2026 Common Crawl - Blog - March 2026 Crawl Archive Now Available Common Crawl - Blog - IPv6 Adoption Across the Top 100K Web Hosts Common Crawl - Blog - Web Graph Statistics Gets a Proper Upgrade Common Crawl - Blog - Measuring Web Accessibility from Crawl Archives Common Crawl - Blog - Announcing the Whirlwind Tour of Common Crawl's Datasets Using Java Common Crawl - Blog - Host- and Domain-Level Web Graphs December 2025 and January/February 2026 Common Crawl - Blog - Introducing the New Examples & Resources Browser Common Crawl - Blog - February 2026 Crawl Archive Now Available Common Crawl - Blog - AI Plumbers at FOSDEM’26 Common Crawl - Blog - CC-Citations: A Visualization of Research Papers Referencing Common Crawl Common Crawl - Blog - CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data Common Crawl - Blog - Host- and Domain-Level Web Graphs November/December 2025 and January 2026 Common Crawl - Blog - January 2026 Crawl Archive Now Available Common Crawl - Blog - Web Archives for Social Sciences Datathon, Bristol Common Crawl - Blog - How SEOs Are Using Common Crawl's Web Graph Data for AI Ranking Signals Common Crawl - Blog - GneissWeb Annotations Examples Common Crawl - Blog - Common Crawl at the Mozilla Festival 2025 Common Crawl - Blog - Host- and Domain-Level Web Graphs October, November, December 2025 Common Crawl - Blog - December 2025 Crawl Archive Now Available Common Crawl - Blog - A Sampling of 2025 Research Referencing Common Crawl Common Crawl - Blog - Host- and Domain-Level Web Graphs September, October, and November 2025
Common Crawl - Blog - April 2026 Common Crawl Newsletter
2026-04-06 · via Common Crawl

Here at Common Crawl we have been busy in the first quarter of 2026 improving our tools, building new resources, publishing studies, and, of course, releasing our monthly crawl archives and our host and domain level Web Graphs.

Web Graph Statistics Gets a Proper Upgrade

Our Web Graph Statistics site has been updated with interactive charts, a domain lookup tool for tracking harmonic centrality and PageRank over time, mobile improvements, unified rank tables with OR filtering, and merged degree plots.  More details in our blog post.

Introducing the New Examples & Resources Browser

We've replaced our old Examples and Use Cases pages with a single searchable, filterable browser. 119 resources from 115 contributors, all in one place. Search, filter by type or language, sort, and share links. We welcome community submissions.

Here it is in action:

CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data

In February, we announced the release of CommonLID, a language identification benchmark for the web, covering 109 languages. CommonLID was developed in collaboration with multiple open-source organizations and language community groups.  See our announcement for more.

CC-Citations: A Visualization of Research Papers Referencing Common Crawl

We released an interactive visualization of thousands of research papers using or citing Common Crawl data.  Learn more in our blog post, and browse the visualization on our Hugging Face space.

Measuring Web Accessibility from Crawl Archives

"What can we learn about accessibility from crawl archives?"

A WCAG colour contrast audit of 240 top domains using Common Crawl's February 2026 archive found that four in ten colour pairings fall short of accessibility thresholds.  Only one in five sites are fully compliant.  Read the paper on arXiv, and find more details and links on our blog post.

IPv6 Adoption Across the Top 100K Web Hosts

We probed the 100,000 most-linked web hosts for IPv6 support using the Common Crawl Web Graph.  Only 36.9% are fully reachable over IPv6, with adoption ranging from 71% among the top 100 to 32% in the long tail.  Read more on our blog post.

Whirlwind Tour of Common Crawl's Datasets Using Java

We introduced whirlwind-java,  the second installment in our Whirlwind Tour series, covering crawl structure, index access, and content extraction, giving developers a practical foundation for building Java-based data workflows.  This follows our Whirlwind Tour in Python.  Keep an eye out for our Rust version, coming in the future.