惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Google DeepMind News
Google DeepMind News
The Last Watchdog
The Last Watchdog
腾讯CDC
Apple Machine Learning Research
Apple Machine Learning Research
有赞技术团队
有赞技术团队
Last Week in AI
Last Week in AI
量子位
雷峰网
雷峰网
宝玉的分享
宝玉的分享
美团技术团队
阮一峰的网络日志
阮一峰的网络日志
博客园 - 叶小钗
P
Privacy International News Feed
NISL@THU
NISL@THU
V
V2EX
博客园 - 三生石上(FineUI控件)
T
The Exploit Database - CXSecurity.com
T
Threat Research - Cisco Blogs
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
IT之家
IT之家
T
Tor Project blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
T
Threatpost
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
大猫的无限游戏
大猫的无限游戏
V
Vulnerabilities – Threatpost
V
Visual Studio Blog
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
The Hacker News
The Hacker News
Scott Helme
Scott Helme
H
Hacker News: Front Page
罗磊的独立博客
Hacker News - Newest:
Hacker News - Newest: "LLM"
K
Kaspersky official blog
S
Secure Thoughts
www.infosecurity-magazine.com
www.infosecurity-magazine.com
The Cloudflare Blog
D
Darknet – Hacking Tools, Hacker News & Cyber Security
C
Cybersecurity and Infrastructure Security Agency CISA
小众软件
小众软件
月光博客
月光博客
人人都是产品经理
人人都是产品经理
AI
AI
Help Net Security
Help Net Security
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
酷 壳 – CoolShell
酷 壳 – CoolShell
Webroot Blog
Webroot Blog
博客园 - 【当耐特】
PCI Perspectives
PCI Perspectives
A
Arctic Wolf

Common Crawl

Common Crawl - Blog - Introducing the AI Visibility Audit Common Crawl - Blog - Host- and Domain-Level Web Graphs March, April, and May 2026 Common Crawl - Blog - May 2026 Crawl Archive Now Available Common Crawl - Blog - April 2026 Crawl Archive Now Available in a Hugging Face Storage Bucket Common Crawl - Blog - You can now build directly on Common Crawl from the browser Common Crawl - Blog - Host- and Domain-Level Web Graphs February, March, and April 2026 Common Crawl - Blog - April 2026 Crawl Archive Now Available Common Crawl - Blog - April 2026 Common Crawl Newsletter Common Crawl - Blog - Announcing a Change to Common Crawl Dataset Size Reporting Common Crawl - Blog - Host- and Domain-Level Web Graphs January, February, and March 2026 Common Crawl - Blog - March 2026 Crawl Archive Now Available Common Crawl - Blog - IPv6 Adoption Across the Top 100K Web Hosts Common Crawl - Blog - Web Graph Statistics Gets a Proper Upgrade Common Crawl - Blog - Measuring Web Accessibility from Crawl Archives Common Crawl - Blog - Announcing the Whirlwind Tour of Common Crawl's Datasets Using Java Common Crawl - Blog - Host- and Domain-Level Web Graphs December 2025 and January/February 2026 Common Crawl - Blog - Introducing the New Examples & Resources Browser Common Crawl - Blog - February 2026 Crawl Archive Now Available Common Crawl - Blog - AI Plumbers at FOSDEM’26 Common Crawl - Blog - CC-Citations: A Visualization of Research Papers Referencing Common Crawl Common Crawl - Blog - CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data Common Crawl - Blog - Host- and Domain-Level Web Graphs November/December 2025 and January 2026 Common Crawl - Blog - January 2026 Crawl Archive Now Available Common Crawl - Blog - Web Archives for Social Sciences Datathon, Bristol Common Crawl - Blog - How SEOs Are Using Common Crawl's Web Graph Data for AI Ranking Signals Common Crawl - Blog - GneissWeb Annotations Examples Common Crawl - Blog - Common Crawl at the Mozilla Festival 2025 Common Crawl - Blog - Host- and Domain-Level Web Graphs October, November, December 2025 Common Crawl - Blog - December 2025 Crawl Archive Now Available Common Crawl - Blog - A Sampling of 2025 Research Referencing Common Crawl Common Crawl - Blog - Host- and Domain-Level Web Graphs September, October, and November 2025 Common Crawl - Blog - November 2025 Crawl Archive Now Available Common Crawl - Blog - Common Crawl Celebrates World Digital Preservation Day Common Crawl - Blog - Setting the Record Straight: Common Crawl’s Commitment to Transparency, Fair Use, and the Public Good Common Crawl - Blog - October/November 2025 Newsletter Common Crawl - Blog - Common Crawl Foundation at Stanford HAI Common Crawl - Blog - Host- and Domain-Level Web Graphs August, September, and October 2025 Common Crawl - Blog - October 2025 Crawl Archive Now Available Common Crawl - Blog - Common Crawl Foundation at COLM 2025 Common Crawl - Blog - Announcing GneissWeb Annotations Common Crawl - Blog - Web Languages Needing Review by Native Speakers Common Crawl - Blog - Host- and Domain-Level Web Graphs July, August, and September 2025 Common Crawl - Blog - From SEO to AIO: Why Your Content Needs to Exist in AI Training Data Common Crawl - Blog - September 2025 Crawl Archive Now Available Common Crawl - Blog - Common Crawl Foundation Opt-Out Registry Common Crawl - Blog - Trip Report: AI_dev (Linux Foundation) August 2025 Common Crawl - Blog - Common Crawl Foundation at Stanford HAI: A Shared Legacy of Data and Innovation Common Crawl - Blog - July/August 2025 Newsletter Common Crawl - Blog - Host- and Domain-Level Web Graphs June, July, and August 2025 Common Crawl - Blog - August 2025 Crawl Archive Now Available Common Crawl - Blog - Common Crawl Foundation at ACL 2025 Common Crawl - Blog - AI Optimization Is Here: Are You Ready for Search 2.0? Common Crawl - Blog - IETF 123 Report Common Crawl - Blog - Host- and Domain-Level Web Graphs May, June, and July 2025 Common Crawl - Blog - July 2025 Crawl Archive Now Available Common Crawl - Blog - WMDQS Shared Task on Language Identification Common Crawl - Blog - The First WMDQS-Masakhane LangID Hackathon Common Crawl - Blog - Host- and Domain-Level Web Graphs April, May, and June 2025 Common Crawl - Blog - Common Crawl at the United Nations Open Source Week, June 2025 Common Crawl - Blog - June 2025 Crawl Archive Now Available Common Crawl - Blog - May/June 2025 Newsletter Common Crawl - Blog - Announcing the Whirlwind Tour of Common Crawl's Datasets using Python Common Crawl - Blog - Host- and Domain-Level Web Graphs March, April, and May 2025 Common Crawl - Blog - May 2025 Crawl Archive Now Available Common Crawl - Blog - Announcing the First Workshop on Multilingual Data Quality Signals Common Crawl - Blog - Host- and Domain-Level Web Graphs February, March, and April 2025 Common Crawl - Blog - April 2025 Crawl Archive Now Available Common Crawl - Blog - Introducing the Host Index Common Crawl - Blog - IIPC General Assembly & Web Archiving Conference 2025 Common Crawl - Blog - March/April 2025 Newsletter Common Crawl - Blog - Providing Authenticity & Data Provenance for Common Crawl Using Blockchain: Our Work with Constellation Network Common Crawl - Blog - Host- and Domain-Level Web Graphs January, February, and March 2025 Common Crawl - Blog - March 2025 Crawl Archive Now Available Common Crawl - Blog - Introducing Common Crawl AI Agent by ReadyAI Common Crawl - Blog - Submission to the UK’s Copyright and AI Consultation Common Crawl - Blog - Host- and Domain-Level Web Graphs December 2024 and January/February 2025 Common Crawl - Blog - February 2025 Crawl Archive Now Available Common Crawl - Blog - Opening the Gates to Online Safety Common Crawl - Blog - January/February 2025 Newsletter Common Crawl - Blog - Host- and Domain-Level Web Graphs November/December 2024 and January 2025 Common Crawl - Blog - January 2025 Crawl Archive Now Available Common Crawl - Blog - Introducing cc-downloader Common Crawl - Blog - Host- and Domain-Level Web Graphs October, November, and December 2024 Common Crawl - Blog - December 2024 Crawl Archive Now Available Common Crawl - Blog - Expanding the Language and Cultural Coverage of Common Crawl Common Crawl - Blog - October/November 2024 Newsletter Common Crawl - Blog - Host- and Domain-Level Web Graphs September, October, November 2024 Common Crawl - Blog - November 2024 Crawl Archive Now Available Common Crawl - Blog - Reflections on Recent Talks at the Turing Institute and UCL Common Crawl - Blog - Introducing the Common Crawl Errata Page for Data Transparency Common Crawl - Blog - Host- and Domain-Level Web Graphs August, September, and October 2024 Common Crawl - Blog - October 2024 Crawl Archive Now Available Common Crawl - Blog - White House Briefing on Open Data’s Role in Technology Common Crawl - Blog - IAB Workshop on AI-CONTROL Common Crawl - Blog - Host- and Domain-Level Web Graphs July, August, and September 2024 Common Crawl - Blog - September 2024 Crawl Archive Now Available Common Crawl - Blog - August/September 2024 Newsletter Common Crawl - Blog - Host- and Domain-Level Web Graphs June, July, and August 2024 Common Crawl - Blog - August 2024 Crawl Archive Now Available Common Crawl - Blog - The Increase of Common Crawl Citations in Academic Research
Common Crawl - Blog - Common Crawl Foundation at NeurIPS 2024: Expanding Horizons and Building Connections
2024-12-17 · via Common Crawl

Last week, members of the Common Crawl Foundation team—Chris, Greg, Jason, Rich, Sam, Stephen, and Wayne—attended the Neural Information Processing Systems (NeurIPS) Conference at the Vancouver Convention Centre in downtown Vancouver, BC. Set against the backdrop of Vancouver’s stunning waterfront, with snow-capped mountains and vibrant cityscape, the conference drew over 7,000 attendees from around the world.

Attendees at the Neurips 2024 conference in Vancouver BC.

Attendees at the NeurIPS 2024 conference in Vancouver BC.

Meaningful Connections and Opportunities

We attended NeurIPS with the goal of understanding potential partnerships and learning from the AI research community. During the conference, we had the opportunity to meet with people from over 40 organizations, each conversation offering insights into potential collaborations and ways we might support the broader AI ecosystem.

Common Crawl and Wikimedia Social: Bridging Tech and Social Impact

Our signature event at NeurIPS was a compelling social gathering titled "Nonprofits Bridging Tech and Social Impact." This two-hour event brought together over 60 participants from academia and industry, showcasing the critical work of nonprofit technology organizations.

Common Crawl’s CTO Greg Lindahl presenting at NeurIPS 2024.

Common Crawl’s CTO Greg Lindahl presenting at NeurIPS 2024.

Presentations

  • An introduction to Wikimedia and Common Crawl, illuminating our respective missions
  • An exploration of Common Crawl's dataset quality and the complexities of web crawling presented by Greg Lindahl
  • Chris Petrillo provided a deep dive into Wikipedia's editing landscape, exploring community dynamics
  • An interactive Q&A session that sparked robust discussion

The event transitioned into roundtable discussions, also providing a unique networking opportunity. Participants from various backgrounds exchanged their ideas about AI, technology, and social impact.

Additional Conference Highlights

We were excited to support our colleague Professor Ludwig Schmidt, who delivered a highly effective tutorial titled "Advancing Data Selection for Foundation Models: From Heuristics to Principled Methods." His presentation explored critical approaches to data selection in foundation model training, discussing everything from algorithmic foundations to practical data curation techniques. The session delved into attribution-based approaches, diversity-based methods, and emerging strategies for optimizing model performance through intelligent data selection.

Prof. Ludwig Schmidt presenting his slide featuring his annotations on the celebrated xkcd 2347.

Prof. Ludwig Schmidt presenting his slide featuring his annotations on the celebrated xkcd 2347.

We were also lucky to meet Dr. Fei-Fei Li, a key industry leader often referred to as the "Godmother of AI", and co-founder of the Stanford HAI (Human-centered Artificial Intelligence) Department where she reiterated how critical Common Crawl’s work is in the industry. Dr. Li’s was also one of the Key Invited Talks at the conference and highlighted Common Crawl in Slide #104 of her presentation.

Dr. Fei-Fei Li presenting at NeurIPS 2024.

Dr. Fei-Fei Li presenting at NeurIPS 2024.

Team members participated in several standout events. Rich Skrenta and Greg Lindahl attended a dinner with MLCommons which was sponsored by Tola Capital. The evening featured an impressive lineup of speakers, including Lora Aroyo from Google, Sarah Hooker from Cohere, Rishi Bommasani from Stanford University, and Peter Mattson from MLCommons and Google.

Left-to-right: Sam Reddy, Stephen Burns, Wayne Yamamoto, Jason Grey, Greg Lindahl, Chris Tolles, Rich Skrenta

Looking Forward

The NeurIPS conference was a resounding success, strengthening our connections and highlighting Common Crawl’s role in the AI research community. We look forward to building on these partnerships and continuing to provide high-quality, open-access web data to support innovation in AI.