On May 10th, over 100 engineers and researchers from across the AI ecosystem gathered at AGI House in Hillsborough, CA for our very first hackathon: a fast-paced day of hacking, learning, and building with Mojo. The Modular GPU Kernel Hackathon brought together developers of all experience levels to experiment with Mojo on modern GPU hardware, collaborate in person, and accelerate the future of high-performance AI infrastructure.
Participants tackled a wide range of problems, from low-level kernel implementations to full model training frameworks. Many participants had no Mojo or GPU programming experience. By the end of the day, dozens of teams had working prototypes and new insights into what Mojo can do.
We're thrilled by what this community accomplished in just a single day!
Before the hacking began, we were lucky to hear from an all-star lineup of speakers:
Team: Marcel Roed (PhD Student at Stanford University), Herman Brunborg (PhD Student at Stanford University), and Rajat Vadiraj Dwaraknath (PhD Student at Stanford University)
Marcel and the team tackled one of the most ambitious challenges of the hackathon. They built a training framework in Mojo/MAX and implemented the kernels and backward passes needed to train a Transformer model from scratch. This required implementing backpropagation in MAX, along with gradient descent algorithms like AdamW, as well as figuring out how to work with FlashAttention in the MAX kernel library and implementing its derivative to perform gradient training. Although they didn’t have time to fully implement FlashAttention during the hackathon and used standard scaled dot product attention instead, they’ve continued working on the project and plan to complete the implementation soon.
“It often makes sense to build something that works rather than trying to make something fast immediately… It was super fun to debug in real-time and discuss errors and solutions with the Modular employees as we ran into problems as we were trying to use Mojo in ways not previously explored by the Modular team.” - Herman
“It was difficult to get our kernels to be correct, and figuring out how to use the FlashAttention implementations... was quite challenging. But we showed that we can build useful SoTA-level tools from bare-bones with Mojo in a short period of time.” - Marcel
Team: Kirill Bobyrev (Software Engineer at Waymo)
Kirill set out to implement the optimal parallel prefix scan algorithm for high-performance GPUs—and in the process, helped improve Mojo’s standard library. His contributions included new, corrected implementations of warp- and block-level scans, and an efficient, tunable full device-wide scan. These updates are now part of Mojo’s open-source repo.
“GPU programming is notoriously challenging, but Mojo makes it surprisingly pleasant. The language feels modern, its templates and meta-programming features enable rapid experimentation, and the code is highly readable—akin to Rust's standard library, a clear contrast to the often cumbersome C++ standard library.” - Kirill
Even more impressively, Kirill had never written a line of CUDA or Mojo before the week of the hackathon.
Team: Sandeep Menon (Software Engineer, Deep Learning at Kodiak) and Owen Leather (Perception Software Engineer at Kodiak)
Gaussian splatting is a rendering technique that, until now, had only been implemented in CUDA. Sandeep’s team aimed to break new ground by porting this kernel to Mojo so it could run on GPUs like AMD’s MI300X. While they weren’t able to fully finish the implementation by the end of the event, they made strong headway and are continuing to build on it.
“Learning Mojo and meeting the amazing Modular team was a highlight. It is humbling to code in the era of AI programming. I would love to be invited to future events and join many more hackathons as a way to kickstart learning on new topics.” - Sandeep
Beyond our winners, teams dove into a wide range of challenges using Mojo:
Many participants wrote Mojo or GPU code for the first time and were eager to build on their projects in the coming weeks. We’re incredibly proud of everyone who participated in the hackathon. Whether you were building compilers, training frameworks, or GPU kernels from scratch, your work is what makes this community so exciting!
Thank you again to our sponsors, AMD, Crusoe, and GPU MODE, for making the event possible. And if you weren’t able to join us in person, be sure to catch the talk recordings on the Modular YouTube channel.
Until next time, keep building!
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。