Cybersecurity educators have widely introduced hackathons to facilitate practical knowledge gaining in cybersecurity education. Introducing such events into cybersecurity courses can provide valuable learning experiences for students. The nature of the hackathon format encourages a learning-by-doing approach, and the hackathon outcomes can serve as evidence for students knowledge, capability and learning gains. Prior work on hackathons in education mainly focused on collocated hackathon events in the traditional classroom setting. These hackathon events often took place as a one-off event at the end of the course. However, one-off hackathon events at the end of a course might not be sufficient to improve learning. Instead, we focus on analyzing the integration of a series of online hackathon events into an online cybersecurity course and explore how this integration can address online education issues by encouraging collaboration and developing a practical understanding of the delivered course by solving real-world challenges. We evaluate interventions to foster learning and analyze its effect on collaboration and learning gains for students in the course. Our findings indicate that students attribute learning benefits to the introduced interventions that supported teamwork and collaboration, maintained student participation and interest in the course, and encouraged learning-by-doing.