COVID-19, a pandemic that the world has not seen in decades, has resulted in presenting a multitude of unprecedented challenges for student learning across the globe. The global surge in COVID-19 cases resulted in several schools, colleges, and universities closing in 2020 in almost all parts of the world and switching to online or remote learning, which has impacted student learning in different ways. This has resulted in both educators and students spending more time on the internet than ever before, which may be broadly summarized as both these groups investigating, learning, and familiarizing themselves with information, tools, applications, and frameworks to adapt to online learning. This paper takes an explorative approach to further investigate and analyze the impact of COVID-19 on such web behavior data related to online learning to interpret the associated interests, challenges, and needs. The study specifically focused on investigating Google Search-based web behavior data as Google is the most popular search engine globally. The impact of COVID-19 related to online learning-based web behavior on Google was studied for the top 20 worst affected countries in terms of the total number of COVID-19 cases, and the findings have been published as an open-access dataset. Furthermore, to interpret the trends in web behavior data related to online learning, the paper discusses a case study in terms of the impact of COVID-19 on the education system of one of these countries.