When I started working on my usability study project with KServe, I interacted with KServe users to understand the challenges they were experiencing while using the platform. During these conversations, users frequently mentioned GitHub issues and the problems they encountered during development and deployment.

This led me to explore how engineers solve problems by using GitHub repositories and collaborating in open-source communities. As part of my research, I started reading articles and academic research papers online, and I came to know this is the best approach and many organizations are already following this methods to identify the developer problems.

GitHub mining (also called repository mining, issue mining, or software repository mining) is increasingly used as a user research and UX research method in open-source ecosystems because it allows researchers to understand user challenges from real-world interactions rather than only interviews or surveys.

Here is the research paper links: 1. Mining Developer Behavior Across GitHub and Stack Overflow (Xiong et al., 2017)., 2. Understanding Java Usability by Mining GitHub Repositories (Lemay, 2018), 3. Insights from GitHub Community on the Matter Standard: Developer Perspectives and Challenges (Hassan, 2026).

I began exploring the GitHub mining process alongside conducting 1:1 usability study sessions with users. Through this research, I learned a great deal about how developers work, collaborate, report issues, and solve problems in open-source communities.

In this article, I am going to explain:

• What the GitHub mining process in user research is?
• Why it is important to conduct this type of research?
• How it helps identify developer pain points?
• How the findings can support communities and developers in improving the overall developer experience (DX) and usability of open-source tools like K-Serve.

Let's start...

What is GitHub Mining in User Research?

GitHub mining is a research method that systematically collects and analyzes data from GitHub repositories—such as issues, pull requests (PRs), discussions, comments, commits, documentation changes, and feature requests—to understand how people use software and where they experience problems.

From a UX perspective, GitHub becomes a large archive of user feedback and user behavior.

Instead of asking users directly:“What usability problems do you face?”

researchers examine: Bug reports, Questions, Configuration failures, Feature requests, Support discussions, Workarounds, Documentation complaints, Delayed PR discussions, Community conversations, These become signals of user experience.

Why is GitHub Mining Conducted?

GitHub mining is conducted because traditional UX methods alone do not always reveal the full picture. Open-source projects often have: thousands of users, globally distributed contributors, limited access to end users, asynchronous communication. GitHub provides a historical record of actual user struggles.

Researchers conduct GitHub mining to: Discover usability issues at scale, Understand real user behavior, Study product evolution over time, Generate evidence-based design decisions

1. Discover usability issues at scale:

Researchers conduct GitHub mining because it allows them to study user experiences and product challenges at a much larger scale than traditional research methods. Instead of interviewing a small group of users—such as 10 participants in interviews or usability sessions—researchers can analyze hundreds or even thousands of GitHub issues, pull requests, discussions, and feature requests to identify recurring patterns across an entire user community.

For example, researchers may discover that among all reported issues, 300 are related to deployment problems, 100 focus on observability complaints, and 50 request better documentation. Looking at this data collectively helps reveal trends that individual interviews may not uncover. This large-scale approach makes it easier to identify which problems occur most frequently and which areas of the product create the greatest friction for users.

2. Understand real user behavior:

Another important reason researchers use GitHub mining is to understand real user behavior rather than relying only on assumptions or controlled testing environments. When users open issues on GitHub, they often describe what they were trying to achieve, where they became stuck, what configuration mistakes they made, which expectations were unmet, and which onboarding barriers prevented them from completing tasks successfully.

These issue discussions provide a detailed record of actual workflows and real-world usage conditions. For example, a developer deploying a tool may explain that installation instructions were unclear, required settings were missing, or error messages did not provide enough guidance. By studying these interactions, researchers gain insight into how users actually experience a product rather than how designers assume they use it.

3. Study product evolution over time:

GitHub mining also helps researchers study product evolution over time. Because GitHub maintains historical records of issues, discussions, fixes, and releases, researchers can observe when specific problems first appeared, how long they remained unresolved, and whether implemented solutions reduced future complaints.

This longitudinal view helps teams evaluate whether design improvements, documentation updates, or technical changes created measurable improvements in usability and developer experience. For example, researchers can compare issue frequency before and after a deployment redesign to determine whether users encountered fewer deployment failures after the change.

4. Generate evidence-based design decisions:

Finally, GitHub mining supports evidence-based design decisions. Instead of making design changes based on assumptions or subjective opinions. for example, saying, “Users seem confused during deployment”—researchers can present measurable findings supported by real community data.

They may conclude that “37% of reported issues involve deployment discoverability,” which provides a stronger foundation for prioritizing product improvements. This evidence helps UX researchers, product teams, engineers, and open-source maintainers make informed decisions about where to invest effort, improve usability, reduce developer friction, and create better experiences for the broader community.

When Should GitHub Mining Be Conducted?

GitHub mining is valuable during multiple stages.

GitHub mining can be conducted at different stages of the research and product development lifecycle, with each stage serving a specific purpose in improving usability and user experience. During the discovery phase, researchers use GitHub data to identify user pain points by analyzing issues, discussions, and feature requests. This helps teams understand the most common challenges users face before making design or development decisions.

Before conducting surveys or interviews, GitHub mining can be used to generate hypotheses. Instead of starting research with assumptions, researchers review historical issue data to identify patterns and form evidence-based questions. For example, if many users report deployment difficulties, researchers may design interview questions specifically around deployment workflows and onboarding experiences.

During a product redesign, GitHub mining helps teams validate recurring issues and confirm whether previously identified problems continue to affect users. This ensures redesign efforts focus on meaningful improvements rather than isolated opinions. By reviewing historical and current issue reports, teams can prioritize areas that consistently create friction.

GitHub mining is also valuable for continuous UX monitoring, where researchers regularly track issue trends to measure overall usability health. Ongoing analysis allows teams to detect emerging problems early, observe changes in user sentiment, and monitor whether user experience improves or declines over time.

After a product update or feature launch, GitHub mining supports post-release evaluation by helping teams measure the impact of changes. Researchers can compare issue volume and themes before and after release to determine whether updates reduced complaints, improved workflows, or introduced new challenges.

Finally, GitHub mining is especially useful in longitudinal studies, where researchers analyze data across extended periods to understand trends over time. This allows teams to observe how user needs evolve, how products mature, and whether long-term improvements lead to sustained reductions in usability issues.

Through these stages, GitHub mining becomes a continuous source of evidence that supports data-driven decisions and helps create better experiences for users and developer communities.

How Does the Community and Developers Benefit?

GitHub mining provides several important benefits for developers because it helps teams make decisions based on actual user experiences rather than assumptions.

One of the major advantages is better prioritization. By analyzing issue reports, discussions, and recurring complaints, developers can identify which problems affect the largest number of users and focus their efforts on fixing the areas that create the most difficulty. Instead of allocating time based only on internal opinions, teams can prioritize improvements that deliver the greatest value to the community.

Another important benefit is reduced support burden. When developers repeatedly analyze and address the root causes of commonly reported issues, users encounter fewer recurring problems and require less direct support. Over time, this reduces duplicate issue reports, lowers maintenance effort, and allows development teams to spend more time building new features rather than responding to the same questions repeatedly.

GitHub mining also contributes to improved onboarding for new users and contributors. By identifying patterns in issue reports related to installation challenges, setup confusion, missing documentation, or early-stage errors, teams can improve guidance materials and simplify user workflows. These improvements create a lower learning curve, helping users become productive more quickly and reducing frustration during initial adoption.

Finally, GitHub mining supports data-driven decisions across product planning and development. Rather than creating roadmaps based on assumptions about what users might need, teams can use measurable evidence from issue trends and community feedback to guide future work. This makes product roadmaps more strategic, more transparent, and more aligned with actual user needs, ultimately leading to stronger developer experience and more effective product evolution.

Benefits for Open Source Communities

GitHub mining and usability research provide several important benefits for open-source communities by helping projects become more accessible, sustainable, and user-centered.

One key benefit is creating a healthier contributor experience, where new contributors can understand project workflows, contribution processes, and technical expectations more quickly, reducing barriers to participation. These improvements also support higher adoption, because better usability makes projects easier to learn and use, attracting more users and contributors over time.

In addition, insights gathered from user issues and discussions lead to better documentation, allowing communities to create more targeted and practical guidance that addresses real user challenges instead of assumed needs. Together, these improvements contribute to increased retention, as users and contributors are more likely to remain active in a project when frustration decreases and their experience becomes smoother and more productive.

Benefits for Product Teams

GitHub mining and usability research provide valuable advantages for product teams by creating a stronger connection between user feedback and product decisions. One major benefit is providing evidence for redesign, allowing teams to make improvements based on actual user-reported issues instead of assumptions or internal opinions.

This research also helps establish usability benchmarks, enabling teams to measure and compare user experience over time and determine whether product changes are producing meaningful improvements.

In addition, GitHub data creates a continuous feedback loop, where ongoing issue reports, discussions, and community input help teams identify emerging problems and continuously refine the product experience. Finally, it supports release quality monitoring by allowing teams to evaluate how updates perform after launch, detect new usability concerns early, and measure whether releases successfully reduce user friction and improve overall product quality.

Are UX Researchers and Organizations Adopting This Method?

UX researchers and organizations are increasingly adopting repository mining and GitHub mining as research methods, especially as software development becomes more collaborative, distributed, and community-driven. Traditional research approaches such as interviews and usability testing remain valuable, but researchers now complement them with large-scale behavioral data collected from repositories to better understand how people actually use and contribute to software products.

This approach is becoming common across areas such as Open-Source UX Research, Developer Experience (DevEx), Human–Computer Interaction (HCI), Software Engineering Research, Empirical Software Studies, and AI/ML Operations Usability, where understanding real-world workflows and developer challenges is essential.

Rather than relying on a single research method, many research communities and organizations use a mixed-method approach that combines repository mining, interviews, surveys, ethnographic observation, and telemetry data. Combining multiple methods improves research validity because findings can be verified from different perspectives—what users say, what users do, and what product data shows.

Through this approach, researchers aim to understand developer pain points, improve onboarding experiences, measure usability debt (the accumulated usability problems that slow users down), and support evidence-based product decisions. As a result, organizations can design products that better reflect actual user needs, strengthen contributor experiences, and continuously improve overall usability and developer experience.

Why This Method Is Growing in UX Research?

GitHub mining is becoming increasingly important in UX research because modern digital products extend beyond traditional consumer applications and now include complex technical environments such as cloud platforms, Kubernetes ecosystems, AI platforms, and DevOps tools.

Traditional UX research has often focused on studying end users through interviews, surveys, and usability testing. However, modern software systems require researchers to also study developers as users, since developers interact directly with interfaces, documentation, workflows, configuration systems, and deployment processes as part of their daily work.

GitHub mining supports this shift by allowing researchers to observe user experience through engineering artifacts—such as issue reports, discussions, pull requests, feature requests, and contribution patterns—rather than relying only on questionnaires or self-reported feedback. These artifacts provide evidence of where users become confused, what tasks create friction, which expectations are unmet, and how workflows perform in real environments.

A strong research framing would be: “GitHub issues are not only defect reports; they represent observable traces of user experience and can be analyzed as usability evidence.”

This perspective aligns closely with modern practices in UX research, Human–Computer Interaction (HCI), and Developer Experience (DevEx) studies, where understanding real user behavior and evidence-based decision-making has become increasingly important.

Conclusion:

Through my K-Serve usability research, I learned that understanding developer experience requires looking beyond traditional interviews and usability testing. By combining 1:1 user sessions with GitHub mining, I was able to observe real-world developer challenges through issues, discussions, pull requests, and community collaboration.

This approach showed that GitHub repositories are not only places where technical problems are reported—they also contain valuable evidence of user experience, usability barriers, and product friction. GitHub mining helps researchers, product teams, and open-source communities make more informed, evidence-based decisions that improve usability, reduce developer pain points, strengthen onboarding, and create better developer experiences (DX).

As modern software ecosystems continue to grow in complexity, repository mining is becoming an increasingly valuable method for understanding how people truly work, collaborate, and build in open-source environments.