Across Scottish Water’s Capital Investment (CI) programme, teams need fast answers about project status, financial performance, delivery milestones, and risks. That information already existed, but finding it often meant navigating a large estate of reports or relying on data specialists to extract the right information from underlying tables. Databricks Genie changed that. Today, project teams can ask questions in plain English directly in Microsoft Teams and receive trustworthy answers in seconds.
The challenge: data was available, but not accessible
Scottish Water’s CI function did not have a data shortage. It had an access problem.
In practice, that showed up in several ways:
A large volume of reports already existed, but limited awareness and visibility often led to duplicated efforts, with new reports being created to answer questions that existing reporting had already answered.
Valuable data remained difficult for non-technical users to access, slowing down decision-making across the function.
The result was predictable: analysts spent time reproducing work that already existed, delivery teams waited for figures to be extracted, and important project data did not always reach the people responsible for making day-to-day decisions.
The solution: a conversational interface for governed project data
SPARK is Scottish Water’s internal brand for the natural-language interface to its project portfolio data, built on Genie.
Instead of searching for the right report, users can ask a question and receive an answer grounded in governed data. SPARK brings this experience directly into Microsoft Teams, where teams already work, removing the need to shift to a new interface.
Users submit a question in Teams via Copilot, which is then orchestrated by a Copilot supervisor agent, which connects to the Databricks Genie Space via the Model Context Protocol (MCP). Genie translates the question into a query, runs it against governed data in Unity Catalog, and returns the result back to the user.
Figure 1: High-level architecture showing how Microsoft Teams, Genie, the semantic layer, and curated data layers work together under governance and monitoring.
From report hunting to direct questions
With SPARK, teams can ask practical business questions such as:
List all open project risks that are expiring in August, including project name, risk description, risk owner, and risk expiry date.
What is the current live risk score for project X?
For project X, what is the risk with the highest current risk exposure? Please provide the risk name, description, and current exposure value.
Who is the future contractor for project X?
Each question returns an immediate answer derived from the governed metric views. That means less time spent locating reports and more time spent acting on insight.
The impact: faster access, less friction, broader use of data
By making data access conversational, Scottish Water is helping project teams reach answers faster and with far less effort.
For project data, a typical lookup that previously took around 8 clicks plus dashboard load time can now start with a single question in Teams.
For report-based questions, users no longer need to navigate through SharePoint, the Reporting Hub, report categories, and individual report links just to find the right asset. In many cases, that turns a 4-step to 5-step search journey into a direct question-and-answer experience.
If 100 users ask just 3 questions per week, that is roughly 300 information requests each week. At a conservative saving of 2 to 5 minutes per request, that equates to about 10 to 25 hours saved per week, or roughly 520 to 1,300 hours per year.
The benefit is not just speed. Teams get answers tailored to the question they are asking, rather than having to interpret static reports designed for broad audiences.
It also reduces dependence on specialist support and makes governed insight more accessible to non-technical users.
In practice, that means less time spent hunting through tools, less waiting for someone else to extract the data, and more time acting on trusted answers in the flow of work.
SPARK is going to completely change how our portfolio and project teams interact with data. It moves us from static reports to real-time, intelligent conversations with our information, empowering our people to make quicker, better-informed decisions and unlocking value we simply couldn't reach before. It's genuinely exciting!—Allan Mason, Programme and Project Delivery Manager, Business Analytics
Built for trust and scale
Conversational analytics only work when users trust the answers. Scottish Water’s implementation was designed with that in mind from the start.
Governance by default
Governance was built in from the start, with the Genie experience grounded in governed Unity Catalog data and shared semantic definitions so that answers would be consistent, explainable, and aligned with existing business logic.
The solution is built on governed Unity Catalog data, which makes access control, lineage, and a single source of truth part of the foundation.
Rather than exposing Genie directly to a large set of raw tables, Scottish Water first curated the data needed for this use case into the gold layer.
On top of that, a semantic layer was built using metric views that standardises measures, dimensions, and business terminology so the same definitions are reused consistently. This helped in reducing ambiguity and improving consistency in answers.
Tuned for accuracy
To make the Genie space reliable in practice, Scottish Water configured it around its own business rules, terminology, and real user questions rather than relying on a generic configuration.
Curated business-rule instructions were added so Genie could interpret Scottish Water conventions correctly, such as how fiscal periods are defined, how milestone gates are ordered, and how project IDs and dates should be handled.
Worked example questions and matching SQL were used to teach the assistant how Scottish Water users phrase questions and how those should map to the right query patterns.
A benchmark suite was used to provide a repeatable way to test accuracy whenever the space definition or underlying data changed.
User testing with Scottish Water teams helped validate real-world phrasing and fed additional refinements back into the space over time.
Ongoing monitoring
To support the solution in production, Scottish Water built monitoring around it so the team could track adoption, answer quality, and performance over time.
During the development phase, answer quality was monitored through user feedback, which helped identify responses that needed further improvement.
In production, adoption is monitored through measures such as conversation duration and conversations per user.
The team reviews recurring questions to understand what users are asking most often and if visualizations can be built around repeatedly asked questions.
Query performance is tracked through execution times, total query volume, and the slowest-running queries.
The team also built a visualization to track Genie cost per user.
Repeatable, reliable delivery
Scottish Water also designed the solution so it could be promoted safely and consistently across environments, rather than treated as a one-off build.
The solution is packaged as environment-parameterised configuration using Databricks Asset Bundles, so the same definition can be deployed across environments.
Separate development, test, and production environments are used, each with its own workspace and SQL warehouse.
Deployments run through Azure DevOps, with automatic deployment on change and a manual approval gate before production.
Authentication is handled through a Microsoft Entra ID service principal, with credentials retrieved at deploy time from Azure Key Vault rather than stored in code.
Deployments are idempotent, so the Genie space can be created or updated in place and access granted automatically to the appropriate groups.
Conclusion: A simpler and efficient way for teams to work with data
The result is straightforward but powerful: Scottish Water’s teams can access project data through natural language and trust the answers they receive. For leaders, that means faster visibility and less reporting friction. For delivery teams and data users, it means direct access to governed insight in the flow of work.
SPARK shows that when well-designed, well-governed data is combined with Genie, organisations can build a game-changing tool that truly democratises trusted business data for their users.
