How Databricks Genie Turns Plain English Into SQL Code

If you have spent time working inside a data team, you already know how a typical Tuesday looks.

A message comes in from the sales manager. Then one from finance. Then someone from the product team who just needs "a quick number." Before 10 AM, your backlog is three queries deep. None of them are complicated on their own. But together they eat up the hours you were planning to use on the pipeline work that actually needed you.

This is not a small problem. Research from Wren AI found that data analysts in fast-paced industries spend up to 50 to 70 percent of their time handling ad-hoc data requests. And as OWOX points out, each one-off request keeps analysts stuck in reactive mode instead of doing the forward-looking work that actually moves the business.

Databricks built AI/BI Genie to take a serious chunk of that workload off the data team. And based on how it works under the hood, it is worth understanding before you dismiss it as just another chatbot.

What Is Databricks Genie?

AI/BI Genie is a conversational analytics tool built directly into the Databricks platform. It became Generally Available in June 2025 and is free for all Databricks SQL customers with no extra license needed.

The idea is simple on the surface. A business user types a question in plain English. Genie writes the SQL, runs it, and returns a table of results along with a chart and a plain-language summary.

But what makes it different from the dozen other "ask your data a question" tools out there is what happens behind that simple interface.

How Genie Actually Works: The Compound AI System

Genie is not just one model reading your question and guessing. DataCamp's deep dive into the architecture describes it as a compound AI system, which means it uses a chain of specialized agents working together.

Here is the rough breakdown of what happens when someone asks a question:

An intent parsing agent figures out what the user is really asking, including the metric, the time range, the filters, and the aggregation type.
A planner agent breaks multi-step questions into an ordered execution plan.
A retriever agent finds the right tables, columns, and example queries to ground the request in your actual data.
A SQL generation agent turns the plan into a real, executable SQL query.
The query runs against your Databricks SQL warehouse.
A verifier checks the result. If something looks off, it can trigger a re-run or ask the user to clarify.
A summarizer writes a plain-language takeaway and picks the right visualization.

That is a lot of steps happening in seconds. And the reason this matters is that a simple single-model text-to-SQL approach fails a lot in production. Genie's multi-agent design is specifically built to reduce that failure rate.

Genie Spaces: Where the Real Setup Happens

The part most articles skip over is what makes Genie useful versus what makes it unreliable. That difference comes down to how well a Genie Space is configured.

According to the official Databricks documentation, a Genie Space is where a domain expert, such as a data analyst, sets up the context that Genie works from. This includes:

Which tables and views Genie can access
How business terms are defined ("active user" means X, "net revenue" means column Y)
Example queries that show Genie how to handle common question patterns
Text instructions for edge cases

This setup matters more than most people expect. Genie uses the names and descriptions from annotated tables and columns to convert natural language questions into equivalent SQL queries. If your column is named amt_net_rev_adj with no description, Genie will guess. If it is named adjusted_net_revenue and described clearly, Genie has the context it needs.

You can build different Genie Spaces for different teams. One for finance. One for sales. One for operations. Each one has its own tables, its own vocabulary, and its own guardrails. This keeps a sales rep from accidentally querying financial tables they should not see, and it keeps Genie focused on the questions that actually matter to each group.

Security and Governance Are Built In, Not Bolted On

One worry that comes up every time you let non-technical users query data directly is access control. What happens if someone asks a question that would return data they are not supposed to see?

Genie handles this through Unity Catalog, which is Databricks' governance layer. According to the Databricks Genie documentation, each user's own Unity Catalog data permissions are applied to the query results. Row filters and column masks are automatically enforced per user. If a user does not have SELECT access to a table, they will not see results from that table, even if they ask Genie a question that would normally involve it.

This is not a new access control layer you have to build. It extends the permissions your team already set up in Unity Catalog. That makes the conversation with your security and compliance teams a lot shorter.

Benchmarking: The Step Most Teams Skip

This is where a lot of Genie rollouts go wrong.

A team sets up a Genie Space, tries a few questions manually, gets answers that look right, and rolls it out to the business team. Then an executive asks something the space was not tested on, gets a weird result, and suddenly nobody trusts Genie anymore.

The Databricks team is direct about this: any AI effort should start with an evaluation phase. Failure to do so means failure in production.

Genie has a built-in benchmarking tool for exactly this reason. You write a list of test questions that represent the real questions users will ask. You add the correct SQL answer for each one. Genie runs its own queries and compares the results to yours.

According to Databricks' production readiness guide, the typical expectation is that Genie benchmarks should be above 80 percent accuracy before you move on to user acceptance testing. They also recommend adding two to four different phrasings of the same question, because users will not always ask the same question the same way.

There is also an "Ask for Review" feature. If a user gets an answer they are not sure about, they can flag it. A space admin gets notified, reviews the SQL, and corrects it if needed. The user gets notified once the answer is verified. This feedback loop is how Genie gets better over time instead of drifting.

The October 2025 release notes also added a "Knowledge Extraction" feature. When a user gives a thumbs up to a generated query, Genie analyzes that interaction and proposes knowledge snippets such as metric definitions or filter patterns that the space admin can approve and add to the knowledge store.

That is a real improvement over tools that treat every question as if it is the first one.

What Good SQL Schema Documentation Does for Genie

This is worth its own section because it surprises a lot of engineers.

When you first set up a Genie Space, you will quickly discover that the quality of Genie's answers is almost entirely dependent on how well your tables and columns are documented. This is not a new idea. Good data teams have always known that schema documentation matters. Genie just makes that documentation pay off in a way that is immediately visible to everyone, not just other engineers.

Here is a practical example from the Databricks benchmarking blog. One team wanted Genie to calculate the "best sales rep in Asia." Genie kept failing that question. The fix was not a model update. It was adding a single example SQL query to the instructions page showing exactly how to calculate that metric. After that, Genie answered it correctly every time.

That is the pattern you will see over and over. The fix is almost never "change the model." It is "give Genie more context about what the question actually means."

Genie Code: Writing Dashboards With Natural Language

One feature that deserves more attention is Genie Code.

When you create an AI/BI Dashboard in Databricks, it automatically creates a companion Genie Space. But Genie Code goes a step further. It lets you write and edit the actual SQL and Python cells in your dashboard notebooks using natural language prompts.

Instead of writing a complex window function from scratch, you describe what you want in plain English and Genie writes the code. You review it, tweak it if needed, and move on. This is especially useful for analysts who know what they want but do not always remember the exact SQL syntax for a specific aggregation or join pattern.

This is part of the same thinking that drives tools like GitHub Copilot, but scoped specifically to the Databricks analytics environment with all the governance context already built in.

Who Benefits and How

The next-generation Genie announcement points to something real in how teams are using this. Customers created over 1.5 million Genie Spaces in 2026 alone. That adoption happened because different roles found different value in the same tool.

Business analysts and managers stop waiting. A question that used to take two days to get answered from the data team now takes thirty seconds. This is the most visible benefit, and it is the one that gets internal champions bought in.

Data engineers get time back. As Sigma Computing writes, the BI bottleneck is not just stressful, it also delays decisions that need to be made quickly. When business users can self-serve the common questions, data engineers can stay focused on the work that actually requires an engineer.

Data analysts turn their existing knowledge into a reusable asset. They set up the Genie Space once, document it well, add example queries, and the business team can self-serve on top of that work without sending messages every time.

Executives get faster decisions. Questions that need a quick answer before a meeting get an answer before the meeting.

Embedding Genie Outside of Databricks

One of the more practical things in the latest release is that Genie does not have to live only inside the Databricks workspace.

Using the Genie Conversation APIs, developers can embed Genie into Slack, Microsoft Teams, or custom internal applications. A sales team that never opens Databricks can ask questions directly from Slack and get back a chart and a summary without leaving the tool they already work in.

The latest version of Genie also connects to enterprise knowledge sources like Google Drive and SharePoint, according to the next-gen Genie release post. This means Genie can now blend structured data from your Delta tables with unstructured content from documents to answer questions that used to require a human to piece together.

How This Connects to Broader AI Agent Work on Databricks

Genie is a great starting point, but it is part of a larger picture on the Databricks platform.

Once teams get comfortable with Genie handling their self-serve analytics layer, the next question that usually comes up is: what about workflows that go beyond answering questions? What about agents that can take action, run multi-step reasoning tasks, or be deployed as part of a production application?

That is where the Mosaic AI Agent Framework comes in. If you are thinking ahead to that kind of work, it is worth reading about how Mosaic AI handles evaluation, governance, and production deployment for AI agents on Databricks. The evaluation mindset is the same. The MLflow tracing and Unity Catalog governance carry over. But the scope is broader.

What You Need to Make Genie Work in Production

To be direct: setting up Genie is easy. Getting it to work well in production takes real work.

Here is what consistently makes the difference:

Clean, well-described tables. Column names and descriptions need to match how your business teams actually talk. If marketing calls something "activation rate" and your table calls it usr_actv_rt_wk, Genie will have trouble making that connection.

Real example queries. The example queries in a Genie Space teach Genie how to handle your organization's specific metric logic. The more representative they are, the better Genie handles questions it has never seen before.

A benchmark set before launch. According to Databricks' own best practices, most Genie Spaces should reach above 80 percent benchmark accuracy before they go to user testing. That bar exists for a reason. Missing it means users lose trust quickly and it is hard to rebuild.

Someone who owns the space long term. Genie Spaces need a person responsible for reviewing flagged responses, updating example queries as data changes, and approving knowledge snippets from user feedback. Without that owner, quality drifts.

Proper Unity Catalog setup. If your tables are not already in Unity Catalog with access controls in place, that needs to happen first. Genie's governance layer depends on it.

A lot of teams underestimate how much foundational data engineering work feeds into a good Genie rollout. If your team is already stretched thin on that infrastructure layer, it can make sense to bring in specialized help. That is why some teams choose to hire experienced data engineers who already understand how the Databricks ecosystem fits together, rather than trying to figure it out while also building the Genie Space.

Where to Start

If you already have a Databricks SQL workspace, you can create a Genie Space today. No extra license. No new tool to install.

Start small. Pick one team, one topic, and a focused set of tables. Write clear column descriptions. Add ten to fifteen example queries that cover the most common patterns. Build a benchmark test set before you open it to users. Then release it to a small group and watch what they ask.

The questions that Genie cannot answer well are your roadmap for improving the space. That feedback loop, questions, failures, fixes, is how good Genie Spaces are built over time. It is the same loop that any good data product depends on. Genie just makes each iteration faster and more visible.

Final Thought

Genie is not magic. It is a well-engineered system that works best when the data behind it is clean, documented, and governed correctly.

The teams that get the most out of it are the ones that treat the Genie Space setup like they treat any other production data product. That means documentation, testing, ownership, and a willingness to iterate based on real user feedback.

That is not a high bar. It is the same bar good data teams already hold themselves to. Genie just gives them a way to deliver the output of that work directly to the people who need it, without requiring a SQL ticket for every question.

Have you set up a Genie Space yet? What was the hardest part of the setup? Drop a comment. Real-world experience from different environments is always useful.

Sources Referenced

推荐订阅源

DEV Community