Everything about AI is evolving so rapidly: what it can do, how users interact with it, and the skill set required (or not required) to use it. AI prompt engineering is no exception.
Two years ago, we thought AI prompting would be the defining new skill for data analysts and engineers. Companies were opening job requisitions for AI prompters left and right, and LinkedIn postings with “prompt engineering” surged by 400%+. Two years ago, that made sense for AI’s maturity. Most models were either too literal, doing exactly what you asked–nothing more, nothing less—or too liberal, overinterpreting your prompts. As a result, we really thought AI prompting was going to be its own industry.
However, today’s LLMs are so. much. better. AI has gotten really good at understanding your intent, reading between the lines, and doing what you want it to. Nonetheless, it still can’t read your mind and doesn’t (yet) have all the business context or institutional knowledge of a seasoned human analyst. While the new AI is certainly more user friendly, your success with it still depends heavily on the quality of your prompts.
Let’s explore three best practices for prompting in Amplitude Global Agent. For now, they’ll set you up for the most success with your agentic analyses.
Every company coins its own internal language from acronyms, kitchy code names, and jargon. New employees regularly jot down words in the margins to ask someone about them after the meeting. “Um, what are QBC Plus Delta, PRD V3 Final, and Project Phoenix, again?”
It takes AI time to get up to speed on your organizational lingo, too, learning the language as data analysts define it. That’s why, even though we’ve trained Amplitude AI Agents to think like an analyst, we recommend initially treating Global Agents more like an intern than someone who’s worked at your company for years.
For example, I might refer to “self-serve” customers when talking to a seasoned Amplitude colleague, and they’ll know that I’m referring to customers on our Starter Plan. But if I ask AI to show me 30-day retention for self-serve customers, it wouldn’t intuitively know which segment of customers I’m referring to. Without that context, it would likely return an incorrect or incomplete answer.
AI knows your taxonomy and your definitions. Your data admins can define your internal lingo for AI to reference, but as a user, it’s always better to err on the side of caution until it builds a deeper understanding of your business. The good news is that the Global Agent can remember context from your conversations, learn from feedback, and apply it to future queries so you don’t have to re-explain context every time. Nonetheless, for the best results, treat your AI as graciously as possible to start. Assume it doesn’t know your secret language and use defined, “official” language in your prompts.
TLDR: Think of AI as a useful member of the team that can access all your internal data, but still needs some help understanding context about your business before it can interpret anything.
Most of us want AI’s help answering big, difficult questions. We can answer the easy ones ourselves, glancing at a simple dashboard, but more nuanced questions take more time, digging, filtering, slicing, and dicing. Answering complex questions involves long workflows with numerous steps and stages. Although AI can get you the answer (much faster than you could on your own!), each one of those steps introduces room for error if AI misunderstands your request at any point along the way.
Consider the steps involved in answering something like “Why did our conversion dip in September?”
It’s easy to think that AI can orchestrate these steps in perfect order, but if it makes a mistake during any of these steps, you won’t get the right answer. For example, maybe you define conversion as a completed purchase within seven days of signup, while AI treats any purchase as a conversion regardless of timing. A small error so early in the analysis will compound into larger downstream errors, since the steps all build on each other.
The best way to mitigate this is to break down the steps required to answer your question and engage with the agent after each one. This enables you to insert yourself into the process and ensure the process never gets too far away from the intended goal.
So, instead of asking “Why did our conversion dip in September?” you could:
You could also try framing the conversation differently in your first prompt: “Tell me why our conversion dipped in September. Stop at every stage of your analysis to ask for my verification before proceeding to the next step.”
This approach enables you to work iteratively and collaboratively with AI agents to ensure that you have a perfect answer at the end. Verifying the facts at each step before performing extra synthesis avoids frustration and gives you more confidence in the answer. Over time, you’ll learn which questions AI can and can’t tackle without as much supervision.
TLDR: Don’t ask AI for a big answer right out of the gate and expect a miracle answer. Ask it to take several small steps in a row and build human checks into your analysis process. Remember, the bigger the leap you ask for, the more room for error there is.
Good data inquiries require three elements: the right data, the right time frame, and the right user segmentation. These three components are second nature to veteran users of Amplitude dashboards and charts. AI agent users generally do a good job of including the right data and time, but it’s easy to forget the segmentation.
In the absence of further instruction, Global Agent will typically default to “all users.” However, most product managers and marketers aren’t typically looking for that broad of an audience. They’re typically looking for a subset by region, industry, product line, etc., and they might inadvertently bring existing mindsets or assumptions to the table that could result in misinterpreting AI’s answers.
Imagine you're a marketing manager for the Americas. You always look at Americas data, and all your charts and dashboards are filtered to that region. You ask AI, “What’s our average order size?” and it tells you 13. AI’s answer applies to all orders, but your brain is still in the Americas. Your usual assumptions have gotten in the way of your communication with the agent. So you walk away thinking that the Americas’ average order size is 13, when it’s actually 24.
The easiest way to avoid this is to always specify segmentation in your question, which is easy to do. Instead of asking “How were user signups trending last month?” just ask “How were user signups trending in EMEA last month?”
TLDR: Every AI prompt should specify the data set, time frame, and segment you care about. Nailing those three removes a lot of room for error or misinterpretation.
With AI, you get what you give. Give it jargon-free, complete prompts, and you’ll get accurate answers to bet your business on. Writing prompts isn’t rocket science, especially when you’re working with Global Agent.
Be conscious of jargon, check AI’s answers along the way, and specify your segment. Do these three things and you’ll be on your way to unlocking deeper insights and moving the business outcomes that matter.
For more AI analytics best practices, check out these blog articles.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。