Putting Technology in Search of a Problem

With competitors launching AI, investor pressure on strategy and the same headlines circulating internally, the temptation is to add AI regardless of whether it makes sense. It's just something you have to do. That puts technology in search of a problem, and the result is almost always a feature that performs well in a demo but doesn't survive contact with users.

People try it once, decide it doesn't help them do anything they couldn't already do, and go back to whatever they were using before.

Some Problems Have Only One Correct Answer

AI is good at handling ambiguity, and if you give it a messy, open-ended question, it’ll often synthesize something useful. However, some problems have just one correct answer.

For example, if you ask an AI chatbot what your monthly mortgage payment would be on a $400,000 loan at 6.8% interest over 30 years, it might give you a number that sounds right, but it might also be off by $70, because the model isn't computing the answer, but approximating one. For a user about to make a major financial decision, that's not ideal.

In fact, in the third iteration of our ORCA Benchmark (Omni Research on Calculation in AI), which puts leading AI models through a mathematical and logical gauntlet, the best-performing free model (Grok 4.20) scored 70.4% accuracy on math tasks, while Claude and ChatGPT landed at 53.2% and 48.4%, respectively. What makes those numbers worse is what we call the Instability Metric: how often a model reverses a correct answer when you ask, "Are you sure?" For Claude and ChatGPT, that rate currently sits between 60% and 65%, meaning they're fundamentally uncertain about their own logic on most math problems they attempt.

Therefore, for problems with just one correct answer, deterministic tools (e.g., online calculators) are a better choice than AI because they produce the same output every time.

Some Problems Are Too Simple For AI

If users encounter something rare, the overhead of an AI feature, including its cost and ongoing maintenance as models change, is hardly worth it.

One example that comes to mind is onboarding. Some products have added AI chatbots to walk users through setup, because AI can answer follow-up questions in natural language. However, if your onboarding flow is simple, the chatbot will likely sit there unused.

Before committing to a feature like this, I'd ask two questions: How often do users actually get stuck here, and how varied are the problems they run into? If support tickets are low and the questions are repetitive, a FAQ or a cleaner UI will do the job at a fraction of the cost. If the questions are varied and crop up often, that's a stronger case for AI.

A Purpose-Built Tool Already Exists

If a purpose-built tool already exists for what you're trying to achieve, use it.

Google Maps is better at navigation than any AI, a weather API is more reliable than a chatbot for forecasts and a spreadsheet formula for compound interest will outperform AI by being right every time. These tools were built to do one thing perfectly, whereas AI does many things adequately.

There’s a version of this that works, though. Wrapping AI around a purpose-built tool can add value when the bottleneck is the interface, rather than the underlying calculation. If users struggle to know which tool to use, a natural language layer that interprets the request and routes it to the right tool can genuinely help (think Claude connectors).

3 Questions To Ask Before Adding An AI Feature

Before committing to an AI feature, it's worth running through a quick check:

1. Does this problem have only one correct answer?

2. Is the use case too simple or infrequent to justify the overhead?

3. Does a purpose-built tool already handle this better?

If the answer to any of them is yes, AI is probably the wrong tool for the job.

Conclusion

Teams are under pressure to move quickly on AI due to competition, investor expectations and user demand. But the instinct to add AI everywhere, to make every feature "smarter," can lead to products that feel innovative but are hardly ever used.​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?