惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Y
Y Combinator Blog
V
V2EX
Jina AI
Jina AI
爱范儿
爱范儿
M
MIT News - Artificial intelligence
量子位
L
LangChain Blog
Google DeepMind News
Google DeepMind News
酷 壳 – CoolShell
酷 壳 – CoolShell
罗磊的独立博客
腾讯CDC
MongoDB | Blog
MongoDB | Blog
P
Proofpoint News Feed
宝玉的分享
宝玉的分享
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Last Week in AI
Last Week in AI
H
Hackread – Cybersecurity News, Data Breaches, AI and More
F
Fortinet All Blogs
The GitHub Blog
The GitHub Blog
Engineering at Meta
Engineering at Meta
博客园 - 聂微东
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
Vercel News
Vercel News
T
The Blog of Author Tim Ferriss

Forbes - Innovation

Why Do Humans Have Fingerprints? Hint: It’s Not What You Think Booking.com Confirms Data Breach, Reservation PIN Codes Changed Why Major News Sites Are Blocking The Internet Archive’s Wayback Machine iPhone Fold Release Date: New Report Details Frustrating Apple News Comet Tracker: How To See Pan-STARRS And Three Planets On Wednesday NYT Mini Crossword Today: Tuesday, April 14 Hints And Answers Today’s NYT Strands Hints, Spangram, Answers: Tuesday, April 14 (It’s A Little Unclear) Today’s Wordle #1760 Hints And Answer For Tuesday, April 14 Most Of The Microplastics In Urban Air Come From Tires Today’s Wordle #1759 Hints And Answer For Monday, April 13 NYT Mini Crossword Today: Monday, April 13 Hints And Answers NYT Pips Today: Hints, Answers And Walkthrough For Monday, April 13 The YC Chief Who Codes 10,000 Lines A Day Has A Simple Secret Samsung Expands One UI 8.5 Beta To More Galaxy Owners Why You Should Stop Using Your iPhone If It’s On This List Chamath Says Firms That Treat AI As A Strategy Hand Rivals Their Edge 3 Unexpected Habits Of Secure Couples, By A Psychologist The First Lamp That Folds Your Clothes Samsung’s Disappointing Price Update For Galaxy Phone Buyers 3 Subtle Signs Someone Is Falling In Love With You, By A Psychologist Do Mantis Shrimp See More Colors Than Humans? A Biologist Explains NYT Connections Answers Explained For Monday, April 13 (#1,037) NYT Connections Hints Today: Monday, April 13 Clues And Answers (#1,037) LEGO Luigi & Mach 8 (72050) Review: 2026’s Best Set Yet? Marc Andreessen Says AI Productivity Will Trigger A Hiring Boom 3D Printing Is The Ultimate Hack To Reduce Household Spending Apple iPhone Fold: Striking Design Revealed In Leaked Photos Apple Smart Glasses: New Leak Reveals A Major Design Twist To Beat Meta Tested: The AI Coming To The Rivian R2 Quordle Hints Today: Monday, April 13 Clues And Answers
Human-In-The-Loop AI: What It Takes To Find The Right Bal...
Ryan Johnson · 2026-05-13 · via Forbes - Innovation

Ryan Johnson is the chief product officer at CallRail, an AI-powered lead engagement platform that serves over 225,000 businesses worldwide.

getty

​Every new wave of technology promises speed, efficiency and cost savings. AI does, too. As an example, for many overloaded companies, it can help triage incoming requests, separating what’s urgent from what can wait.

​Still, history has shown that adopting new technology does not guarantee immediate returns. Those lessons are not lost with AI. Despite 88% of organizations citing regular use of AI in McKinsey’s research, less than 20% have seen AI make a tangible impact on enterprise-level EBIT.

This dichotomy isn’t new. Prior tech implementations, such as ERPs and CRMs, also had high failure rates. Just like with those implementations, the disconnect with AI often stems from a misunderstanding of how it changes the way a business runs.

In the case of AI, one of the most important concepts to understand is how to architect a system that augments rather than replaces human judgment.

Human-In-The-Loop AI: The Missing Piece

AI typically performs best when tasks are structured, clearly defined and predictable. Its strength lies in analyzing vast datasets within a confined set of rules and providing recommendations based on observed patterns. That’s why it’s effective in generating reports, routing calls or identifying duplicates in a spreadsheet.

However, AI falls short when it lacks the context and specificity to be truly intelligent and effective. Without domain or market knowledge that usually resides in a person's mind, the output becomes unreliable.

In a business setting, this lack of accuracy can cause negative downstream effects, such as when Deloitte had to refund much of its fee for producing a report that contained AI-generated errors or when lawyers for MyPillow CEO Mike Lindell were fined for filing briefs with errors generated by AI. ​​

The implication for businesses is the need to layer human oversight into AI workflows. While AI is getting better at handling more complex tasks, human intervention is still needed in the process to ensure the right outcomes.

Why Workflow Orchestration Sits At The Center Of AI Success

To successfully integrate humans in the loop, answer this question: Where in the process should you trigger the AI-to-human handoff? ​The answer becomes even more pressing as the deployment of AI agents ramps up, since they act autonomously, often without natural points for human intervention. ​

The goal is to blend the speed and efficiency of AI with the accuracy and quality of human input to improve outputs. A study by Stanford and Carnegie Mellon, for example, found that AI-human teams are 68.7% more effective and accurate than AI or humans alone. ​

A way to think about it is to view tasks on a spectrum, with AI and humans at opposite ends. The AI handles ​repeatable and operational tasks, such as answering inbound calls or routing messages, while the human focuses on edge cases that require judgment before execution. ​

Take, for example, a pool company using an AI voice agent to manage inbound call leads. The AI voice agent can be used in the first half of the workflow to triage and qualify calls based on the lead-scoring rules the pool company has established.

Depending on the complexity, the AI voice agent can prompt the caller to book an appointment or a quick service request, at which point, the workflow is complete.

But if the caller requests a more complex service, like new pool construction, the AI voice agent will need to be orchestrated to route the call to the best-fit sales agent based on the conversation. ​

The net positive for the pool company is that it can deliver better customer experiences by ensuring timely and consistent support. Likewise, the company benefits from ensuring that no inbound leads slip through the cracks, thereby supporting bottom-line results. ​

Architecting The AI-To-Human Handoff

As a chief product officer, I view the orchestration of AI-driven workflows with human-in-the-loop the same way I approach solving product challenges. It starts with understanding the problem, outlining next steps, finding where technology can help and then engineering the system based on these outputs.

The same logic applies to designing the AI-to-human handoff. Specific business tasks need to be broken down into steps. Using the spectrum example above, the tasks need to be organized into those better suited to AI and those that require human approval or feedback.

As the system takes shape, the next step is to map the required data streams to facilitate the workflow. Not all businesses have the context for AI to reference, and that’s okay. This insight can guide how to optimize the perfect handoff as more data becomes available with use.

While automating end-to-end tasks can be difficult, the best approach is to start simple. See what AI can automate as much as possible without compromising accuracy for efficiency. Once the workflow design is completed, deciding which AI solution to implement, if one has not been selected, becomes easier.

Achieving AI-Human Harmony

AI might be one of the most powerful tools we’ll see in our lifetimes. But the promise of immediate productivity benefits exists only when we carefully balance how AI and humans work together as a complete system.

At face value, this might seem like a slower approach. Yet AI can also be used to expedite discovery and workflow design by running interactive tests that enable you to make iterative changes in real time.

Ultimately, it’s the integration of AI and humans working together as one system, not the AI solution alone, that will deliver the best results for businesses and their customers. ​


Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?