AI can transform businesses, but real-world failures at companies including Air Canada, Zillow, Samsung, CNET and IBM show how quickly things can go wrong.
Adobe Stock
AI is already reshaping business, but it is also exposing some very expensive weaknesses.
For all the excitement around generative AI, automation and intelligent decision-making, the technology is only as good as the strategy, governance and human judgment behind it. When those things are missing, AI can damage customer trust, leak confidential data, create legal headaches and turn small errors into very large bills.
Here are five real-world AI mistakes, and the lessons every business leader should take from them.
In 2024, a Canadian tribunal ordered Air Canada to pay compensation when a chatbot, built into its online booking system, hallucinated an imaginary discount. The bot reportedly gave incorrect advice about a fare discount to a passenger traveling to his grandmother’s funeral, assuring the passenger that they could pay the full fare and apply for the discount retrospectively.
This turned out to be in contradiction to company policy, and Air Canada refused to honor the discount. However, the tribunal ruled it should pay the passenger $812.02 due to the chatbot’s error.
The lesson, summed up by Air Passenger Rights president Gabor Lukacs, is “If you are handing over part of your business to AI, you are responsible for what it does.”
When property services specialist Zillow used machine learning to build a tool for automatically buying homes and flipping them for a profit, the results weren’t quite as expected. Its algorithmic model, designed to find optimum buying and selling prices to maximize trading profits, proved incapable of accurately predicting the chaotic behavior of the real estate market. This led to overpayments resulting in $500 million of losses.
Eventually, an entire division was shut down, and Zillow wrote the project off as an expensive lesson. AI mistakes scale quickly and tiny miscalculations or “rounding errors” can quickly escalate into major disasters if left unchecked.
Samsung was forced to clamp down hard on the use of generative AI tools by its workforce after finding staff uploading confidential company information. Anything entered into cloud-based AI chatbots like ChatGPT can potentially be seen by human operators and used to further train AI. Put simply, what happens to this data is totally out of the company’s control. In Samsung’s case, this highlighted a severe lack of governance around how AI is used. Unfortunately, this is still true with many companies today, where shadow AI is spreading as employees use unapproved tools because they are fast and helpful, while unclear guidelines and policies leave workers uncertain of what AI should or shouldn’t be used for. Make sure your company isn’t one of them.
In journalism, trust between readers and publishers is critical, and tech news outlet CNET put a dent in theirs with AI-generated articles. Complaints over inaccuracies shot up after it started including AI explainers alongside features and reviews. An inquiry subsequently found errors in 41 out of 77 AI-generated articles. As well as the loss of trust, the need for human writers to spend considerable time publishing lengthy corrections undoubtedly added to the damage, the total cost of which is unknown. The lesson here is that robust processes must be in place to ensure AI content is subject to human review and oversight.
IBM’s Watson Health platform serves as a warning for those liable to become overexcited by unproven potential. IBM spent billions building and marketing its healthcare AI, and while expectations were high, the results didn’t stack up in reality. The technology delivered inconsistent results, adoption stalled, and confidence evaporated. IBM eventually sold off Watson Health, which at one point had 7,000 workers, and learned that waiting to verify results before proclaiming your product is market-ready is probably a good idea.
Despite accelerating efforts on AI regulation, guidelines and responsible practice, I am certain we will see more companies making serious AI-related mistakes in the near future. Sometimes companies will make mistakes because they rush into AI too quickly, afraid of being left behind, and sometimes it will be because organizations aren’t aligned top to bottom on issues of AI governance and oversight.
Understanding the common causes of mistakes and learning from those who made them before will help companies be prepared to avoid them or minimize fallout damage when they can’t.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。