AI isn’t meant for us

The question on people’s minds today seems to be: How did we get into this mess?

It applies to multiple aspects of current life. Many Americans will immediately think of presidential politics – and that’s all I’ll say about that. But it also says something about technology.

If you’ve been watching the AI headlines recently, you might have seen an interesting trend starting to develop:

• Google scrambles to manually remove weird AI answers in search (The Verge, May 24, 2024)

• Microsoft delays Recall again (Ars Technica, June 13, 2024)

• Apple reportedly cuts Vision Pro production due to low demand (The Verge, April 23, 2024)

• McDonald’s pulls AI ordering from drive-thrus (CNN, June 17, 2024)

These three Big Tech companies and a company serving ultraprocessed fast food, together, have a combined market cap of several trillion dollars. All of them are pausing, delaying, pulling back, or re-evaluating major product decisions. Three of the four products in question are powered by AI (and arguably Apple’s face jail is AI-related as well).

Thus we have our question, which I’ll restate as such: What caused some of the world’s biggest companies to publicly pull back on their plans for high-profile tech products?

By way of an answer I’ll quote my column from February 2023, Where are the customers’ chats?

When evaluating a new technology, it’s a dangerous oversight to forget customers (i.e., users, patients, students, etc.). Yet it happens. It’s typical of the tech industry to launch tools, products, whole platforms without a thought toward the user: my book Customers Included (2nd edition) offers plenty of examples. But the problem has grown worse in recent years as Big Data, algorithms, and now AI generators have captured the imagination of Silicon Valley. The Platonic ideal of digital tech these days is a predictable, controllable, fully automated system in which customers are, at best, a detail to be slotted in and manipulated for the benefit of the growth of the machine.

And that brings us to ChatGPT and other AI generators. We hear all about what the tool can do (write an essay, a limerick, some song lyrics), but not much about the reasons customers might want to use a chat tool. It’s the why that’s missing.

Although it’s “a dangerous oversight to forget customers,” that’s what all four companies did. Google went bananums for AI Overviews without considering how customers would feel about a pizza recipe with glue in it; Microsoft failed to consider that, as I wrote earlier this month, a “recall” is what happens when companies harm customers; Apple failed to heed my warning about customers’ rejection of face-mounted surveillance gear; and McDonald’s apparently didn’t consider whether customers might just hate the inefficient, buggy, hallucinating drive-thru bot adding hundreds of chicken nuggets to their orders.

Customers, in other words, were not included in these AI rollouts. So of course they failed.

Despite these high-profile failures, large companies are hiring consulting firms to help them embrace AI hype. Here’s the NYT (June 26, 2024):

From Boston Consulting Group and McKinsey & Company to IBM and Accenture, sales are growing and hiring is on the rise because companies are in desperate need of technology Sherpas who can help them figure out what generative A.I. means and how it can help their businesses.

One of those companies, IBM, was the consultant hired to develop the McDonald’s drive-through. Who knows what sorts of AI follies are being suggested by McKinsey and the others – though I do have a guess, based on what I saw from those consultancies in the dotcom boom of the late 1990s. Quoting the article:

The demand for tech-related advice recalls the industry’s dot-com boom. Businesses stampeded consultants with requests for counsel in the 1990s. From 1992 to 2000, sales for Sapient, a digital consulting firm, went from $950,000 to $503 million.

From what I saw back in those days, the big consulting firms weren’t much interested in building a good customer experience. Today’s big tech firms and their consulting partners are repeating history.

How did we get into this mess? How did we enter into an economy drenched in AI investments with (so far) little or no appreciable payoff for us, the citizens and students and customers? Because the AIs weren’t developed for us. They were never meant for us. They’re instead meant for the owners of the corporations: promising to cut costs, or employee count, or speed up operations, or otherwise juice the quarterly metrics so that “number go up” just a bit more – with no regard for how it affects the customer experience, the worker experience, the career prospects of creators and writers and musicians who have been raising the alarm about these technologies for years.

Damon Krukowski, musician and past Techtonic guest, from his June 11 essay:

Whatever AI is capable of in music – even if it never gets more than pretty good – it is supremely well suited to making music in a culture that does not reward labor. Content machines are a dream of corporate platforms, and corporate platforms are a dream of venture capitalists. AI is an answer to a problem of scale encountered by a platform economy unwilling to center itself on labor.

For that matter, read any number of AI takedowns that have been published recently. Creative Good members have access to our Forum thread called Satisfying takedowns of AI hype, which I’ve been adding to as these screeds have appeared. And then today Brian Merchant, another past Techtonic guest, posted his own roundup of AI takedowns. Merchant writes about AI:

so far, the tech *doesn’t even work reliably*. So we’re getting a Google flooded with bad and specious AI “overviews,” a continued stream of hallucinated content, and creative workers who are losing their jobs despite it all. Large corporations buying up hundreds of thousands of enterprise subscriptions for ChatGPT despite it all. Militaries adopting the tech despite it all.

That’s why I think we’re watching the simmer of discontent rise to a boil: More and more people are skeptical, even incredulous. Folks from every background are pitching in to raise their voices, to say ‘why this?’ Why is this being foisted on our platforms, our workplaces, our lives? (The answer is probably ‘corporations are desperately hoping the tech can give them leverage over workers’ but I digress.)

I have to wonder about the companies that continue to plow millions of dollars into harebrained, hype-driven AI dreams. Here’s one, as reported in the NYT (June 20, 2024): At Target, Store Workers Become A.I. Conduits. “The retailer is rolling out a chatbot to help workers answer questions from shoppers — and workers,” the story says.

What are the chances that this is really oriented toward serving customers and workers, rather than merely cutting costs?

How to include customers in AI

If I had to give advice to any company making an AI investment, it would be this:

1. First, spend time with customers (or whatever constituency you serve: patients, students, employees, citizens, etc.) and find out what their pain points are. What do they need, first and foremost? What are the annoyances that could be fixed, or the opportunities to deliver a better experience?

2. Next, create the product vision based directly on those findings, so that you’re building what will help customers.

3. Then consider what tools or technologies will enable you to build the product. If AI is part of it, great. But if not, that’s also fine. Don’t build something with AI for the sake of AI. You’ll likely fail.

4. Move forward with the building process and keep customers included in the development loop as you gather feedback and make revisions along the way.

As always, if I can help your team, drop me a line. I can deliver workshops, keynotes, and custom advisory work.

In the meantime, be careful of AI hype:

Creative Good members can post a comment on this column. If you’re not a Creative Good member, join our community!

Until next time,

-mark

Mark Hurst, founder, Creative Good – see our services or join as a member
Email: mark@creativegood.com
Listen to my podcast/radio show: techtonic.fm
Subscribe to my email newsletter
Sign up for my to-do list with privacy built in, Good Todo
On Mastodon: @markhurst@mastodon.social

- – -