Earlier this month, a German court ruled that Google is liable for its AI search summaries. Rejecting defenses like “users can check for themselves”, and that they generally know “that information generated with AI should not be blindly trusted”, the court held that the AI’s summaries are reflections of the company and “above all an expression of Google’s business activities”.

This is the latest skirmish in a decades-old battle over internet publishing. Historically, there were two different types of information distributors: carriers and publishers. A phone company is a carrier. It’ll transmit whatever you say, even discussions about committing a crime. Words are words, and the phone company does not know – nor is it liable for – the words you choose to speak. A newspaper, on the other hand, is a publisher. It decides the words it publishes, and what quotes to include in its articles. If those words or quotes are defamatory or otherwise illegal, it’s liable.

Internet companies have long tried to play both ends of this distinction. They claim to be a carrier when it suits them, and also to be a publisher when that is advantageous. Section 230 of the 1996 Communication Decency Act enshrined this straddling when it shielded internet providers from liability for the speech of others on their platforms: “No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.”

For years, a debate has continued about how to apply this law to social media platforms. When platforms merely displayed peoples’ posts and comments in reverse-chronological order, they behaved largely like carriers, relaying peoples’ words without regard to their contents. But the next generation of platforms, like Facebook, curated feeds with algorithms and thereby acted more like publishers, making editorial decisions about who sees what. Some experts think section 230 has gone too far and needs reform; others think that it’s what holds the modern internet together.

Google’s AI overviews are far less nuanced. They work differently from traditional search, which courts have held involves archiving and facilitating access to the editorial content of third parties. AI overviews don’t just quote and republish words from different websites. With overviews, the AI rewrites other peoples’ words, exercising editorial discretion like a newspaper article or an original essay on a topic.

It’s not only Google’s AI that falls into this category. Imagine a restaurant review site that provides AI summaries, or a site summarizing laws and government procedures. Or a traditional publisher that uses AI to summarize its own publication. Accuracy matters, and liability is one of the most important ways we as a public can demand accuracy and hold companies accountable when they cause harm.

Two years ago, Air Canada learned this lesson. Its AI chatbot promised a discount the company later rescinded, arguing in court that the airline wasn’t responsible for the promises the bot made because it was a “separate legal entity that is responsible for its own actions”. The court sided with the flyer, saying that the airline was just as responsible for what its chatbot says as what’s on its website. The potential precedent here is that corporations have a duty of care for the performance of the AI chatbots they employ.

AI agents are agents of the person or organization that deploys them – and should be treated by the law as such. If a company hired human writers to write its summaries, that company would be liable for inaccuracies in those summaries. If a company’s human agent signed contracts in the company’s name, that company would be bound by those contracts. And if a doctor gave dangerously wrong medical advice, they would be liable for malpractice.

To allow businesses to hide behind the excuse of faulty AI in those same circumstances would be a massive handout to companies, and would introduce disastrous incentives for corporate misbehavior. Why hire human writers, lawyers or doctors when AIs are not only cheaper, but also absolve employers whenever they make a mistake?

We are rapidly moving to a world where AI-powered chatbots will be at the other end of all sorts of corporate communications channels. It makes no sense for a company to be able to honor its statements when it wants to and disavow them when it doesn’t.

Visa and OpenAI recently announced a partnership to build personal AI agents to, among other things, make purchases on our behalf. This is just one of many similar projects in the works, as companies race to provide us all with AI assistants. Will Visa take responsibility when its AI makes a purchase in your name that you don’t want? And if Visa won’t, why would anyone trust the system? Properly allocating liability is key to make this kind of thing work.

If the German ruling holds, it could be devastating for Google’s AI Overview feature. Tests from earlier this year found that it had mistakes about 10% percent of the time. At more than 5tn searches per year, that’s 16,000 erroneous summaries every second. And while most of those errors are benign, some of them will cause harm, be defamatory, or otherwise trigger liability.

Earlier this year, Google’s AI summary falsely identified the Canadian fiddler Ashley MacIsaac of being a sex offender. His lawsuit, filed in Ontario, is ongoing. If Google is forced to invest in improving its AI system until those kinds of errors are exceedingly rare, that seems like a good outcome for users, as well as the subjects of search, like MacIsaac.

More generally, liability concerns could mean that many current use cases for agents won’t be commercially viable. Companies may not be able to profitably operate AI lawyers, doctors, and media influencers if they are held responsible for what they say and do.

We’re OK with this outcome. There’s nothing in the law that requires us to accommodate AI systems if they are fundamentally untrustworthy, just as we don’t need to accommodate untrustworthy human systems. Any company that won’t stand by the statements its agents make – whether human or AI – doesn’t deserve users’ time or money.

• Bruce Schneier is a security technologist who teaches at the Harvard Kennedy School at Harvard University

• Nathan E Sanders is a data scientist affiliated with the Berkman Klein Center of Harvard University and co-author, with Schneier, of the book Rewiring Democracy: How AI Will Transform Our Politics, Government, and Citizenship