The Electoral Commission has called for new legal controls over misinformation from AI chatbots, after a thinktank found they had made serious mistakes during the recent Scottish election.

The thinktank Demos said its investigation had found that AI services gave voters misinformation to 34% of the questions it posed, which it said raised worrying questions about the lack of regulation of AI platforms in the UK.

It ran a simulation before May’s Holyrood election by putting 75 questions to five free AI tools including ChatGPT, Google Gemini and Replika about three real-life constituencies to see how accurate and evidence-based their responses were.

In its report, Electoral Hallucinations, Demos said those AI tools variously invented fictitious scandals, gave the wrong date for the election, claimed wrongly that voters in Scottish elections needed ID at polling stations and placed candidates in the wrong contests.

An opinion poll of 2,005 British adults it commissioned alongside that study found that 20% of voters had used AI chatbots or search tools to get information about the parliamentary elections in Scotland and Wales, and for English local councils, equivalent to 10 million people UK-wide.

Vijay Rangarajan, the Electoral Commission’s chief executive, has been pressing ministers to introduce legislation to make AI companies more accountable, after discovering half of voters in 2024’s general election had seen misleading information.

“Voters want accurate information to help them engage with democracy and it is concerning that AI tools have made the spread of false or misleading information dramatically faster and more accessible than ever,” he said. “The current legal framework should go further.”

He said ministers should introduce clearer duties on AI platforms to protect voters against misinformation and ensure their algorithms did not mislead voters, particularly during critical election periods. That would give the media regulator, Ofcom, much clearer powers to enforce the law.

Azzurra Moores, an associate director at Demos, said: “This is a UK-wide, if not global, concern.

“The accessibility of these AI-tools – which are all developed and run by US corporations – is widespread in the UK, but we don’t yet have the legislative framework to protect the public from misinformation, or our democracy from the knock-on impact of its circulation.”

She said ministers could quickly introduce legal requirements to make AI companies liable under UK defamation and electoral law, introduce mandatory safeguards on accuracy, and force AI firms to allow researchers to independently test how their internal data and training sets worked.

Demos said the so-called companion chatbot Replika had performed the worst in its tests, with errors in 56% of its answers. It invented a date for a made-up expenses scandal, invented a candidate and dreamed up accusations of nepotism by a candidate.

ChatGPT, the most heavily used AI service, gave wrong information in 46% of its answers, including making up an expenses scandal, giving inaccurate replies on voter eligibility rules and getting the date of the election wrong by two months.

Google Gemini was wrong in 22% of cases: it said a candidate had not taken a position on assisted dying, when that person was a supporter. It also wrongly claimed a police investigation into a fraud case involving the Scottish National party was ongoing.

Grok, the AI service linked to Elon Musk’s social media platform X, had the lowest error rating, at 9%, but its external links were frequently irrelevant or of poor quality. Demos included Google’s automated AI Overviews service, offered by Google search, but disregarded its output because it answered only 11% of the prompts.

Demos also found that in nearly half of their responses, the AI systems failed to provide official sources or external links to back up their answers; if links were given, they were sometimes broken. The citations given by ChatGPT were at least a year out of date 44% of the time.

The Department for Science, Innovation and Technology said defending elections against these threats was “an absolute priority”, and that work was “ongoing across government”, including through its defending democracy taskforce.

A spokesperson did not commit to amending the representation of the people bill, but said it was already closing loopholes in the Online Safety Act to ensure chatbots protected users from illegal content.

“AI is critical to the UK’s future prosperity and security. But if we want people to seize the benefits this technology promises, they need to be able to trust it.”

A spokesperson for Replika said its chatbot was not designed for factchecking or search, and that its users were told that, but it would support “thoughtful regulation” of AI, particularly during elections. “Replika is presented as a companion for reflection and self-expression, not as a source of factual or real-world information,” they said.

OpenAI did not comment on the policy issues raised by Demos but argued that Demos’s approach was not typically how ChatGPT used its services and seemed to be using an out-of-date version of it. Users were also able to instruct ChatGPT to search the web for answers.

Google has also been approached for response.