The ChatGPT app logo displays on the screen of a smartphone in Reno, United States, on November 21, 2024. (Photo by Jaque Silva/NurPhoto via Getty Images)
NurPhoto via Getty Images
People use AI Assistants to answer all sorts of random questions and they likely expect the answers they’ll receive to be accurate ones.
But according to new data from the streaming data and metadata platform Reelgood, a controlled accuracy analysis of streaming title availability data found that ChatGPT scored 43.76% and Claude scored 50.21% when asked where major television and movies titles could be streamed.
The analysis, conducted by Reelgood on March 5th, 2026, tested each source against the same set of 50 movies and 50 TV shows using identical queries.
In comparison, the same queries received an accuracy rate of 96.89% accuracy using data collected by Reelgood, which provides availability data and content metadata across over 300 services in more than 25 countries.
In an interview I did with Reelgood CEO David Sanderson in April, he explained that the problem with using an AI assistant is they rely primarily on data from the web, which can significantly affect the accuracy of the answers.
“You need accurate data behind it,” Sanderson said. "And the other thing is, with AI or chat experiences, if you want to know where a piece of content is available, a lot of their data is compiled from crawling the web. But sites don’t write a follow-up article that says, 'This film is now off Hulu.’ And that can create incorrect data that the LLMs are presenting."
In this specific study, Reelgood identified several specific issues that led to accuracy problems when using ChatGPT and Claude:
Models confidently report titles as currently streaming on services they’ve already left. The cause is structural: Entertainment press covers new additions to a catalog extensively but rarely follows up when a title quietly leaves weeks or months later. The training corpus skews heavily toward those announcements, so the model treats outdated positives as current. This is the most pervasive error pattern observed.
Models frequently treat titles available through paid add-on channels (such as Starz or Paramount+ on Amazon Prime Video) as if they were part of the parent service's base subscription. Users are told a title is streaming "on Prime Video" when accessing it actually requires a separate Starz or Paramount+ add-on inside Prime Video, creating the false impression that their existing subscriptions cover it.
Free and ad-supported services like Tubi, Pluto TV, Fawesome, Hoopla and Kanopy are consistently omitted, even when they're valid sources for a given title.
Models sometimes list a service as a subscription (SVOD) option when the title is only available there for rent or purchase, misleading users about what their existing subscriptions actually cover.
Both models almost entirely omit transactional video-on-demand (rent/buy) options from services like Apple TV and Amazon, affecting the majority of titles tested.
When multiple versions of a title exist (such as One Piece, which has both an anime series and a live-action Netflix adaptation), models conflate availability across different versions.
Reelgood’s study provides some real-world examples of the answers users might receive when asking for the availability of some current titles.
Both ChatGPT and Claude confused the 2023 live-action Netflix series with the long-running anime, incorrectly listing Crunchyroll and Hulu as valid sources. Claude additionally listed "Crunchyroll Amazon Channel" and "TV Guide" as sources.
Reelgood correctly identified the title as a Netflix exclusive (available on Netflix and Netflix with Ads). This is a clear title disambiguation failure, where the models could not distinguish between two distinct productions sharing the same name.
ChatGPT incorrectly listed Hulu as a source, and Claude incorrectly listed Netflix. In both cases, the title wasn't actually available on the platform they named.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。