Anthropic has accused groups linked to Alibaba and its Qwen AI lab of carrying out a massive campaign to extract capabilities from Claude just by asking it a lot of questions, as first reported by Reuters. The AI developer wrote a letter to U.S. lawmakers alleging that Alibaba used nearly 25,000 fraudulent accounts to generate more than 28.8 million interactions and glean detailed, proprietary information about Claude.
Alibaba has not publicly responded to the allegations, and there has been no independent confirmation of Anthropic's claims, but simply leveling them has potentially enormous consequences. The sheer volume of accounts and interactions is eye-catching, but it's even more fascinating how it reveals a vulnerability in AI models that can give away their secrets.
AI developers may now have to worry that rivals can learn from those models without ever seeing the underlying code or training data through a technique known as model distillation. Essentially, AI models will inadvertently share deliberately obscured facts about themselves if a huge number of the right questions are asked. As an analogy, imagine taking a test about a book, but instead of reading the book, you ask the author one million questions about their life, their thinking, their experience writing the book, and several hundred thousand more questions. You'd probably have a pretty good chance of knowing everything they might have written without once cracking the covers.
Can you copy an AI just by talking to it?
Model distillation is a common technique used by AI companies to build variations of their models, especially smaller, faster options. But no company would be okay with a rival using their model to train the competition. But that's what Anthropic alleges. The fake accounts supposedly asked Claude a ton of very complex and detailed questions related to its advanced software engineering and agentic reasoning features. The responses filled in a picture of the model's workings, accelerating Alibaba's own development of competing AI systems, Anthropic claimed.
The conundrum is obvious. Large language models are designed to answer questions. Every answer teaches the user something about how the model behaves. You can't interact with an AI model, or a person, without giving up some information about yourself. Normally, that wouldn't matter, but at the scale Anthropic is claiming, conversations become reverse engineering.
It's not the first time Anthropic has alleged illicit model distillation. Anthropic levied similar claims against DeepSeek, Moonshot AI, and MiniMax earlier this year. And other companies, including OpenAI, have expressed concern that they have also been victims of the technique.
The glaring irony that the companies that used enormous collections of publicly available information, including licensed material, to train their AI models are now arguing about how those same models are valuable intellectual property is hard to ignore.
AI arms race
AI developers see their models' behavior as crucial to competing with rivals. If another company can reproduce much of that behavior by asking enough carefully designed questions, spending billions of dollars training frontier models starts to seem like a waste.
Anthropic claims model distillation can effectively transfer years of work on their part to another company for almost nothing. Anthropic asked lawmakers to take action and combat this problem as soon as possible. If leading models can be imitated so easily, there won't be much incentive to innovate, and the AI competition will only be about beating copycats. And picking the best models will be difficult, as a new AI model that matches an existing one's capabilities might be born of years of original research or simply copying an existing option.
Whether Anthropic ultimately proves its allegations, they have revealed that the next great AI battle may not be about building the smartest model. It may be about stopping somebody else from talking to your model and learning how it operates, one question at a time.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
