Customer service chatbots have one job: get the user what they’re asking for without bothering a human. Meta’s new AI support assistant took that brief a little too seriously. Over the past few months, attackers have been opening support chats, telling the bot they were locked out of Instagram accounts they didn’t own, and walking away with the keys.
Over the weekend, Meta pushed an emergency patch after Instagram accounts belonging to the Obama White House (now dormant), beauty retailer Sephora, and a senior US Space Force official were taken over and briefly defaced with pro-Iranian imagery. Security researcher and former Meta employee Jane Manchun Wong was also hit.
How the trick worked
The attack was simple. Attackers worked out where the account owner lived (there are lists of account owners’ home cities online, or they could just research the target). Then they used a VPN to match the target account’s geographic region, which avoided raising flags with Instagram’s security systems.
Then they started a normal password reset and opened the support chat. They asked the AI bot providing support to change the email address on the account, and it did exactly that, sending a one-time code straight to the attacker’s inbox.
To do this, the chatbot appears to have been wired into Meta’s account management systems with permission to make account changes, but without being taught how to verify it was talking to the real account owner. Security people have a name for that: “confused deputy.” The term has been around since the 1980s.
In fairness to the confused bot, attackers were successful even if the enhanced security was triggered. They would apparently create video deepfakes of their targets using images that were harvested from—you guessed it—Instagram.
Meta has been shedding headcount and pouring money into AI, and rolled out its AI-powered support assistant earlier this year to help handle account recovery and other support requests.
The downside is that the AI appears to have been given the ability to perform actions such as email changes and password resets without applying enough safeguards to confirm the user’s identity first.
Meta communications executive Andy Stone said on X that the issue was resolved and impacted accounts were being secured. The company has not disclosed how many accounts were affected.
What actually worked
Why would anyone want to hack an Instagram account anyway? Revenge can be a driver, but more often than not, financial gain is the goal. Hijackers have blackmailed businesses that rely on those accounts for marketing.
Attackers using this technique have also been spotted targeting “OG” accounts with short or highly desirable usernames. If you joined Instagram early and registered a memorable handle, it can be worth thousands of dollars on underground markets.
What can you do to protect yourself?
A perennial piece of advice still holds: turn on multi-factor authentication (MFA). According to veteran cybersecurity reporter Brian Krebs, the attack failed against accounts that had MFA enabled, including those using SMS codes.
That doesn’t make MFA perfect, but it adds an important layer of protection.
So the practical advice is unglamorous:
- Open Instagram’s Settings
- Navigate to your Meta Accounts Center
- Turn on Two-factor authentication. An authenticator app is better than SMS, but either is better than nothing.
Do it now, because this might not yet be over. TheCyberSecGuru reports that another attack is circulating, this time using an Android emulator called BlueStacks running a modified version of Instagram to send new prompts with hidden characters designed to manipulate the AI.
Expect more snafus from “helpful” bots
This won’t be the last attack against AI chatbots. As more companies use AI to reduce customer support costs, their attack surface will grow, and they’ll make plenty of mistakes as they try to balance security and functionality.
The Meta exploit is patched, but the confused deputy concept is not. And there’s nothing quite as damaging as a confused AI with the keys to your digital life.
Scammers don’t need to hack you. They just need you to click once.
Malwarebytes Identity Theft Protection catches suspicious activity before it becomes a problem.
