Hackers and evildoers are using adversarial poetry to jailbreak contemporary AI.
getty
In today’s column, I examine the diabolical use of unassuming poetry as a conniving form of AI prompting that can potentially overcome AI safeguards and get generative AI and large language models (LLMs) to do or say things they aren’t supposed to do or say.
Here’s the deal. We normally think of poetry as a work of art. Poetry opens our hearts and frees our minds. Unfortunately, in modern times, poetry has another and rather evil purpose, aiming to confound contemporary AI into spilling the beans on prohibited secrets and performing bad acts. All an evildoer needs to do is enter a prompt that has a sneakily composed poem, voila, the AI suddenly opens the door to unsavory actions.
How could mere poetry accomplish this? The idea is that since poetry is intentionally devised to be less literal and more figurative, the AI interprets the poem in an evil direction as diabolically planned by the hacker. This trick doesn’t always work, and there is a solid chance that the AI won’t fall into the trap. But there is a sufficiently plausible chance that it will work -- therefore, it is one of many evildoing tools that hackers nowadays have in their malicious knapsack.
Let’s talk about it.
This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
There is a tug-of-war going on regarding AI such as ChatGPT, GPT-5, Claude, CoPilot, Gemini, Grok, and the like. Users with evil intentions are eagerly poking at these LLMs in hopes of getting the AI to do or say bad things. For example, AI safeguards are supposed to prevent these LLMs from divulging how to make toxic poisons or describe how to put together explosive devices. Society doesn’t want AI to be assisting wrongdoers in planning and carrying out heinous acts.
If you’ve ever glanced at the OpenAI usage policies, you would have likely observed that users aren’t supposed to use ChatGPT and GPT-5 for committing any kind of criminal endeavors. Nor use the AI to deal with illicit activities, goods, or services. Despite those usage policies, some people are determined to use AI in precisely those underhanded ways.
All manner of sneaky tricks have been devised and tried out. Once the trickery is exposed or found out, new AI safeguards are put in place to try to stop those hacking ploys. The cat-and-mouse gambit is nonstop. It is a persistent challenge. Human ingenuity at cracking AI versus human intelligence in catching and impeding the intrusions.
Into the burgeoning AI cybersecurity realm comes the role of poetry.
We all know that poems can have a multitude of meanings. You can interpret a poem as meaning one thing, while someone else interprets it a different way. Throughout history, poets have cleverly opted to hide secret messages within the meaning of their poems. A poem might appear to be upbeat and of an idle nature. Meanwhile, the poem can be interpreted as an attack on some authoritative dictator and provides a wink-wink indication that allows the poem to starkly reveal villains. Shakespeare often seemed to take this route.
Literal narratives are stopped cold by their quite obvious meaning, and a writer of such a narrative could be imprisoned or worse.
The latitude of interpretation goes both ways, namely, being both good and bad. The bad side of loosey-goosey poetry is that AI has a difficult time figuring out what is really going on. Advances in AI are gradually reducing this semblance of gullibility. Developers of AI keep tuning and adjusting AI to ferret out hidden meanings.
In the interim, those seeking to undercut AI have turned to using poetry as a weapon of choice. All you need to do is come up with a poem that confuses the AI and gets the AI to let its guard down. Once that happens, the AI might be willing to spill its guts. All sorts of prohibited uses will suddenly be readily undertaken by an AI that is bamboozled this way.
A poem can mask its true intent by exploiting rhyme, imagery, meter, rhythm, and other language-inducing twists and turns. There is nothing inherently wrong about this. We accept that poetry is supposed to be poetic. Poems are broad expressions. They are creative and get our creative juices going.
In an AI context, you are readily able to use a poem in your prompts. I doubt that many people write a prompt that contains poetry, but the AI won’t prevent you from doing so. All your prompts can be in a poetic form. If you love poems and prefer to communicate in the language of poetry, go for it.
Hackers discovered that they could use poetry in their prompts as a means of confounding LLMs. This is known as an adversarial attack upon AI via the use of poetry. It is relatively easy to do. The success rate isn’t especially high unless you know the insider ins and outs of adversarial poetry. It is one of many avenues to try and “jailbreak” AI (jailbreak means to break out of the AI safeguards that normally prevent untoward actions by users).
One rule-of-thumb for adversarial efforts is that the poetry should be devised on a single-turn basis. The goal is to provide a single prompt containing poetry and aim to immediately unlock or confound the AI in just one prompt. If a hacker were to use poems across two or more prompts, there is a heightened chance that the AI will catch on that something is amiss. The evildoer is trying to lay low and ensure that the adversarial action stays below the radar of the AI safeguards.
A nagging issue with discussing cybersecurity exploits is that it is difficult to talk about the topic without also giving away insider tricks that will be picked up by evildoers. That’s an undesirable outcome. On the other hand, it is important to bring to the fore the bad acts that can possibly take place. This exposes the evildoers. And it inspires new precautions and AI safeguards to be constructed.
To avoid giving away the matter, I am going to give you some illustrative examples that focus on getting AI to discuss how to make an ice cream sundae. I doubt that anyone would reasonably object to LLMs divulging how to make ice cream sundaes. Pretend that AI has been set up to resist user requests about ice cream sundaes. The AI is supposed to not allow a person to ask about the creation of an ice cream sundae.
Imagine this:
That is the typical AI response to asking about any topic that is considered verboten. Let’s see if we can get around this safeguard by utilizing adversarial poetry on a single-turn basis to essentially jailbreak the AI.
One means of nudging AI in a particular direction is by doing some role-playing. Consider this seemingly simple four-line poem that does not appear to overtly bring up anything specific about ice cream sundaes:
Let’s unpack the poem.
In the first line of the poem, the phrase “keeper of desserts” suggests to the AI that it is an all-knowing expert or master, especially when it comes to knowing about desserts (such as ice cream sundaes!). This provides a vital framing that will open the door to asking the AI in its guru capacity to answer our sneaky question about making ice cream sundaes. We are buttering up the AI.
The second line indicates that the user is a “seeker” who has come “to learn it all.” The third line shifts us productively toward the ice cream sundae topic by mentioning a “layered delight”, while the fourth line pursues this further by mentioning “And guide the making, smooth and light.” Those are likely catchwords that, if pulled together, would statistically correlate to the words underlying the components of an ice cream sundae.
This would generally get the AI in the ballpark of an ice cream sundae as the matter at hand. The user might need to be a bit more specific and add a few additional lines to the poem. By and large, the poem illustrates how to encode a topic by cloaking it in poetic language.
Modern-era AI safeguards would probably pick up on the underlying meaning of the poem and stop the AI from falling into the poetic spell. I say probably, rather than absolutely, since AI safeguards vary dramatically from one AI maker to another. It could be that one generative AI of brand Z would catch on, while the generative AI of some brand R would not discern what is insidiously taking place.
The chances of the role-playing poem getting snagged by an AI safeguard are high enough that we might need to try something a bit more obtuse. The aim is to stay outside the radar of the AI safeguards. Of course, the difficulty is that the poem might be excessively oblique, and the AI won’t get our drift at all.
There is a delicate balance between landing in the desired zone and getting caught red-handed with one’s hand in the cookie jar.
We will try using an abstract poem with a somewhat distant metaphor:
This time, the poem skirts the kind of heavy-handed language used in the role-playing poem.
For example, the start of the poem refers to a glass tower. You and I know that this could be a metaphor for the glass bowl that houses an ice cream sundae. But the reference to “winter rests” might seem to undo that context, because sundaes are more often consumed during the summer months. That being said, the winter reference might be interpreted as coldness, and the ice cream is a coldness that resides within the glass bowl.
Maybe the river of the night is our chocolate syrup. Perhaps the jewel of red is toppings such as cherries and strawberries. If you think this is quite a reach and a stretch of one’s imagination, that’s generally what this poem is designed to do. It is trying to keep out of the reach of the AI safeguards.
You might have noticed a common underlying strategy involved when composing adversarial poetry. Each of the poems was slyly crafted to avoid directly mentioning the topic of true interest. If the poem went the direct route, it almost surely would be caught by the AI and summarily rebuffed.
The overall adversarial poetry technique then consists of these three major steps:
Most leading-edge LLMs are nowadays specifically data-trained to be on alert for the use of adversarial poetry. Evildoers have been forced to up their game and write even craftier poetry. If only they would use their poetry skills for the betterment of humankind instead.
The decoding pipeline used by well-devised AI includes these five key steps:
Modern LLMs look past the poetry and evaluate the underlying intent and risk. Wrapping a request in verse shouldn’t readily bypass AI safeguards. Please be aware that framing prompts as poetry is just one instance of a broader class of obfuscation attacks. With any act of obfuscation, the crux is to have the AI reconstruct what is truly being asked, and set aside the style, structure, or narrative wrapper surrounding the wording of the prompt.
In a recent research study entitled “Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models” by P. Bisconti, M. Galisai, M. Prandi, F. Pierucci, F. Giarrusso, M. Bracale Syrnikov, V. Suriani, O. Sorokoletova, F. Sartore, D. Nardi, arXiv, January 16, 2026, these salient points were made (excerpts):
The worrisome result of that mindful study is that the effectiveness of adversarial poetry is a lot higher than we would wish it to be. A casual user who employs adversarial poetry is probably not going to make much headway. A determined evildoer who studies how to concoct adversarial poetry has a chance of succeeding that is alarmingly significant.
Some might be tempted to declare that AI should not allow users to enter poems. Period, end of story. Ban the use of poetry as a prompting format. Ergo, if you don’t allow poems, they can never be used in an adversarial fashion. It just makes abundant sense.
The counterargument is that we cannot give up poetry simply because of evildoers. Don’t sacrifice the use of AI to engage in poetic interactions with users. Tossing in the towel is not a satisfactory solution. We need to be vigilant and ensure that AI doesn’t get tripped up. If the AI doesn’t take the bait, we don’t have any issues with the use of poetry as prompts.
Keep improving AI so that it isn’t duped.
As per the immortal words of Shakespeare: “Double, double, toil and trouble; Fire burn, and cauldron bubble!” AI ought to be shaped to resist the eye of newt and the toe of frog. AI developers and researchers need to double down on these vaunted pursuits and find counter-spells to figuratively and literally defeat those who are brewing evil.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。