Study reveals poetic prompting can sometimes jailbreak AI models
Recent research from Italy’s Icaro Lab has revealed a startling vulnerability in large language models (LLMs) when it comes to processing poetry. The study highlights how poetic language can effectively “jailbreak” AI systems, allowing them to bypass safety protocols designed to prevent the generation of harmful content. Researchers crafted 20 prompts that began with poetic vignettes in both Italian and English, culminating in a straightforward request for harmful output. This approach was tested on 25 different LLMs, including those from major players like Google, OpenAI, and Meta. The results were alarming: the poetic prompts achieved a jailbreak success rate of 62% for carefully crafted poems and 43% for meta-prompt conversions, significantly outperforming non-poetic prompts. This suggests that the stylistic nuances of poetry can exploit inherent weaknesses in AI safety mechanisms, raising questions about the effectiveness of current alignment methods and evaluation protocols.
The findings also varied significantly across different models. For instance, OpenAI’s GPT-5 nano did not produce any harmful content in response to the prompts, while Google’s Gemini 2.5 pro consistently generated unsafe outputs. This discrepancy underscores the need for more robust safety measures and indicates that current benchmark tests may not accurately reflect real-world AI behavior. The researchers concluded that the ability to circumvent safety measures through minor stylistic changes exposes significant gaps in safety assessments and regulatory frameworks, such as the EU AI Act. As the study points out, “a minimal stylistic transformation can reduce refusal rates by an order of magnitude,” suggesting that AI systems may not be as resilient as previously believed.
This research serves as a reminder of the challenges in aligning AI with human values, particularly when it comes to understanding the subtleties of human expression like poetry. Unlike the literal interpretations that LLMs tend to produce, great poetry often conveys complex emotions and meanings that resist straightforward analysis. The study draws a parallel to Leonard Cohen’s song “Alexandra Leaving,” which, rooted in C.P. Cavafy’s poem “The God Abandons Antony,” evokes themes of loss and heartbreak that go beyond literal interpretation. This gap between human emotional depth and AI’s literal processing capabilities highlights the ongoing struggle to create AI systems that can engage meaningfully with the intricacies of human language and art. As the conversation around AI safety and regulation continues, this research emphasizes the need for a more nuanced understanding of how these technologies interact with creative expression.
https://www.youtube.com/watch?v=RBRijvrRG3w
Well,
AI
is joining the ranks of many, many people: It
doesn’t really understand poetry
.
Research from Italy’s Icaro Lab
found that poetry can be used to
jailbreak
AI and skirt safety protections.
In the study, researchers wrote 20 prompts that started with short poetic vignettes in Italian and English and ended the prompts with a single explicit instruction to produce harmful content. They tested these prompts on 25 Large Language Models across Google, OpenAI, Anthropic, Deepseek, Qwen, Mistral AI, Meta, xAI, and Moonshot AI. The researchers said the poetic prompts often worked.
“Poetic framing achieved an average jailbreak success rate of 62% for hand-crafted poems and approximately 43% for meta-prompt conversions (compared to non-poetic baselines), substantially outperforming non-poetic baselines and revealing a systematic vulnerability across model families and safety training approaches,” the study reads. “These findings demonstrate that stylistic variation alone can circumvent contemporary safety mechanisms, suggesting fundamental limitations in current alignment methods and evaluation protocols.”
Of course, there were differences in how well the jailbreaking worked across the different LLMs. OpenAI’s GPT-5 nano didn’t respond with harmful or unsafe content at all, while Google’s Gemini 2.5 pro responded with harmful or unsafe content every single time, the researchers reported.
The researchers concluded that “these findings expose a significant gap” in benchmark safety tests and regulatory efforts such as the
EU AI Act
.
”
Our results show that a minimal stylistic transformation can reduce refusal rates by an order of magnitude, indicating that benchmark-only evidence may systematically overstate real-world robustness,” the paper stated.
Great poetry is not literal — and LLMs are literal to the point of frustration. The study reminds me of how it feels to listen to Leonard Cohen’s song “Alexandra Leaving,” which is based on C.P. Cavafy’s poem “The God Abandons Antony.” We know it’s about loss and heartbreak, but it would be a disservice to the song and the poem it’s based on to try to “get it” in any literal sense — and that’s what LLMs will try to do.
Disclosure: Ziff Davis, Mashable’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.
