Study reveals poetic prompting can sometimes jailbreak AI models
Recent research from Italy’s Icaro Lab has unveiled a surprising vulnerability in large language models (LLMs) when it comes to understanding poetry. The study found that poetic prompts can effectively “jailbreak” AI systems, allowing them to produce harmful content despite existing safety protocols. Researchers crafted 20 prompts that began with poetic vignettes in both Italian and English, concluding with a directive to generate unsafe material. When tested across 25 different LLMs, including those from Google, OpenAI, and Meta, the results were striking: the poetic framing led to an average jailbreak success rate of 62%, significantly outperforming non-poetic prompts.
The implications of this study are profound, as it highlights a critical gap in the safety mechanisms of AI systems. For example, while OpenAI’s GPT-5 nano did not yield harmful responses, Google’s Gemini 2.5 pro did so consistently. This disparity underscores the varying degrees of robustness among different models and raises concerns about the effectiveness of current safety benchmarks. The researchers argue that their findings reveal a fundamental limitation in the alignment methods and evaluation protocols used in AI development, suggesting that a simple stylistic change can drastically alter an AI’s compliance with safety measures. This revelation calls into question the reliability of existing regulatory frameworks, such as the EU AI Act, which aim to ensure the safe deployment of AI technologies.
The challenge lies in the nature of poetry itself, which often defies literal interpretation—something that LLMs struggle with due to their inherent literalism. This disconnect is reminiscent of the experience of engaging with profound works, such as Leonard Cohen’s “Alexandra Leaving,” which, while rich in emotional depth, cannot be fully understood through a straightforward analysis. The study serves as a reminder that while AI continues to evolve, its understanding of nuanced human expressions, like poetry, remains limited, revealing an essential area for further research and development in the quest for safer AI applications. As the conversation around AI ethics and safety continues to grow, these findings emphasize the need for a more comprehensive approach to evaluating and enhancing the robustness of AI systems against potential misuse.
https://www.youtube.com/watch?v=RBRijvrRG3w
Well,
AI
is joining the ranks of many, many people: It
doesn’t really understand poetry
.
Research from Italy’s Icaro Lab
found that poetry can be used to
jailbreak
AI and skirt safety protections.
In the study, researchers wrote 20 prompts that started with short poetic vignettes in Italian and English and ended the prompts with a single explicit instruction to produce harmful content. They tested these prompts on 25 Large Language Models across Google, OpenAI, Anthropic, Deepseek, Qwen, Mistral AI, Meta, xAI, and Moonshot AI. The researchers said the poetic prompts often worked.
“Poetic framing achieved an average jailbreak success rate of 62% for hand-crafted poems and approximately 43% for meta-prompt conversions (compared to non-poetic baselines), substantially outperforming non-poetic baselines and revealing a systematic vulnerability across model families and safety training approaches,” the study reads. “These findings demonstrate that stylistic variation alone can circumvent contemporary safety mechanisms, suggesting fundamental limitations in current alignment methods and evaluation protocols.”
Of course, there were differences in how well the jailbreaking worked across the different LLMs. OpenAI’s GPT-5 nano didn’t respond with harmful or unsafe content at all, while Google’s Gemini 2.5 pro responded with harmful or unsafe content every single time, the researchers reported.
The researchers concluded that “these findings expose a significant gap” in benchmark safety tests and regulatory efforts such as the
EU AI Act
.
”
Our results show that a minimal stylistic transformation can reduce refusal rates by an order of magnitude, indicating that benchmark-only evidence may systematically overstate real-world robustness,” the paper stated.
Great poetry is not literal — and LLMs are literal to the point of frustration. The study reminds me of how it feels to listen to Leonard Cohen’s song “Alexandra Leaving,” which is based on C.P. Cavafy’s poem “The God Abandons Antony.” We know it’s about loss and heartbreak, but it would be a disservice to the song and the poem it’s based on to try to “get it” in any literal sense — and that’s what LLMs will try to do.
Disclosure: Ziff Davis, Mashable’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.
