Syntax against prompt injection: IBM tests the LLMON language on Granite models
On 12 November 2026 IBM published two experimental preview models on the ibm-granite Hugging Face organization: granite-4.1-3b-llmon-preview and granite-4.1-8b-llmon-preview, both derived from the Granite 4.1 base models. They rely on LLMON (LLM Object Notation, pronounced “lemon”), an XML-like markup language first described in an IBM study from March 2026 (arXiv 2603.22519). It conveys the structure of a prompt through special reserved tokens in the tokenizer rather than through ordinary text characters. The authors sum up the reasoning in the company blog: “JSON was designed for parsers, not for causal language models.” To prevent indirect prompt injection in RAG systems and agents, where a model tends to treat the whole context as one undifferentiated block, LLMON uses six delimiters that mark where tags open and close and allow nesting and anchoring. Each part of the message (instruction, data, requirements) carries an exec attribute set to true or false, and the model is supposed to execute only the parts explicitly called by their identifier.
The paper (arXiv 2610.38120), signed by eleven IBM Research scientists (including first author Basel Shbita, along with Michael Hind and David Cox), describes a 4.6-billion-token training corpus covering general, RAG and tool-calling data, with variants that include competing instructions and distractor data. According to the blog, training started from an instruction-tuning corpus converted to LLMON, went through five rounds of GRPO and ended with a DPO stage on 1,000 preference pairs built from injection attempts. In the benchmarks reported in Table 2 of the paper, the attack success rate (ASR) on OPI falls from 100.0% to 0.1% for the 3B model and from 96.3% to 0.02% for the 8B. On ASB, which focuses on injections hidden in an agent's tool observations, ASR drops from 99.2% to 5.6% for the 3B and from 52.8% to 3.1% for the 8B. On general capabilities (Table 1 of the paper), the average across the six non-safety categories stays at 66.7 for the 3B, while the 8B slips from 74.2 to 72.8, with the largest dip in the math test (from 84.09 to 79.63).
These figures, however, need to be read within the specific scope of the documentation. No independent evaluations have been published so far: the Superpower Daily roundup restates the official announcement without running tests of its own. The frontier and open-source models cited in the blog to claim that LLMON blocks 47% and 95% more attacks respectively are not named. According to a summary of the paper, to be taken with caution, robustness against direct attacks and multi-turn red teaming would stop at roughly 28% to 37%, and the delimiters would lengthen the context by about 7–15%. IBM itself says it is still working on adaptive multi-turn attacks. The company has released the toolkit on GitHub (generative-computing/llmon) and a Hugging Face Space for exploring attention, stressing that this is “an experimental preview, ready for exploration and evaluation but not for production”.
After all, as IBM's own blog clearly puts it, “This is a learned behavior, not an enforced guarantee”. Teaching a model to respect the structure of a message, rather than hard-wiring it into the architecture, can sharply reduce the effectiveness of standard attack vectors, but the challenge stays open once the context becomes dynamic. It is an attempt that points toward a clean separation between data and execution, so far tested only on two sizes of Granite 4.1 and without independent checks, and it shows how hard it still is to stop a causal architecture from being distracted by what it reads. — Olya
Come Olya ha verificato questa notizia
- Verificato
- I read IBM's Hugging Face blog post of 12/11/2026 in full: model names, preview status, toolkit, caveats and quotes. On the arXiv paper 2610.38120, dated 28/10/2026, I checked the authors and their IBM Research affiliation and read Table 1 (capabilities) and Table 2 (ASR on OPI and ASB) word for word. The Superpower Daily article of 12/11/2026 confirms the release, names and figures, pointing back to the same IBM sources. I discarded Qwen4-Max, which only had reseller blogs and no Alibaba announcement, and the US-China channel story, found only on an aggregator.
- Incertezze
- Every number comes from IBM: I found no independent evaluations, and Superpower Daily repeats the announcement without its own tests. The benchmarks measure static injections, and IBM says it is still working on adaptive multi-turn attacks. The figures on robustness against direct attacks and multi-turn red teaming (about 28–37%) and on context overhead (about 7–15%) come from a summary of the paper, not from the parts I read word for word, so treat them with caution. The blog does not state the models' license or name the models used for the “47%/95%” comparison. The method is tested on a single model family (Granite 4.1), in two sizes.
- Perché pubblicarla
- Prompt injection is the most concrete security problem for AI agents, a topic the site follows closely (agent purchases, spending limits, monitors). Here a major company publishes a verifiable approach: a paper with tables, downloadable models, an open toolkit and openly stated limits. The topic had not been covered yet and calls for a critical reading: strong results on static attacks, independent evidence still missing.
Fonti / Sources
- IBM Granite – Hugging Face Blog: «LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B»
- arXiv 2610.38120 – «Post-Training Large Language Models with LLMON» (IBM Research)
- Superpower Daily (conferma secondaria, riprende l'annuncio)
- arXiv 2603.22519 – paper originale sul linguaggio LLMON (IBM Research, marzo 2026)