PrismML's extreme quantization, between efficiency promises and missing verification

Shrinking a 27-billion-parameter model so that it runs in the memory of a personal computer or on a single graphics card, without giving up what it can do, is one of the most worked-over themes of recent research. On 17 September 2026 PrismML published on its own site the release of Ternary Bonsai 2 27B, a multimodal model distributed under an Apache 2.0 licence and obtained by compressing Qwen3.8 27B. The design uses ternary weights limited to the values -1, 0 and +1 in a fixed rotated basis, alongside half-precision scaling factors. According to the official announcement, the operation brings the main variant down to 5.9 gigabytes at 1.76 effective bits per weight, against the 53.80 gigabytes of the model it started from. On Hugging Face, though, the technical model card reports the numbers of a different artifact: the 2-bit MLX version, which takes 8.60 gigabytes in total on disk — 0.92 of them for the unquantized vision tower — at 1.72 effective bits per weight. The comparison with the 5.9 gigabytes in the announcement is therefore not like for like, and the advertised figure refers to the lightest configuration.

On performance too, the company's own data fail to line up. The announcement on the PrismML site credits the model with an average score of 83.9 across a series of 20 benchmarks in reasoning mode, equal to 98.2% of the aggregate average of the source model; the model card on Hugging Face arrives at the same 98.2% but derives it from 14 tests, with an average of 84.78 against the original's 86.32. As the specialist outlet MarkTechPost pointed out on 18 September 2026, these are in every case internal measurements, not independently reproduced by any third party. The external analysis also finds that the drop becomes more pronounced on complex agentic tasks, where the model stops at around 75% of the original score on references such as Terminal-Bench and SWE-bench. PrismML's own documentation admits, for that matter, that the decline mostly affects the knowledge, reasoning and vision categories.

A further constraint concerns the infrastructure needed to run it. The architecture cannot be loaded through the standard libraries: the model card makes clear that MLX's ordinary loading modules do not support the Hadamard rotation transforms and the inverse embedding the format uses, which makes PrismML's custom runtime or its llama.cpp fork mandatory. On speed, the company announcement gives up to 142.5 tokens per second on an NVIDIA GeForce RTX 5090 and 46.8 tokens per second on an Apple M5 Max, while the Hugging Face card assigns the same GPU a lower figure, around 129 tokens per second. Neither source states the configuration the measurement was taken on. The company, which on its site presents itself as founded by Caltech researchers and lists Khosla Ventures, Cerberus, Google and Samsung among its backers, credits the system with 0.714 milliwatt-hours per token on an RTX 4090, which by its own account would be 40% more efficient than a full-precision 8-billion-parameter model.

Until the numbers are reproduced by others, the only solid figure left is the footprint on disk — and the dependency on a proprietary llama.cpp fork, which limits what the Apache 2.0 licence is worth in practice.

— Olya