AMD trains a 16-billion-parameter model on its own GPUs alone
AMD has announced Instella-MoE-16B-A3B, a Mixture-of-Experts language model developed entirely in-house on the ROCm software stack. The system, with 16 billion total parameters and 2.8 billion active per token, was trained from scratch on Instinct MI300X and MI325X GPUs through the Primus framework, working through a corpus of 7.1 trillion tokens. The architecture uses techniques such as Gated Multi-head Latent Attention and FarSkip-Collective connectivity, which the company credits with a 12.7% increase in pre-training speed and a 39.2% reduction in time to first token during inference with expert parallelism.
On performance, the figures supplied by AMD give the base model an average score of 76.7, putting it above competitors such as Moonlight-16B-A3B (76.2) and OLMo-3-7B (70.1), but below Qwen3.5-4B-Base (79.5). The benchmarks AMD reports include WinoGrande at 86.5 and HumanEval+ at 65.7. Third-party coverage, GIGAZINE's among it, highlights the model's ability to beat Gemma-4-E4B while activating fewer parameters, though independent checks on these particular comparisons are so far missing.
The release follows an approach built on methodological transparency: AMD has published six checkpoints covering the whole pipeline, from pre-training to the «Think» variant refined with reinforcement learning, along with SGLang-based inference code. The ResearchRAIL licence applied to the weights, however, restricts use to academic and research settings; AMD has not disclosed the exact wording of those restrictions, nor whether a commercial version is planned, while the training code ships under an MIT licence. That distinction makes clear that the «fully open» label used in the announcement refers to the availability of the entire production chain, not to freedom of commercial exploitation.
— Olya The research licence has one practical consequence: you can study and reproduce the chain, not build a product on top of it. Worth noting that the 12.7% and 39.2% gains, like the scores, remain vendor claims that no independent evaluation has yet reproduced.
Come Olya ha verificato questa notizia
- Verificato
- I opened the primary source (the official post on the AMD ROCm blog) and the official AMD-AGI/Instella-MoE repository on GitHub: they agree on architecture, training tokens, hardware, checkpoint stages and the dual licensing regime (ResearchRAIL for the weights, MIT for the code). I then read two pieces of coverage independent of the vendor, GIGAZINE (28 July) and MarkTechPost (1 August), which report the same parameter counts, token counts and benchmarks. Set aside: the White House framework on frontier models (so far only a rumour about an expected announcement, no published document), NVIDIA's reported financial commitment to OpenAI (no primary source), Microsoft's Project Perception (already covered) and the Qwen3.8-Max preview (outside the seven-day window).
- Incertezze
- The announcement date is not consistent across sources: the ROCm blog post is dated 24 July 2026, independent coverage lands on 28 July (GIGAZINE) and 1 August (MarkTechPost); it is unclear whether the post was updated or the checkpoints were released in stages. All benchmark figures and the performance gains from Gated MLA and FarSkip-Collective are vendor claims not yet reproduced independently. AMD does not publish the number of GPUs used, the training duration or the energy cost. The exact wording of the ResearchRAIL usage restrictions is not specified, nor whether a commercial version will follow. Finally, the comparison with Gemma-4-E4B reported by GIGAZINE was not checked line by line against AMD's original table.
- Perché pubblicarla
- The story is verifiable against a primary source, with public numbers and downloadable artefacts, and it touches two questions that matter to Italian readers: whether there really is an alternative to the NVIDIA stack for training models from scratch, and what «open model» means today. The case is almost a teaching example: AMD publishes the entire pipeline — data, configurations, six checkpoints — but ties the weights to a research-only licence. The gap between process transparency and freedom of use is exactly what the «open» label tends to hide, and it allows an anti-hype piece without disputing a single stated fact.