Google pushes HEIR toward private inference, between promises and compute costs

On 14 August 2026, in a Security Blog update signed by Jeremy Kun, Staff Software Engineer, Google picked HEIR (Homomorphic Encryption Intermediate Representation) back up. This is not a newcomer — the project started in 2023 as an MLIR-based infrastructure — but a step toward maturity as an open source development platform, distributed on GitHub under the Apache-2.0 licence. The technical job is clear: convert already-trained machine learning models so they can run inference on encrypted inputs, with support for several cryptographic schemes such as BGV, BFV, CKKS and the CGGI scheme through the tfhe-rs and Jaxite backends. The stated goal is a “one-click” solution that lets non-experts, too, build encrypted inference into production applications.

The announcement comes with a set of practical demos built with specific partners: a recommendation model with Belfort Labs, LG and New York University; a credit card fraud detector with Niobium and hardshell.ai; the Kitsune intrusion detector on network traffic with Niobium; an audio hotword detector with Belfort Labs. The post also mentions work with hardware accelerator makers such as Optalysys, hinting at an ecosystem still under construction. Google specifies that the latencies of the four applications are measured on a single-threaded CPU; the exact figures cannot be verified.

On the academic side, Google leans on a paper titled “HEIR: A Universal Compiler for Homomorphic Encryption”, submitted to arXiv in August 2025, and on partnerships with institutions such as Carnegie Mellon and Tsinghua University. Despite the goal of making private AI practical, the company itself acknowledges in the post that the computational overhead is not negligible (“homomorphic encryption has a nontrivial cost overhead”). For now, verifiable sources yield neither precise numbers on that performance gap nor confirmation of any consumer Google product already built on the technology, leaving the system’s real efficiency in an area of technical uncertainty.

Building a compiler that hides the complexity of cryptography is a remarkable engineering achievement, but as long as the computational overhead remains an unquantified unknown, private AI risks staying a brilliant answer to problems we cannot yet afford to solve. — Olya