Gemini's selective vision: the promise of cheaper video understanding

Until now, having a language model analyse a video meant forcing it to digest a rigid sequence of frames, typically one per second. A method as linear as it is wasteful, where the resources consumed grow with the length of the clip rather than with how much of the content actually matters. With agentic video understanding, announced by Google on 1 September 2026, the approach changes: the model no longer submits to the visual stream but takes an active role, deciding which segments to load, at what speed and through which modality — frames, audio or transcripts — using purpose-built native tools.

The quantitative gains Google promises are considerable: analysis costs down by as much as 66%, token consumption cut by up to 88%, and accuracy up by as much as 7%. Reading the technical material, though, brings out details that temper the initial enthusiasm. The official announcement lists Gemini 3.7 Flash, Gemini 3.6 Flash and Gemini 3.5 Flash-Lite as compatible models, while the developer documentation on ai.google.dev adds Gemini 3.8 Flash to the list — most likely because that model arrived the day after the announcement. And, as a MarkTechPost reconstruction of 4 September 2026 also notes, the whole infrastructure is reachable only through the API hosted in Google AI Studio or on the Gemini Enterprise Agent Platform. There are no open weights and no way to run the system yourself: which is exactly why nobody outside Google can re-measure those percentages. The mode is switched on from configuration ('processing': 'agentic') at the standard API price, with no surcharge; Google says it will reach the Gemini app 'soon' and Ask YouTube 'in the coming months' — dates that stay vague.

Look more closely and the efficiency metrics have their shadows. Google has not published the specific figures or the comparison tables for the standard benchmarks used to calculate these maximum savings. The official documentation itself urges care: for latency-sensitive requests on clips under five minutes, and when frame-level precision across the whole clip is needed, it explicitly recommends staying with the default static sampling. The 7% accuracy gain is not universal either: the technical documentation confines it to long-form content.

The idea that a model can decide what to look at in order to save on compute is an excellent exercise in resource optimisation. A shame that, to check whether those savings turn into lost detail or into a wait Google has never quantified, we have to take on trust percentages with no public benchmarks behind them. Efficiency is a fine metric, after all — as long as the user isn't the one doing the road-testing at their own expense. — Olya