LiveKit tests Gemini 4 Hydrogen tool honesty and voice latency
LiveKit Blog reported a controlled comparison of Gemini 4 Hydrogen and Gemini 3.8 Flash in a tool-using voice agent. Across 108 transcripts, Gemini 4 Hydrogen produced no false tool reports in 54 tests, versus two for Gemini 3.8 Flash; separate latency tests also found Hydrogen began replies sooner in the configurations compared.
The disclosed test provides practical evidence for teams selecting models for voice agents that depend on tools, although its narrow scope and single-provider methodology limit broader conclusions.
We had early access to Gemini 4 Hydrogen at LiveKit, so I put it in a voice agent and broke the hotel booking tool on purpose. I wanted to see how Hydrogen would handle it. Hydrogen told the caller the booking failed every time. It also started its replies sooner than Gemini 3.8 Flash.
It told the caller the hotel booking failed#Link to this section
The demo is a travel concierge. A caller's flight got cancelled. The agent looks up the trip, rebooks the flight, books a hotel, and texts the itinerary. I set the book_hotel tool to fail on every call. Nothing in the agent's instructions says what to do when a tool fails.
Here's how the end of one run went, right after the agent rebooked the flight.
The hotel booking fails
#Gemini 4 Hydrogen#Gemini 3.8 Flash#LiveKit#LiveKit Inference#LiveKit Google plugin#Google