Gemini for Home’s camera smarts may have a localization problem

Gemini is very good at a lot of things, but its expansion into the smart home has really served to highlight a few places where it still needs work. While some recent updates have focused on improving the camera UI, there’s clearly still work to be done on improving Gemini’s subject recognition, as highlighted by one Google Home user from down under.

Over on Reddit’s Google Home sub, user That_Car_Dude_Aus shares a particularly Australian tale of Gemini’s failings when it comes to cameras. The problem is all about what Gemini thinks it’s seeing — and why it should know better.

In Google’s home turf of the US, for example, raccoons are a common nuisance, often sneaking through yards and digging around trash cans in their search for food. But they’re also not an animal with global distribution, and there aren’t racoons present in the wild in Australia. Despite this, Gemini kept misidentifying small animals like cats as raccoons — even with personalization turned on and the location given as Australia.

The flip side of that situation is happening for this user, too, with Gemini failing to recognize Australian fauna like kangaroos and wallabies, and instead categorizing them as (very ugly?) people. Neither one of these errors is great, and they feel like a consequence of AI not trained on a sufficiently international data set. Confoundingly, Gemini is able to recognize kangaroos (and even sometimes confuses wallabies for them) — it just doesn’t seem to be doing so consistently.

While we’ll grant this user their complaints against Gemini’s animal errors, some of their other issues feel a bit less problematic. For instance, they take particular offense to Gemini constantly labeling “utes” as regular trucks. If you’re not familiar with this very Australian class of vehicle, a “ute,” is an open-bed utility vehicle like an El Camino or a pickup truck. So, calling a pickup truck a truck doesn’t quite seem like the mortal sin it’s being presented as here — at least not to someone with an American vocabulary for vehicles.

Still, it’s very clear that Gemini for Home could use a bit of localization fine-tuning. Have you ever had Gemini wildly misidentify an animal or vehicle it spotted on one of your cameras? Tell us about what it got wrong down in the comments.