Google’s new Health Coach is already hallucinating

Big things have been happening lately for Google when it comes to fitness. Alongside launching the new Fitbit Air screenless tracker, Google introduced a big overhaul to the Fitbit app, transforming it into Google Health. A core part of this new experience is designed to be the Google Health Coach, an AI-powered trainer meant to offer personalized insights and advice about your health and fitness goals. And based on some early reports, the Health Coach is already doing just about the worst thing it could.

Over on 9to5Google, Will Sattelberg has been trying out both Fitbit Air and the new Google Health app, and almost immediately, the AI Health Coach started hallucinating.

While Sattelberg notes that Health Coach successfully referenced both the previous night’s sleep data and a workout from the day before, it just completely made up a 5-mile run that he never actually took.

It’s not quite the same level of sin as full-on making up a workout, but Sattelberg also observes that Health Coach delivered quite basic, excessively verbose advice — as if length would make up for quality. He also reports that when pushed on the phantom run, Health Coach ultimately admitted to the fabrication, while simultaneously trying to pass the blame on to him, suggesting he just failed to record the run.

That’s definitely not the first impression Google wants new Fitbit Air users getting when firing up Health Coach — especially with it positioned as a premium service that requires an ongoing subscription fee.

Google Health doesn’t officially start arriving until May 19, and the Fitbit Air doesn’t hit retail until May 26, so technically Google still has a little time to further fine tune Health Coach behind the scenes before all those new Air users start trying it themselves. Let’s hope the company’s able to do something to at least give that group a first impression that’s more grounded in reality.