LLMs also hallucinate in medical contexts

This shouldn't surprise anyone, but it turns out LLMs also make up stuff when used by doctors:

[Professors Allison Koenecke and Mona Sloane] determined that nearly 40% of the hallucinations were harmful or concerning because the speaker could be misinterpreted or misrepresented.

(From: Researchers say AI transcription tool used in hospitals invents things no one ever said | AP News)

The article lists some examples: the tools made up violence, racial details and medication out of thin air.

Hidde de Vries (@hdv@front-end.social) is a web enthusiast and accessibility specialist from Rotterdam (The Netherlands). He currently works with the NL Design System team and is a participant in the Open UI Community Group. Previously, he worked for W3C (WAI), Mozilla, the Dutch government and others as a freelancer. Hidde spoke at 71 events, most recently in Amsterdam, Netherlands.

Buy me a coffee Follow @hdv on Mastodon