Study: harmful AI health advice leaves no trace, urges reporting system
A new study reported by TechNews warns that when users act on harmful medical advice from chatbots, the conversations typically stay inside the AI company's systems, leaving outside clinicians, researchers and regulators unable to review them and without a standard reporting or investigation channel. The researchers cite a case in which a 60-year-old man who asked ChatGPT how to cut chloride in his diet was advised to replace table salt with sodium bromide, then developed hallucinations and delusions and was hospitalized with bromide poisoning; because his doctors could not directly access the chat, tracing the cause and planning treatment was extremely difficult. Co-author Kai-Cheng Yang, an assistant professor at Binghamton University, said many people already entrust health questions to AI with no mechanism overseeing the answers, and the team recommends that AI companies let users retrieve and share their chat logs with physicians, that the industry build a transparent reporting and investigation system, and that social platforms label AI-generated health content.
According to the study, the absence of externally accessible records makes it hard for doctors to establish what information a patient received, which the authors say hampers medical judgment and treatment.


隨著 AI 聊天機器人逐漸成為大眾 24 小時隨身待命的健康顧問,研究人員發出警告:最令人頭痛的危機並非 AI 給出錯誤答案,而是這些致命錯誤幾乎「無跡可尋」。
最新研究指出,當使用者聽信了聊天機器人有害的醫療建議,對話紀錄往往只停留在 AI 公司的系統內。外部的臨床醫師、研究者及監管單位根本無從查閱,更缺乏標準的通報與調查管道。研究作者強調,這種「隱形風險」導致醫師在事後極難釐清病患到底接觸了什麼資訊,進而阻礙了醫療判斷。