Artificial intelligence has made enormous strides during the past few years, but it's probably not ready to try and save your life.
According to Yahoo! News, the online patient portal MyChart has run into serious problems with an AI-powered tool that helps clinicians respond to patient queries. When MyChart receives a question, the AI assistant, called In Basket Art, automatically drafts a response, which clinical staff will edit, approve, and send.
In Basket Art is intended to help doctors save time and allow them to devote their mental energy to more pressing tasks. But users have reported that the feature often hallucinates, generating inaccurate and sometimes dangerous information that can be inadvertently passed on to patients. One primary care physician in North Carolina reported that the bot provided a patient with information about her vaccine history — though it did not even have access to her vaccine records.
According to the New York Times, these errors are not altogether uncommon. A recent study examined a batch of 116 AI-generated drafts and found that seven of them contained hallucinations. Another study found that the unedited drafts contained potentially harmful advice about 7 percent of the time.
And patients may be none the wiser, since physicians, health systems, and MyChart are not required to notify patients when providers send AI-generated messages.
Epic, which owns MyChart, says it has refined In Basket Art to prevent it from providing clinical advice, but it's not perfect, and users report continued problems with In Basket Art's responses.