Research in The National Cancer Journal Institute The spectrum of cancer looked at chatbots and artificial intelligence (AI) as they become popular resources for cancer information. They found that these resources provided accurate information when asked about common myths and misconceptions about cancer. In the first study of its kind, Skyler Johnson, MD, a physician-scientist at the Huntsman Cancer Institute and associate professor of radiation oncology at the University of Utah, evaluated the reliability and accuracy of ChatGPT’s cancer information.
Using the National Cancer Institute’s (NCI) common myths and misconceptions about cancer, Johnson and his team found that 97% of the answers were correct. However, this finding comes with some important caveats, including the team’s concern that some ChatGPT responses may be misinterpreted. “This can lead to wrong decisions by cancer patients. The team suggested caution when counseling patients about whether they should use chatbots for cancer information,” says Johnson.
The study found that the reviewers were blinded, meaning they did not know whether the responses came from the chatbot or the NCI. While the responses were accurate, reviewers found ChatGPT’s language to be indirect, vague, and in some cases unclear.
“I recognize and understand how difficult it is for cancer patients and their caregivers to access accurate information,” Johnson says. “These sources need to be explored so that we can help cancer patients navigate the murky waters that exist in the information-interactive environment as they try to find answers about their diagnoses.”
Incorrect information can harm cancer patients. In a previous study by Johnson and his team, published in Journal of the National Cancer Institutethey found that misinformation was common on social media and could harm cancer patients.
The next steps are to assess how often patients use chatbots to search for cancer information, what questions they ask, and whether AI chatbots provide accurate answers to unusual or uncommon cancer questions.
The study was supported by the National Institutes of Health/National Cancer Institute, including P30 CA042014, and the Huntsman Cancer Foundation.