The growing integration of AI tools, particularly public chatbots like ChatGPT, into clinical practice has sparked increasing debate. While some doctors are using these tools to assist in clinical decision-making, the trend raises important questions about accuracy, patient safety, and ethical considerations. While these tools are not designed specifically for medical use, some healthcare professionals have started incorporating them into their practice for tasks like drafting patient communications, suggesting diagnoses, or reviewing complex cases. This trend raises important questions about whether the public should be concerned.

AI tools like ChatGPT can process vast amounts of information quickly, potentially offering time-saving solutions for overburdened healthcare systems. Studies have shown that AI can sometimes rival or even exceed human accuracy in certain diagnostic areas, such as eye disease or cancer detection when used in specialized contexts. For example, AI models specifically trained for medical purposes have demonstrated high accuracy in diagnosing glaucoma and analyzing mammogram scans. These results suggest that AI could help improve efficiency and accuracy in some areas of healthcare.

However, AI chatbots designed for general use, like ChatGPT, present a range of concerns when used in clinical settings. One of the primary issues is accuracy. ChatGPT, while often providing informative responses, has been known to make mistakes, sometimes providing incomplete or inaccurate medical advice. According to a Pew Research Center survey, 60% of Americans feel uncomfortable with the idea of their healthcare provider relying on AI for their medical care​.

ChatGPT can misinterpret nuanced medical data, leading to incorrect diagnoses or treatment suggestions, which could be dangerous in real-world clinical scenarios. Additionally, the lack of regulation and standardized guidelines for how AI should be used in medicine means there is no clear framework to ensure patient safety or data privacy.

Another critical concern is limited training, AI models, including ChatGPT, are not trained on up-to-date medical guidelines. They draw from a broad dataset that might not always reflect the latest medical standards or specific patient needs. This raises questions about liability, especially if AI tools are used inappropriately without proper oversight.

A notable concern is bias in healthcare, 64% of Black adults say that bias based on race or ethnicity is a major problem in health and medicine, compared with smaller shares of White adults (27%), Hispanic adults (42%), and English-speaking Asian adults (39%). This highlights a deep-rooted concern that AI could either perpetuate or reduce bias in healthcare​.

Interestingly, those who are concerned about racial and ethnic bias in healthcare tend to be more optimistic about AI’s potential to address the issue. The Pew survey revealed that 51% of respondents who acknowledge bias in healthcare believe that increasing AI’s role could help reduce it, compared to only 15% who believe it would make the problem worse. Many argue that AI could be more neutral or objective than human clinicians, as it is not influenced by personal prejudice​.

However, there is also skepticism. Around 28% of those who believe AI would have little impact on bias argue that AI tools may inherit the biases present in the data they are trained on. Others are concerned that the developers of AI systems might introduce their own biases, limiting the tool’s ability to provide fair treatment​.

The potential for AI to address bias in healthcare is a double-edged sword. Some respondents to the Pew survey noted that AI’s objectivity could eliminate human bias, such as being “blind” to a patient’s race or ethnicity. However, critics worry that AI might perpetuate existing inequities if it is trained on biased datasets, potentially leading to unfair treatment based on race or ethnicity. Approximately 28% of those who believe AI will exacerbate bias argue that human oversight and personalized care are still necessary and that AI lacks the nuanced understanding required for equitable patient care​.

While many people are opposed to the idea of doctors using AI chatbots for diagnosing patients, there are also significant reservations about chatbots designed for specific health issues, such as mental health. These specialized AI tools, often developed to provide mindfulness check-ins or automated conversations, aim to complement or even replace traditional therapy in some cases. A large majority of U.S. adults (79%) have expressed discomfort with the idea of using AI chatbots for their own mental health support, according to a Pew Research Center survey. Even when chatbots are tailored to specific challenges, like mental health, many people question their effectiveness and believe they should only be used alongside professional therapy​.

According to a Pew Research Center Survey, 46% of U.S. adults think that mental health chatbots should only be available to individuals who are already working with a licensed therapist, while 28% argue that such chatbots should not be available at all. This hesitancy suggests that, even when chatbots are developed for particular health needs, people still have doubts about their ability to provide the nuanced, empathetic care that human professionals offer.

While the use of AI chatbots in healthcare has potential, their use in clinical decision-making without specialized medical training can be problematic. Medical professionals should ensure that AI is used ethically and as a supplementary tool rather than a replacement for human expertise. Regulatory bodies are starting to recognize the need for clear guidelines on AI’s use in healthcare, which could help address some of these concerns in the future.

Overall, AI in healthcare holds promise, but the current use of general AI chatbots for clinical decisions requires careful consideration and strict oversight to protect patient safety.