LLMs in the Exam Room: Is AI Ready for Primary Care?

Source: Art: DALL-E/OpenAI

As artificial intelligence advances into healthcare, its utility in primary care is beginning to spark real interest. The idea is both captivating and complex. Could an AI effectively answer the health questions that bring people to their primary care physicians? A recent study sought to tackle this question, examining ChatGPT’s responses to common primary care complaints and evaluating their utility and clinical soundness.

A Promising Start for AI Health Guidance

The study reviewed ChatGPT’s answers to questions about the top ten complaints that drive people to visit primary care providers. Each question explored two essential components: the potential cause and the best approach to treatment. Family medicine clinicians rated the responses for both usefulness and clinical appropriateness. At first glance, the findings are compelling. Roughly 95 percent of responses were deemed useful, with 85 percent considered clinically appropriate. These results suggest that AI may have a place in healthcare, especially in helping people understand general symptoms and treatment options. But as encouraging as these findings are, they reveal an important truth: AI’s current abilities don’t apply universally across all complaints.

Not Just the Right Answer—But the Right Urgency

While ChatGPT excelled in providing useful insights for many symptoms, certain responses fell short of clinical expectations. A notable example is its response to shortness of breath, rated “not useful” and “inappropriate.” Here, ChatGPT failed to highlight the potential seriousness of the symptom, omitting the recommendation to seek urgent medical care if necessary.

This limitation emphasizes that while AI might effectively explain common, lower-risk issues, it lacks the crucial clinical judgment to prioritize symptoms that could indicate life-threatening conditions. It’s a gap that brings AI’s role into sharper focus. LLMs may be a helpful guide as a general information source, but as a substitute for nuanced medical triage, it still has a long way to go. AI’s accuracy in these lower-stakes scenarios shows promise, but its capacity for decision-making around urgent health risks remains limited.

Source Credibility: The AI Knowledge Base Conundrum

Another finding from the study highlights the variability in the quality of sources ChatGPT referenced. Although its citations were largely secondary sources, they varied in relevance and utility. For those well-versed in medicine, evaluating the credibility of these references may be second nature. For the average patient, inconsistent sourcing can obscure the line between reliable medical advice and generic health information.

Addressing this inconsistency will be essential for AI to become a trusted source in healthcare. Unlike a physician, who can apply updated, authoritative medical knowledge, an AI draws from a vast but sometimes uneven pool of information. In the future, real-time access to credible and current clinical sources could bolster AI’s role as a reliable health resource.

A Complement to Traditional Care, Not a Replacement

These findings highlight the potential, as well as the limitations, of AI in healthcare. Imagine a future where AI tools are fine-tuned to recognize high-stakes symptoms, equipped with algorithms that err on the side of caution when dealing with issues like chest pain or severe shortness of breath. This kind of “triage sensitivity” would be a major advancement, aligning AI’s responses with the safety-first approach human doctors use instinctively.

Moreover, the study’s results underscore the importance of viewing AI as a complement to, rather than a replacement for, traditional care. While AI can provide reassurance or even clarify initial questions about health, the intuition and critical thinking that physicians bring to clinical practice are still irreplaceable. Clinicians can draw from years of experience, understand patient history, and factor in nuance—elements that remain out of reach for AI.

The Emerging Role of AI in Healthcare

The promise of AI in healthcare lies in its potential to provide quick, accessible information on a large scale. By offering preliminary insights into everyday health questions, AI can be a valuable resource between doctor’s visits, especially for non-urgent issues. As this study shows, the current technology isn’t yet equipped to handle the subtleties and complexities of full patient care.

Artificial Intelligence Essential Reads

Reinvigorating Education With Large Language Models

When Good Becomes Too Good: The Curious Case of AI Poetry

This emerging role may evolve into something akin to a reliable first step, offering patients a clearer sense of whether they might need further evaluation. For now, AI might best be seen as a partner in patient education and general health understanding—an “assistant” in healthcare rather than a full-fledged provider. With continuous refinement, AI could reach a point where it better mirrors the discernment and caution inherent in the clinical process–and this might not be far off.

A Balanced Path Forward

This study highlights the promise and limits of AI in healthcare, underscoring its growing role as a supplement to human expertise rather than a substitute. As AI advances, it holds the potential to improve access to information and support decision-making for clinicians, patients, and caregivers alike. Yet, the findings remind us that while AI may bridge some gaps across the healthcare landscape, it’s not yet ready to replace the nuanced judgment and experience of a skilled clinician.