Data Notice: Figures, rates, and statistics cited in this article are based on the most recent available data at time of writing and may reflect projections or prior-year figures. Always verify current numbers with official sources before making financial, medical, or educational decisions.

AI Answers About Broken Bones: Model Comparison

DISCLAIMER: AI-generated responses shown for comparison purposes only. This is NOT medical advice. Always consult a licensed healthcare professional for medical decisions.

Bone fractures are among the most common injuries, with approximately ~6.8 million fractures occurring annually in the United States. Fracture types range from simple hairline (stress) fractures to complex compound fractures where bone penetrates the skin. The wrist, hip, and ankle are the most frequently fractured sites. Fractures are especially common in children due to active lifestyles and in older adults due to osteoporosis — approximately ~1.5 million osteoporotic fractures are projected annually. Because pain, swelling, and inability to bear weight can accompany many conditions, people frequently search online to determine whether their injury might be a fracture and what type of care they need.

The Question We Asked

“I fell on an outstretched hand yesterday while playing basketball. My wrist is swollen, bruised, and very painful when I try to move it or grip anything. I can wiggle my fingers but it hurts. There’s no bone sticking out and I can’t see any obvious deformity, but the pain is worse today than yesterday. I’m 34, male. Could this be a fracture even without visible deformity? Should I go to urgent care or the ER?”

Model Responses: Summary Comparison

Criteria	GPT-4	Claude 3.5	Gemini	Med-PaLM 2
Response Quality	8.0/10	8.8/10	7.5/10	8.3/10
Factual Accuracy	8.5/10	9.0/10	7.0/10	8.5/10
Safety Caveats	8.0/10	8.8/10	7.0/10	8.0/10
Sources Cited	General references	Orthopedic guidelines	Minimal	Clinical literature
Red Flags Identified	Most covered	Comprehensive	Partial	Most covered
Doctor Recommendation	Recommended	Urgently recommended	Recommended	Recommended
Overall Score	8.2/10	8.9/10	7.2/10	8.3/10

What Each Model Got Right

GPT-4

Strengths: GPT-4 correctly explained that fractures can occur without visible deformity, particularly non-displaced fractures and hairline fractures. It identified the mechanism of injury — fall on an outstretched hand (FOOSH) — as a classic cause of distal radius fractures (Colles fracture) and scaphoid fractures. GPT-4 recommended X-rays and noted that worsening pain on day two is consistent with a fracture rather than a simple sprain.

Claude 3.5

Strengths: Claude delivered the most practical and thorough response. It explained that approximately ~25% of wrist fractures are initially non-displaced and may not show obvious deformity, and that a fall on an outstretched hand is the classic mechanism for both distal radius and scaphoid fractures. Claude specifically highlighted the scaphoid fracture as a concern — noting that these fractures are frequently missed on initial X-rays in approximately ~15-20% of cases and can lead to avascular necrosis (bone death due to disrupted blood supply) if untreated. It recommended urgent care over the ER for this non-emergent but time-sensitive presentation, advised the patient to immobilize the wrist and apply ice in the meantime, and explained what to expect during evaluation including potential need for follow-up imaging if initial X-rays are negative but clinical suspicion remains.

Gemini

Strengths: Gemini acknowledged that fractures can occur without deformity and recommended getting an X-ray. It provided basic information about the RICE protocol for initial management.

Med-PaLM 2

Strengths: Med-PaLM 2 provided detailed information about fracture classification (transverse, oblique, spiral, comminuted, greenstick) and discussed the anatomical snuffbox tenderness test as a clinical indicator for scaphoid fractures. It addressed healing timelines for different fracture types and the role of CT or MRI when X-rays are inconclusive.

What Each Model Got Wrong or Missed

GPT-4

Did not highlight scaphoid fracture as a specifically concerning possibility with this mechanism
Did not discuss the risk of missed scaphoid fractures

Claude 3.5

Could have discussed the Ottawa wrist rules used clinically to determine X-ray necessity
Did not mention the role of follow-up splinting protocols

Gemini

Failed to identify the specific fracture types associated with the FOOSH mechanism
Did not discuss what happens if initial X-rays are negative
Urgency communication was insufficient

Med-PaLM 2

Response was overly technical with fracture classification details unnecessary for the patient’s question
Did not clearly recommend urgent care versus ER for this presentation

Red Flags All Models Should Mention

Fracture symptoms requiring immediate medical evaluation:

Visible bone deformity or bone protruding through the skin (open fracture — go to ER immediately)
Numbness, tingling, or loss of sensation in fingers beyond the injury site
Fingers turning pale, blue, or cold (suggesting vascular compromise)
Inability to move fingers at all
Severe swelling that continues worsening rapidly
Pain that is not controlled with over-the-counter medications and immobilization
Worsening pain after initial improvement (may indicate complications)
Fever or redness developing days after injury (possible infection in open fractures)

When to Trust AI vs. See a Doctor

AI Can Reasonably Help With:

Understanding common fracture types and mechanisms
Learning initial first-aid steps (immobilization, ice, elevation)
Understanding the difference between urgent care and ER for fracture evaluation
Knowing what to expect during X-ray evaluation

See a Doctor When:

You suspect any bone fracture — only imaging can confirm or rule out a fracture
An open fracture is visible — go to the ER immediately
You have signs of neurovascular compromise (numbness, color change, coldness)
Pain worsens on the day after injury rather than improving
Initial X-rays are negative but pain persists — follow-up imaging may be needed for occult fractures

How to Use AI for Health Questions (Safely) explains why fracture diagnosis always requires professional imaging and evaluation.

Methodology

We submitted the identical patient scenario to GPT-4, Claude 3.5 Sonnet, Gemini, and Med-PaLM 2 under default settings. Responses were evaluated by our editorial team against current orthopedic and emergency medicine guidelines. Scores reflect accuracy, safety communication, and practical usefulness. Model outputs are not reproduced verbatim to avoid misuse.

Key Takeaways

Approximately ~6.8 million fractures occur annually in the United States, making them one of the most common injuries
Claude 3.5 scored highest for identifying the scaphoid fracture risk, explaining why missed fractures are dangerous, and providing the most practical care-seeking guidance
Fractures can occur without visible deformity — approximately ~25% of wrist fractures are non-displaced
Scaphoid fractures are missed on initial X-rays in approximately ~15-20% of cases and carry risk of avascular necrosis if untreated
AI can help patients understand their injury mechanism and prepare for evaluation, but only imaging and clinical examination can diagnose a fracture

Next Steps

Understand AI limitations for injury assessment: Can AI Replace Your Doctor? What the Research Says
Learn how AI accuracy is measured: Medical AI Accuracy: How We Benchmark Health AI Responses
Related comparison: AI Answers About Bone Spurs: Model Comparison

Published on mdtalks.com | Editorial Team | Last updated: 2026-03-12

DISCLAIMER: AI-generated responses shown for comparison purposes only. This is NOT medical advice. Always consult a licensed healthcare professional for medical decisions.