Diagnostic Accuracy of Physicians vs Computer Algorithms

HealthDay News – Physicians are twice as likely to get the right diagnosis on the first try as 23 popular symptom-checking computer programs, according to a research letter published online in JAMA Internal Medicine.

The study involved 234 physicians and 23 computer symptom checkers. They were presented with 45 vignettes involving hypothetical patients and were asked to determine the illness each person likely had. The symptom checkers included web offerings from places like the Mayo Clinic, the American Academy of Pediatrics, and England’s National Health Service, as well as smartphone apps for the iPhone and Android.

Doctors provided the correct diagnosis immediately 72.1% of the time, compared with just 34.0% of the time for symptom-checking programs, the researchers found. Physicians also outperformed computers when given the chance to provide three suspected diagnoses. The correct diagnosis was in their top-3 list 84.3% of the time for doctors, and 51.2% of the time for system-checking programs. 

For simpler health problems such as conjunctivitis and sinusitis, computers guessed right 40.5% of the time, compared with 65.3% of the time for physicians. Human doctors performed 3 times as well for very complicated health problems, getting the diagnosis right 79.1% of the time, compared with 24.% of the time for the computer.

“While in this project we compared diagnostic performance, future work should test whether computer algorithms can augment physician diagnostic accuracy,” the authors wrote.

Reference

Semigran HL, Levine DM, Nundy S, et al. Comparison of physician and computer diagnostic accuracy. JAMA Intern Med. 2016. Oct 10. doi:10.1001/jamainternmed.2016.6001 [Epub ahead of print]