Microsoft's AI Beats Doctors at Diagnosis by Inventing a Panel of Imaginary Doctors, Which Is Also How Most Hospitals Already Feel

Medicine has finally found a doctor who reads the whole chart. Unfortunately, it is several doctors, none of them exist, and they all work for Microsoft.

According to a study published Thursday in Science, Microsoft's diagnostic system, MAI-DxO, solved 83.2% of 304 fiendish New England Journal of Medicine cases. Generalist physicians with textbooks and the internet managed 36.1%. Without the internet, they managed 22%, which is roughly the success rate of a patient diagnosing himself at 2 a.m.

The trick is called an "orchestrator." Instead of one AI blurting an answer, it simulates a panel of virtual physicians who propose diagnoses, challenge each other and debate over several rounds. Humanity spent centuries trying to get doctors to consult a second opinion. Microsoft simply built the second opinion, the third opinion and the colleague who always says "have we ruled out lupus?"

It is the first medical conference with no buffet. Everyone argues, nobody golfs, and the keynote is a spreadsheet.

The system also weighs each test's cost before ordering it, an act of financial restraint never before witnessed in American healthcare. Its unconstrained version averaged $5,431 per complex case. In a U.S. hospital, that is not a diagnosis budget. That is the parking.

The human doctors were not given a fair fight, and the researchers admit it. The benchmark was text-only, compared the AI with generalists rather than specialists, and used an AI "gatekeeper" that released information only when asked. Which means, for the first time in history, the patient answered exactly the question that was asked, and nothing about their cousin's knee.

Dr. Dominic King of Microsoft AI explained why old tests were useless: "AI has become very good at getting a 100% pass rate on those." Then, a sentence of deep clinical truth: "But of course, that's not real-world. No patient ever came to see me with a nice summary."

Correct. Real patients arrive with a Google search, a WhatsApp voice note and the phrase "it comes and goes."

Harsha Nori, head of AI at Microsoft AI Health, added: "We're not just measuring the accuracy of the systems. We're also measuring how judiciously and carefully it goes through a diagnostic work-up." Judicious and careful. The machine has learned bedside manner from the brochure, not the waiting room.

On ordinary emergency-room cases, the AI hit 80.3% against the doctors' 46.8%. The best single physician reached 62.5%, which in fairness is still higher than any doctor's handwriting legibility score.

Becker's Hospital Review noted that all the study's authors are Microsoft employees who hold equity in the company. So the AI was tested by its parents, who report it is gifted, advanced for its age and absolutely not biting other children.

Next, the tool goes to Beth Israel Deaconess Medical Center in Boston to test triage in emergency waiting rooms. Finally, a clinician who will look at the queue, assess everyone instantly and still tell you the wait is four hours.

Doctors should not panic. The AI cannot hold a hand, cannot sign a sick note and has never once been told "you look too young to be a doctor." It can, however, now call a meeting of six specialists in under a second.

Medicine always wanted a team that never sleeps. It just didn't expect the team to come with a subscription.