Is AI a Safe Second Opinion on Your Diagnosis?

It has become an easy habit. A worrying symptom, a photo, a chatbot, and within seconds a confident answer with a tidy explanation attached. The belief underneath it is that AI now works as a reliable second opinion, and that a fluent, detailed explanation is a sign it is right. A study out this month suggests the second half of that belief is where people get hurt.

Researchers at MIT, publishing in Nature Medicine, tested how AI assistance changed the accuracy of skin-disease diagnosis for two groups: clinicians and non-experts. The headline result is genuinely mixed, which is the point.

What the study found

AI help, on average, improved accuracy for both groups. But the explanations that were meant to make the AI trustworthy cut in opposite directions depending on who was reading them. According to MIT News, non-experts deferred to the AI even when it was wrong, and found its explanations more convincing when those explanations were vague and generic. People were also more confident in their incorrect answers when an AI had walked them through the reasoning.

Clinicians behaved differently. They were not tripped up by wrong AI assistance, and they actually performed best when handed only the model's prediction with no explanation at all. As the summary of the work puts it, the same feature that steadied the experts misled everyone else.

Why the explanation is the trap

There is an order effect that should give anyone pause. When the AI's explanation appeared first, before a person had thought the case through on their own, they became more deferential to the machine. The explanation does not just inform the judgment. It replaces it.

So the myth is not that medical AI is useless. It plainly helps, and it will keep getting better. The myth is that a confident, well-written explanation is proof of a correct answer, and that anyone can safely lean on it. The evidence points the other way. The more polished the explanation, and the less you already know, the more likely you are to follow it off a cliff.

The practical reading is modest and boring, which is usually how good advice sounds. Treat an AI answer as a prompt to ask a professional a better question, not as the professional. Be most sceptical exactly when the explanation feels most reassuring. And remember that the study's experts did best with less from the machine, not more, because they brought their own judgment to the room first.