The FDA Starts Drawing Rules for Generative AI in Medicine

The regulator that clears heart-rhythm monitors and mammography software is now trying to work out what to do about a tool that writes. On 18 August the US Food and Drug Administration published a discussion paper on generative AI in medical devices, its first real attempt to describe how these systems might be judged before they reach a clinic.

The paper is not a rule. It is a set of proposals put out for comment, with feedback invited until 19 October. But the shape of the thinking is clear. The agency suggests grading generative tools on three fronts: performance against benchmarks, confirmation in actual clinical settings, and continuous monitoring once a product is in use. That last part matters, because a generative model is not a fixed instrument. Its answers vary from one query to the next, and a version update can change its behaviour in ways a one-time approval would never catch.

Why the old playbook does not fit

Traditional medical AI tends to do one narrow thing: flag a suspicious shadow on a scan, estimate a cardiovascular risk score. You can measure how often it is right and clear it on that basis. A generative system that drafts a diagnosis, summarises a patient history, or suggests a treatment produces open-ended text, and there is no single correct output to check it against. Validating that reliably is a genuinely unsolved problem, and the discussion paper is, in effect, the agency admitting it needs a new method rather than stretching the old one.

A gap already showing

The timing is not accidental. Earlier this year the FDA relaxed some requirements for clinical decision-support tools, a change that let many AI systems offering diagnostic suggestions reach clinics without full device review. And a study published this month found that of more than 1,300 AI devices the agency has cleared, only three had been tested for whether they actually improve patient outcomes. Against that backdrop, a framework built around live monitoring reads less like bureaucratic caution and more like an attempt to catch up with tools already sitting in exam rooms.

What comes next depends on the comment period and on whether the proposals harden into binding guidance. For now the useful signal is simply that the FDA has stopped treating generative AI as an ordinary device with an unusual label. If the agency follows through, approval for these tools would look less like a single stamp and more like an ongoing check-up, which is probably the right way to regulate something that keeps changing after it ships. The risk, as ever, is that the writing of the rules moves slower than the spread of the thing they are meant to govern.