Clinical AI: Prediction Is Not a Decision

#What a prediction says

Clinical AI may produce a risk estimate, identify a possible finding or rank cases for review. These outputs describe patterns learned from data. A prediction about a future event is not a diagnosis, and a classification is not automatically a complete clinical assessment. Its meaning depends on the task and setting.

A model can identify an association without establishing cause. Predicting that a person is more likely to experience an outcome does not show which action would prevent it. In particular, a model trained to predict risk does not automatically estimate the benefits or harms of a treatment.

#What a decision requires

Clinical decisions combine information about the person, the reliability of the evidence and the available options. Relevant factors may include symptoms, other conditions, treatment burdens and the person's priorities. Some of this context may be absent from the model's inputs or difficult to represent in a dataset.

Action also involves trade-offs. Investigating every positive alert may lead to unnecessary tests, while dismissing alerts may miss important illness. The consequences depend on the proposed action and the clinical situation. A useful prediction therefore needs a defined place in a care process, not simply a score on a screen.

#Accountability and evidence

Accountability requires clarity about who reviews outputs, who makes decisions and how concerns are escalated. A human reviewer is not a complete safeguard if time, training or access to information is inadequate. Teams need a practical way to question outputs and continue care when a tool is unavailable or unsuitable.

Good statistical performance alone does not prove that using a model improves health. Evaluation may also need to examine workflow, delays, unnecessary interventions and patient outcomes. The strength of evidence needed depends on the intended use and potential harm. Uncertain benefits should be described as uncertain.

#Common misunderstandings

A prediction is not a diagnosis, a treatment recommendation or a guarantee. It estimates an outcome from patterns in data. Even a confident-looking score can be wrong, and its meaning depends on who was studied, what was measured and how the model is used.

High accuracy does not automatically mean better care. A model may perform well overall while missing important cases or producing unnecessary alerts. Performance can also differ across patient groups and care settings. Whether acting on its predictions helps patients requires separate evidence.

A risk score does not set its own action threshold. The same estimated risk might support different choices depending on the likely benefits, possible harms, available alternatives and a patient’s priorities.

Human review is not a complete safeguard by itself. Clinicians need enough information, time and authority to question the output. Responsibility for checking results and following up should remain clear.

#Questions worth asking a clinician

• What does this model predict, and what additional evidence would you need before using its output to diagnose me or recommend treatment?
• Does the model identify an association, or is there evidence that acting on its prediction improves health outcomes?
• How would my symptoms, other conditions and current medicines change your interpretation of this prediction?
• How would you weigh potential benefits, harms and my preferences when deciding whether to act on this prediction?
• Beyond its performance score, has using this model improved patient outcomes, and who is accountable for decisions based on its output?