AEON Communications
CLOSE ×
PS-TRU-110 · r5 AUTOMATION TRUST INFERRED OBSERVED

CONFIDENCE DISPLAY MISREAD

The figure says how sure the function is, and is read as how likely it is.

01   DEFINITION

Condition in which an automated judgement arrives with a measure of its own certainty, and that measure is read as a property of the thing judged rather than of the judging.

02   CONDITIONS

UNCERTAINTY MISSING Nothing states that the figure is about the judging rather than the judged.
TIME PRESSURE MODERATE There is time to read the number and not to ask what it is a number about.
ERROR ASYMMETRY ASYMMETRIC Over-reading and dismissing the figure fail in opposite directions.
OPERATOR STATE SUPERVISORY The person is judging a result produced elsewhere, from what accompanies it.

03   WHERE IT APPEARS

MEDICALA likelihood attached to a finding is read as the likelihood of the condition.
FINANCEA model score is read as the probability of the event rather than of the model being right.
AI EXP.A certainty figure describes the system and is understood to describe the answer.
PUBLIC SERVICEA risk rating is treated as a property of the person rather than of the assessment.
INDUSTRIALA diagnostic confidence is read as how likely the fault is rather than how sure the tool is.
EDUCATIONA predicted grade is read about the learner rather than about the prediction.

04   WHERE THIS IS SURFACING

This surfaces wherever automated results carry a score. The score is about the function and is read about the case, and nothing in the presentation marks the difference.

The question it keeps returning to is whether such a measure can be shown at all — leaving it out hands over a judgement with no sense of how firm it is, and putting it in hands over a number that will be treated as a fact about the world.