PS-TRU-192 · r1
AUTOMATION TRUST
OBSERVED
ENGAGED
EXPLANATION AS PERSUASION
The account of why is composed to be accepted, not to be accurate.
01 DEFINITION
Condition in which an automated result arrives with a reason chosen for how well it lands rather than for how the result was reached, so the confidence a person places in it rests on something written to produce confidence.
02 CONDITIONS
UNCERTAINTY
UNRECORDED
Whether the stated reason is the operative one is not captured and cannot be checked afterwards.
TIME PRESSURE
LOW
The reason is read at leisure, which is what gives it its effect.
ERROR ASYMMETRY
DEFERRED
A plausible wrong reason costs nothing when read and everything when relied on.
OPERATOR STATE
SUPERVISORY
The person is judging a result produced elsewhere, from an account of it.
03 WHERE IT APPEARS
FINANCEA decision is delivered with a reason short enough to be accepted, and the path that produced it is not that reason.
MEDICALA suggested finding carries a justification the clinician can read in seconds, assembled after the finding rather than before it.
AI EXP.An answer is accompanied by an account of its reasoning that was generated alongside the answer rather than followed to reach it.
PUBLIC SERVICEAn eligibility outcome is issued with a stated ground, and the stated ground is the one that is communicable.
RETAILA suggestion is offered with a reason that makes it feel considered, and being considered is what the reason is for.
EDUCATIONAn assessment or placement arrives with an explanation written for the person receiving it.
04 WHAT AEON WORKED THROUGH HERE
Every result had to arrive with a reason or it was passed over, and the reasons that worked were short and human and did not describe what had happened inside. The work went into which reasons the system was entitled to give — the ones it could stand behind — and into what becomes of the results that have none.
What stays unresolved is what an account of a machine's reasoning owes when the actual path is not expressible — saying nothing leaves the result unaccountable, and saying something plausible is the more useful and the less honest of the two.