Put the machine on screen

An AI product that hides its workings is asking to be taken on trust. This one shows the runners-up on every decision, every rule that overruled the model, which brain is answering, and what the session has cost so far.

There is a standard way to present a model's output, confidently and as a finished thing, and it is the wrong one for anything a person is supposed to reason about. Four things are on screen here that usually are not.

The near-miss

approach_host 0.71 · hold_court 0.25 · withdraw 0.02. An almost-did you can see, taken straight from the distribution instead of from a model's account of itself. It is the difference between watching somebody decide and watching a token move.

The override, in amber

Every time a rule overrules the model, the board says which rule and what it changed. Most systems treat that as plumbing and hide it. Here it is the most interesting thing on the screen, because it is the only place where the two halves of the system disagree in public.

Which brain is answering

Live model or house brain, on a badge, always. A product that degrades to a heuristic without saying so is lying by omission, and the fallback is much less embarrassing than the concealment would be.

What it has cost

A running total, like a buy-in. It makes the economics an argument instead of a claim, and it teaches a watcher how small this kind of system can be.

The rule behind all four

Show the thing that would embarrass you slightly. The near-miss, the override, the fallback and the bill are each a small admission, and together they are the reason a watcher can believe the parts that are impressive.

Read next

More on Building one