Evaluation model vs chat model
A chat model returns text you have to trust and parse. An evaluation model returns a probability per option, which is what lets a simulation put a real threshold on a decision instead of a vibe.
Side by side
| Chat model | Evaluation model | |
|---|---|---|
| Output | tokens | a distribution per question |
| Nine questions | nine calls, or one call and a parse | one call |
| Confidence | self-reported, if asked | the mass on the chosen answer |
| Refusals / drift | a real failure mode | no free text to drift |
| Bills on | input and output | input only, here |
The confidence difference is the whole thing
Ask a chat model how sure it is and you get a number it wrote about itself, which is a different object from the probability it assigned. Ask an evaluation model the same question and the answer is already a probability: a character's confidence is the mass on the move they picked.
That makes a rule like Nyx ignores her own reads below 0.72 a real bar. Against a measured distribution (median P(top choice) ≈ 0.57, p75 ≈ 0.77) 0.72 is roughly her top quartile, so she overrides herself often enough to be a character and rarely enough to still act.
Where chat models are still the right call
Anything whose output is text a human reads: dialogue, summary, narration. Parlour has none of that on purpose. The ticker lines are templates filled by the engine, so the room is never describing itself in a voice it might lose.
Read next
What an evaluation model is
An evaluation model takes one shared state and a set of typed questions, and answers all of them in a single request. Each answer comes back as a full probability distribution instead of a sentence.
Using distributions instead of picks
Because every answer is a distribution, the engine can scale an outcome by how sure a character was. A 0.95 read of an open host earns a bigger opening than a 0.55 one, so not every decision is equally committed.
Measuring a threshold instead of guessing it
To set a threshold on a model's answer, ask the same question across a ladder of hand-built situations, look at where the answers separate, and put the bar in the gap. A number picked by intuition usually sits where the model never goes.
Personality as arithmetic, not flavour text
A character in Parlour is a class, meaning what they are allowed to do, plus a rig of code-level modifiers: prices halved or raised, moves forbidden, confidence floors, effects that bounce. No character has a personality paragraph.