Reading creativity from the inside: a competing-routes view of the diversity–coherence trade-off in language models
Raising the temperature does not make a language model more creative, and this paper shows why at the level of the token race: temperature collapses value and coherence without ever raising novelty, while prompts that change which continuations can win raise judged novelty monotonically until value gives way — so usefulness (novelty × value) peaks at an interior breadth. This inverted-U replicates across four models (8B–235B) and a four-judge panel (concave in 15 of 16 model × judge cells). The instrument is a per-token count read from the logprobs of an ordinary forward pass: the number of candidate tokens above a probability threshold (“competing routes”, CR), examined at the choice point — the first generated tokens, where a continuation’s direction is set. Two further results. (1) The known early-position concentration of aligned models’ output distributions is replicated with this count-based instrument under a fixed-format matched-pair control in two of three families (Qwen2.5-7B, Mistral-7B-v0.1) and shown to be graded and post-training-recipe-dependent rather than universal: Llama-3.1-8B shows no genuine narrowing, and across families the size of the internal narrowing tracks the size of the behavioural-diversity collapse. (2) Choice-point CR gives a one-pass estimate of between-sample diversity (median within-model r = +0.34 against semantic spread, +0.54 against Distinct-2) — a positive relation where the nearest prior work reported an inconsistent null, with the discrepancy traced to range restriction rather than metric choice. The paper closes with a substrate-relative account of when a creativity intervention helps: to the degree the substrate does not already supply the operation the intervention performs. All generation, scoring, and analysis scripts are included, with per-response data (per-token CR, raw top-token logprobs, judge scores) as JSONL, in the accompanying archive. Target venue: Transactions on Machine Learning Research (TMLR).