Beyond Final Answers: Information-Architecture Blindness in LLM Belief Updating

Read the paper

• Paper: arXiv link coming soon
• Code and data: GitHub link coming soon
• Figures and benchmark materials: Project page

If you are interested in LLM evaluation, Bayesian reasoning, cognitive science, or process-level supervision, I would be happy to discuss.

Introduction

Most LLM benchmarks ask a simple question: did the model get the final answer right?

But reasoning under uncertainty is sequential. Evidence arrives over time, and a good reasoner should revise its beliefs as new information becomes available — not just eventually land on the correct answer. A model can reach the right final choice through a shortcut, a response prior, or a lucky heuristic while its intermediate beliefs never actually track the structure of the evidence. To catch that, evaluation needs to look at the posterior trajectory, not only the final report.

Stage-wise posterior trajectories by evidence direction, for the Bayesian reference and human participants. Redundancy-dominant trials separate after Cue 1; synergy-dominant trials separate only after Cue 2.

What we tested

On each trial, an agent infers whether a hidden state is System A or System B (base rate $P(A)=0.70$). It observes Cue 1 and reports $P(A)$, observes Cue 2 and reports an updated $P(A)$, then makes a final binary choice. The manipulation that matters is where diagnostic information lives:

In redundancy-dominant trials, each cue is separately diagnostic — a single cue should already shift the posterior. In synergy-dominant trials, individual cue values are uninformative by construction; diagnostic evidence appears only once the relation between the two cues can be observed.

150 human participants completed 96 trials each (14,400 trial-level observations total), under both explicit-prior and implicit-prior conditions. LLMs were evaluated with a screen-faithful replay protocol: each model saw only the information visible to the matched human session at that point in the trial — never the true state, feedback, hidden likelihoods, architecture label, or Bayesian target ahead of time. The main comparison includes Gemini 3.1 Flash, DeepSeek 7B, and DeepSeek R1 Distill Qwen 32B, plus an exploratory partial run of DeepSeek R1 Budget512. A Qwen 3.5 9B run was excluded after showing degenerate posterior-tie behavior (a 100% tie rate) under implicit-prior conditions, pending replication.

What we found

Human participants were not perfectly Bayesian — mean human-Bayesian error was 0.487 under explicit priors and 0.658 under implicit priors. But their errors were structured: they showed early posterior separation in redundancy-dominant trials and delayed, relation-based separation in synergy-dominant trials, closely tracking the qualitative shape of the Bayesian reference.

Most evaluated LLMs showed much weaker sensitivity to this diagnostic structure.

We call this failure mode information-architecture blindness.

Information-architecture blindness is weak sensitivity, at the belief-trajectory level, to whether evidence is carried by individual observations or by relations between observations.

The gap is not visible from outcome metrics alone. DeepSeek 7B had the lowest final posterior error among completed models (1.485, the closest to Bayesian by that metric) and the highest choice accuracy (0.612) — yet its diagnostic contrasts were essentially zero. Gemini 3.1 Flash had the best update-gain calibration (1.113) and was closest to the human response distribution, but its contrasts remained far below the human and Bayesian references. Human participants matched the Bayesian evidence direction on 91.4% of trials after the second cue; DeepSeek 7B matched it on only 17.1%.

Why this matters

Final-answer accuracy can miss process-level failures. A model may choose the right final answer while updating too early, too late, in the wrong direction, or for the wrong evidence. No single aggregate metric — final posterior error, update-gain error, human-likeness distance, or choice accuracy — reliably predicts a model’s architecture-sensitive belief updating; in our results, they produced different, sometimes contradictory, rankings of the same models.

Information-architecture blindness is a behavioral description, not a mechanistic claim about transformer internals. The experiment does not identify whether the failure traces to pretraining, instruction tuning, calibration, limited relational state tracking, or decoding behavior. What it does provide is a concrete behavioral target: a model should update at the stage where diagnostic information becomes available, and its posterior trajectory should reflect whether that information is individual or relational.

What comes next

This study uses a controlled, text-only replay protocol, which gives clean Bayesian targets and a fair comparison to human data. A natural next step is extending the same benchmark to vision-language models, using the original human experimental interface, to test whether models can both perceive the visual cue display and update beliefs according to the same information architecture. The model set is also still evolving — the excluded Qwen run needs replication, and the partial DeepSeek R1 Budget512 run needs to be completed before it can be treated as a full comparison point.

More broadly, we want to understand when LLMs merely produce plausible answers, and when they actually track the structure of evidence over time.

@article{bilgin2026llm-belief-updating,
  title   = {Beyond Final Answers: Information-Architecture Blindness in LLM Belief Updating},
  author  = {Muhammed Bilgin},
  journal = {conference},
  year    = {2026},
  url     = {https://mebilgin.com/publication/2026-06-11-information-architecture-llm-belief}
}