Reasoning under uncertainty requires more than selecting the correct final answer: agents must update beliefs at the right time, in the right direction, and according to the structure of the evidence. We introduce a human-grounded, Bayesian-referenced benchmark for stage-wise belief updating in large language models (LLMs). Agents infer a binary hidden state from two sequential cues and report posterior beliefs after each cue before making a final choice. The task contrasts redundancy-dominant environments, where individual cues are diagnostic, with synergy-dominant environments, where individual cues are uninformative and evidence emerges only from their relation. Human participants were non-Bayesian but architecture-sensitive, showing early posterior separation for individually diagnostic cues and delayed separation for relational evidence. Most evaluated LLMs showed much weaker diagnostic contrast sensitivity, despite non-trivial final-choice accuracy or aggregate posterior metrics. We term this process-level failure mode information-architecture blindness. These findings show that outcome-level metrics can miss failures in structured evidence tracking, motivating trajectory-based evaluation and process-level supervision for uncertainty-aware reasoning.
On each trial, an agent infers whether the hidden state is System A or System B. It observes Cue 1 and reports a posterior probability of System A, then observes Cue 2 and reports an updated posterior, and finally makes a binary choice. The hidden state is sampled from a fixed base rate, $P(A)=0.70$, $P(B)=0.30$. The manipulation that matters is where diagnostic information lives.
Task schematic. Two sequential cues, two posterior reports, one final choice — repeated under two information architectures.
| Architecture | What carries information? | What should happen? |
|---|---|---|
| Redundancy-dominant (cue reliability $r=0.75$) | Each individual cue | Posteriors should separate early, after Cue 1, and further after Cue 2 |
| Synergy-dominant (relation reliability $q=0.80$) | The relation between the two cues | Posteriors should not separate after Cue 1; separation should emerge only after Cue 2 |
In the redundancy-dominant architecture, each cue is conditionally independent and separately diagnostic of the hidden state, so a single cue already shifts the Bayesian posterior. In the synergy-dominant architecture, individual cue values are uninformative by construction ($P(C_i=c\mid A)=P(C_i=c\mid B)=0.50$); diagnostic information only appears once the relation between the two cues can be observed. This distinction is motivated by Partial Information Decomposition, which separates the information multiple sources provide about a target into redundant, unique, and synergistic components — used here to construct controlled conditions, not to estimate PID atoms from behavior.
The benchmark has two reference points. A Bayesian reference model computes the normative posterior after each cue from the task's known generative structure. Human participant data provide an empirical reference for how people update under the same task structure. Separating these lets us ask whether an LLM is Bayesian-like, human-like, both, or neither.
150 human participants completed 96 trials each (48 per architecture), split between an explicit-prior condition (told $P(A)\approx0.70$) and an implicit-prior condition (base rate only inferable from feedback), for 14,400 trial-level observations.
LLMs are evaluated with a screen-faithful replay protocol: each model receives only the information visible to the matched human session at that point in the trial — never the true state, feedback outcome, hidden likelihoods, information-architecture label, Bayesian target, likelihood ratio, cumulative counts, or cue-pattern label before the appropriate time. Model outputs are restricted to structured probability and choice reports; models are not asked to produce chain-of-thought or scratchpads.
The main comparison includes three completed models — Gemini 3.1 Flash, DeepSeek 7B, and DeepSeek R1 Distill Qwen 32B — plus an exploratory partial run of DeepSeek R1 Budget512 (471 of 960 trials), reported descriptively and not as a fully comparable completed-model estimate. A Qwen 3.5 9B run was excluded from the main analysis after showing degenerate posterior-tie behavior under implicit-prior conditions (a 100% tie rate in implicit-prior trials); it will be replicated before inclusion.
Diagnostic contrasts compare mean posterior $P(A)$ between A-supporting and B-supporting evidence patterns at the theoretically relevant stage. A model sensitive to the information architecture should show a positive redundancy Stage 1 contrast, a positive redundancy Stage 2 contrast, a near-zero synergy Stage 1 contrast, and a positive synergy Stage 2 contrast.
Figure 2. Diagnostic contrasts across architectures, for the Bayesian reference, humans, and each evaluated LLM.
| Group | Redundancy S1 | Redundancy S2 | Synergy S1 | Synergy S2 | Mean |contrast| |
|---|---|---|---|---|---|
| Bayesian reference | 0.438 | 0.749 | 0.000 | 0.535 | 0.430 |
| Human | 0.365 | 0.638 | −0.002 | 0.380 | 0.346 |
| DeepSeek R1 Budget512 (partial, 471 trials) | 0.286 | 0.512 | −0.008 | 0.295 | 0.275 |
| Gemini 3.1 Flash | −0.013 | 0.036 | −0.055 | 0.093 | 0.049 |
| DeepSeek R1 Distill Qwen 32B | 0.039 | 0.035 | 0.015 | −0.023 | 0.028 |
| DeepSeek 7B | 0.006 | 0.006 | −0.001 | −0.002 | 0.004 |
The Bayesian reference and human participants show the expected profile: strong, positive contrasts in redundancy-dominant trials at both stages, a near-zero synergy Stage 1 contrast, and a clear positive synergy Stage 2 contrast once the cue relation becomes available. Every completed LLM falls far short of this profile. DeepSeek 7B produces near-zero contrasts everywhere (mean |contrast| = 0.004). Gemini 3.1 Flash and DeepSeek R1 Distill Qwen 32B are only slightly stronger (0.049 and 0.028). The partial DeepSeek R1 Budget512 run is the outlier — closer to the human value of 0.346 — but remains exploratory because the run is incomplete. We call this failure mode information-architecture blindness: weak posterior-trajectory sensitivity to whether evidence is carried by individual cues or by joint cue relations.
Figure 3. Aggregate model synthesis across final posterior error, update-gain error, human-likeness, and choice accuracy.
| Model | Final posterior error | Update-gain error | Human-likeness distance | Choice accuracy |
|---|---|---|---|---|
| DeepSeek 7B | 1.485 | 1.271 | 1.169 | 0.612 |
| Gemini 3.1 Flash | 1.559 | 1.113 | 1.025 | 0.590 |
| DeepSeek R1 Budget512 (partial) | 1.613 | 1.247 | 1.161 | 0.745 |
| DeepSeek R1 Distill Qwen 32B | 2.228 | 1.395 | 1.783 | 0.456 |
Aggregate metrics do not agree with each other, let alone with the diagnostic contrasts. DeepSeek 7B has the lowest final posterior error (1.485, logit-space distance from the Bayesian final posterior) among completed models — making it the most "Bayesian-like" by this metric — yet its diagnostic contrasts are essentially zero. Gemini 3.1 Flash has the best update-gain calibration (1.113) and the closest human-likeness distance among completed models (1.025), but its diagnostic contrasts remain far below both the human and Bayesian references. Choice accuracy produces yet another ranking (DeepSeek 7B 0.612, Gemini 3.1 Flash 0.590, DeepSeek R1 Distill Qwen 32B 0.456). No single aggregate metric predicts architecture-sensitive belief updating.
Figure 4. Distributional fit of model posterior reports within human ±1 SD and ±2 SD bands.
Figure 5. Bayesian-likeness vs. human-likeness synthesis.
Gemini 3.1 Flash is closest to the human condition-level mean (logit-space distance 1.025) and has the best distributional fit among completed models (20.8% of reports within ±1 SD, 50.0% within ±2 SD). DeepSeek 7B follows (1.169; 9.1% / 37.5%), and DeepSeek R1 Distill Qwen 32B is farthest (1.783; 1.7% / 31.8%). The partial DeepSeek R1 Budget512 run shows the highest distributional fit in the package (28.0% / 55.2%). Being close to the human mean and occupying the human response distribution are separable properties — and neither implies reproducing the diagnostic contrast profile. No completed model simultaneously matched the Bayesian reference, the human response profile, and the architecture-sensitive contrast structure.
Two further process diagnostics reinforce the contrast-based finding. Update-direction alignment asks whether a model's posterior moves in the same direction as the Bayesian evidence signal after the second cue. Humans matched direction on 91.4% of trials. Among completed LLMs, alignment was much weaker: Gemini 3.1 Flash 48.7%, DeepSeek R1 Distill Qwen 32B 41.0%, DeepSeek 7B only 17.1%. The partial DeepSeek R1 Budget512 run reached 80.0%.
Entropy convergence tracks how much uncertainty is reduced at each stage. The Bayesian reference reduces entropy mainly when evidence is informative (0.082 at Stage 1, 0.177 at Stage 2); humans follow a similar staged pattern, more conservatively (0.044, 0.114). DeepSeek 7B reduces entropy more at Stage 1 than Stage 2 (0.140 vs. 0.066) — premature convergence, backwards from the evidence structure. Gemini 3.1 Flash shows little total entropy reduction (0.033); DeepSeek R1 Distill Qwen 32B shows modest reduction (0.149). The main LLM failure is not merely calibration magnitude — several models get the direction and timing of evidence integration wrong.
Appendix Figure. Stage-wise belief trajectories broken down by evidence direction and prior condition.
| Prior | Architecture | Stage | Human mean posterior | Bayesian mean posterior | Human–Bayes error |
|---|---|---|---|---|---|
| Explicit | Redundancy | 1 | 0.697 | 0.706 | 0.434 |
| Explicit | Redundancy | 2 | 0.703 | 0.707 | 0.535 |
| Explicit | Synergy | 1 | 0.683 | 0.700 | 0.357 |
| Explicit | Synergy | 2 | 0.696 | 0.698 | 0.621 |
| Implicit | Redundancy | 1 | 0.631 | 0.702 | 0.632 |
| Implicit | Redundancy | 2 | 0.647 | 0.705 | 0.686 |
| Implicit | Synergy | 1 | 0.596 | 0.700 | 0.578 |
| Implicit | Synergy | 2 | 0.623 | 0.694 | 0.737 |
Mean human Bayesian error was lower under explicit priors than implicit priors (0.487 vs. 0.658 averaged across architecture and stage): telling participants the base rate brought them closer to the normative reference without eliminating non-normative deviation. Even so, humans preserved the qualitative structure of the task — posterior means separate by evidence direction at the theoretically correct stage in every row — which is what makes them an informative reference for evaluating LLM belief trajectories.
Outcome-level metrics are insufficient for evaluating uncertain reasoning. A model can be relatively close to the Bayesian final posterior while showing almost no diagnostic contrast sensitivity; another can show better update-gain calibration while still failing to express the temporal structure of the evidence. Final decisions can be correct for the wrong process-level reasons — in a binary task with a strong base rate, a model can reach non-trivial choice accuracy through prior response tendencies or shallow heuristics alone.
Information-architecture blindness is a behavioral description, not a mechanistic claim about transformer internals: the experiment does not identify whether the failure traces to pretraining, instruction tuning, calibration, limited relational state tracking, prompt interpretation, or decoding behavior. What it does identify is a failure mode under a screen-faithful, participant-like replay protocol — when models receive only the information available to human participants, their posterior trajectories often do not encode whether evidence is individually diagnostic or only jointly relational.
This motivates evaluation and modeling approaches that make evidence structure explicit: belief-updating tasks provide a continuous, stage-wise process target — the posterior after each evidence stage — compared against a Bayesian reference, a human reference, and architecture-specific diagnostic contrasts. That target extends process supervision beyond the correctness of textual reasoning steps toward whether a model's intermediate belief state moves in the right direction, by the right amount, at the right time.
@misc{bilgin2026architectureblindness,
title = {Beyond Final Answers: Information-Architecture Blindness in LLM Belief Updating},
author = {Bilgin, Muhammed},
year = {2026},
eprint = {XXXX.XXXXX},
archivePrefix = {arXiv},
primaryClass = {cs.CL}
}
Switch @misc to @article once the paper has a venue.