Baloney Inspection Report: Claim #1059 · Inspected Sep 30, 2026, 11:36 AM PDT · Scoring model v2
45%“Eyewitnesses who are extremely confident are probably accurate.”
45%Rotting Baloney — Throw It Out!
Most of this claim has spoiled.
Significant portions falseContradicted by evidenceUnsupported assertionsDeceptive narrative
✓ BLIND VALIDATION CONFIRMED THIS SCORE
Serve It Fresh
Ready to Serve11 servings
Serve it the way you want it
The Verdict
“Eyewitnesses who are extremely confident are probably accurate.”45% Rotting Baloney — Throw It Out!Verified blind by 5 frontier AIs · Baloney Inspection Report: baloney.ai/baloney/eyewitnesses-who-are-extremely-confident-are-probablyThe Baloney Sampler
The claim misses a crucial scientific distinction between initial and later confidence. According to the National Academies of Sciences, Engineering, and Medicine and the National Institute of Justice, extreme confidence strongly predicts accuracy only if recorded immediately at the first identification attempt under strict conditions—such as a fair, double-blind lineup without suggestive instructions. Under these narrow circumstances, highly confident witnesses are rarely wrong.
Applying this broadly is dangerously misleading. Research in the Proceedings of the National Academy of Sciences and reports from the Innocence Project demonstrate that human memory is highly malleable. Confirming feedback, repeated questioning, and time artificially inflate a witness's certainty without improving their accuracy. By the time a witness testifies in a courtroom, their absolute confidence is often contaminated, poorly predicting the truth and contributing significantly to wrongful convictions.
Resting on National Academies of Sciences, Engineering, and Medicine · National Institute of Justice · Proceedings of the National Academy of Sciences · Innocence Project
Other ways to slice it — phrasings that might have changed the rating
- “An eyewitness's extreme confidence is highly predictive of their accuracy if recorded immediately during a pristine, double-blind initial identification.” The reports universally emphasize that high confidence strongly correlates with accuracy only at the moment of first testing under carefully controlled, non-suggestive conditions.
- “A witness's extreme confidence on the stand in a courtroom is a poor indicator of their accuracy.” The findings highlight that memory is malleable, and post-identification feedback, repeated viewings, and time typically contaminate a witness's certainty long before a trial.
- “Mistaken eyewitness identifications made with high certainty are a leading contributor to DNA-based wrongful convictions.” Real-world exoneration data demonstrates that witness certainty frequently coexists with serious error, specifically when identification procedures were flawed or repeated.
Written from the panel’s reports by Google · gemini-3.1-pro-preview. Nothing here changed the score.
The Order Ticket
18:36:59CLAIM RECEIVED — QUEUED FOR INSPECTION✓
18:36:59claude-opus-5 REPORT CARRIED OVER FROM THE EARLIER RUN — SCORE 38, CONFIDENCE 84✓
18:36:59gpt-5.5 REPORT CARRIED OVER FROM THE EARLIER RUN — SCORE 45, CONFIDENCE 88✓
18:36:59gemini-3.1-pro-preview REPORT CARRIED OVER FROM THE EARLIER RUN — SCORE 65, CONFIDENCE 95✓
18:37:07sonar-pro REPORTED — SCORE 34, CONFIDENCE 96✓
18:36:59grok-4.6 REPORT CARRIED OVER FROM THE EARLIER RUN — SCORE 70, CONFIDENCE 82✓
18:39:18PRIMARY MASTER GEMINI — SCORE 50✓
18:38:51BLIND VALIDATION GROK — SCORE 42✓
18:38:12AGREEMENT CONFIRMED — DELTA 5✓
18:38:12SCORING MODEL V2 — MEDIAN OF 5 COUNTED LAB SCORES: 38, 45, 65, 34, 70 → 45.✓
18:38:12SOURCE RULE — LABS CITED 25 SOURCES: 0 DISCARDED FOR HAVING NO URL, 4 DISCARDED AS DEAD LINKS (404/410), 11 COULD NOT BE OPENED BY US AND ARE MARKED UNVERIFIED.✓
18:38:12SOURCE RULE — PRIMARY MASTER RULED ON 15 SOURCES: 0 DISCARDED FOR HAVING NO URL, 0 DISCARDED AS DEAD LINKS, 9 MARKED UNVERIFIED, 6 VERIFIED.✓
How This One Was Sliced
Station 1 · Blind Inspection — Five Graders, Zero Peeking
Each lab scored the claim independently, without seeing the others. Open a breakdown for its evidence, its sources, and how sure it was.
Anthropic · EQUAL WEIGHT · 40.0s
claude-opus-5
38%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Meta-analytic and laboratory evidence shows that when an identification is made under 'pristine' conditions — a fair (non-suggestive) lineup, a double-blind administrator, proper instructions warning the culprit may not be present, and confidence recorded immediately at the FIRST test — high confidence is associated with very high accuracy, often 95%+ suspect-ID accuracy in the top confidence bin (Wixted & Wells, 2017).
- Wixted, Mickes, Clark, Gronlund & Roediger (2015, American Psychologist) reanalyzed data including field studies (e.g., Houston Police Department lineups) and found initial eyewitness confidence was a strong, reliable predictor of accuracy, contradicting the older blanket claim that confidence is uninformative.
- Calibration analyses (rather than point-biserial correlations, which are attenuated by restricted range) generally show a monotonic confidence–accuracy relationship: witnesses expressing near-certainty are meaningfully more likely to be correct than low-confidence witnesses.
- Because of this research, the U.S. National Academy of Sciences (2014) and many courts (e.g., the New Jersey Supreme Court in State v. Henderson) recommend recording a witness's confidence statement in the witness's own words at the time of the first identification — an implicit acknowledgment that immediate confidence carries diagnostic value.
- Signal-detection framing implies that high-confidence IDs arise from strong memory matches, so at the moment of first test, extreme confidence is a genuine (if imperfect) signal of memory strength.
Evidence against the claim- The claim omits the conditions that make it true. Under suggestive lineups, non-blind administrators, poor viewing conditions, cross-race identification, long delays, or repeated testing, the confidence–accuracy relationship weakens sharply or collapses.
- Confidence is highly malleable. Wells & Bradfield (1998) demonstrated the post-identification feedback effect: telling witnesses 'Good, you identified the suspect' dramatically inflates their retrospective certainty — including among witnesses who picked the wrong person — and also distorts their reported view quality and attention.
- Courtroom confidence is not initial confidence. By trial, witnesses have typically been exposed to feedback, repeated viewings, mugshots, and preparation, so the extreme confidence jurors actually see is often contaminated and far less diagnostic than confidence at the first test.
- Repeated identification procedures corrupt the evidence: once a witness has seen a face in a lineup, subsequent identifications reflect memory of the lineup rather than of the crime (Wixted, Wells, Loftus & Garrett, 2021).
- Real-world error data: the Innocence Project reports that eyewitness misidentification was a contributing factor in roughly 60–75% of DNA-based exonerations in the U.S., and many of those witnesses testified with total certainty (e.g., the Jennifer Thompson / Ronald Cotton case).
- Even in favorable conditions, high confidence is not near-certainty of accuracy: some studies show high-confidence error rates well above zero, and a confident identification of an innocent 'filler-like' suspect in a biased lineup can be systematically wrong.
- Jurors overweight confidence. Research on juror decision-making finds confidence is the single strongest driver of perceived credibility, which makes an unqualified version of this claim practically dangerous in legal settings.
- The word 'probably' is doing a lot of hidden work: base rates matter. If a lineup contains an innocent suspect, no level of witness confidence can make the identification correct.
How sure this lab was84/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesWixted & Wells, Psychological Science in the Public Interest (Association for Psychological Science)journal · 95 · National Research Council / National Academies of Sciences, Engineering, and Medicinegov · 95 · Wixted, Wells, Loftus & Garrett, Psychological Science in the Public Interestjournal · 93 · Wells & Bradfield, Journal of Applied Psychology (APA)journal · 92 · Wixted, Mickes, Clark, Gronlund & Roediger, American Psychologist (APA)journal · 92 · National Institute of Justice, U.S. Department of Justice — 'Eyewitness Evidence: A Guide for Law Enforcement'gov · 88 · American Psychological Associationedu · 85
OpenAI · EQUAL WEIGHT · 34.3s
gpt-5.5
45%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Scientific reviews find a strong confidence-accuracy relationship when confidence is recorded immediately after the witness's first identification and the procedure is fair, double-blind, and non-suggestive; under those 'pristine' conditions, very high-confidence identifications are often highly accurate (Wixted & Wells, Psychological Science in the Public Interest).
- The National Academies concluded that confidence can have evidentiary value if it is captured at the time of the initial identification, before police feedback or other contamination; this supports a limited version of the claim.
- NIJ guidance for law enforcement recommends obtaining and documenting a witness's confidence statement in the witness's own words at the time of identification, reflecting that contemporaneous confidence is considered relevant evidence.
Evidence against the claim- The claim is misleading if applied to courtroom confidence or later confidence: confidence is malleable and can be inflated by confirming feedback, repeated questioning, co-witness discussion, media exposure, or suggestive lineup procedures.
- Wells & Bradfield's Journal of Applied Psychology study found that simple confirming feedback such as 'Good, you identified the suspect' distorted witnesses' later reports, including making them more confident, even when their identifications were wrong.
- The National Academies emphasized that eyewitness confidence is not a fixed measure of memory strength and that jurors may overvalue confident testimony, especially when confidence has been contaminated after the initial identification.
- High-stress witnessing conditions can reduce eyewitness accuracy, according to a Law and Human Behavior meta-analysis; high subjective confidence does not necessarily correct for poor encoding conditions.
- Wrongful-conviction data show that eyewitness misidentification has contributed to many DNA exonerations, demonstrating that eyewitness certainty can coexist with serious error, particularly when procedures are flawed.
How sure this lab was88/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesPsychological Science in the Public Interestjournal · 96 · National Academies Pressother · 95 · National Institute of Justicegov · 92 · Journal of Applied Psychologyjournal · 94
Google · EQUAL WEIGHT · 20.1s
gemini-3.1-pro-preview
65%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Recent psychological research has demonstrated a 'new synthesis' regarding eyewitness memory: when initial identification is made under pristine, double-blind lineup conditions, an eyewitness's confidence is highly predictive of their accuracy.
- If an eyewitness expresses extreme confidence at the very first, uncontaminated identification attempt, statistical models show they are highly likely to be accurate.
Evidence against the claim- Eyewitness memory is highly malleable. Confidence can be artificially inflated by confirming feedback (even unintentional) from police, lawyers, or co-witnesses.
- Confidence expressed later in the legal process, such as on a witness stand in a courtroom, has a weak correlation with accuracy because the witness's memory has often been altered or reinforced over time.
- Data from DNA exonerations reveals that eyewitness misidentification is a leading cause of wrongful convictions, and many of these cases involved witnesses who testified with absolute, yet incorrect, confidence.
How sure this lab was95/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesPsychological Science in the Public Interestjournal · 95 · National Academies of Sciences, Engineering, and Medicineedu · 95
Perplexity · EQUAL WEIGHT · 8.9s
sonar-pro
34%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Research distinguishes confidence expressed immediately during the initial identification from confidence expressed later in court. Studies published after the National Academies report found that confidence recorded at the initial identification can be strongly correlated with accuracy under carefully controlled procedures.[https://www.pnas.org/doi/10.1073/pnas.1706891114]
- The National Research Council reported that the confidence–accuracy relationship is likely strongest at the time of the initial identification, supporting limited circumstances in which high confidence is informative.[https://www.nationalacademies.org/news/report-urges-caution-in-handling-and-relying-upon-eyewitness-identifications-in-criminal-cases-recommends-best-practices-for-law-enforcement-and-courts]
Evidence against the claim- The statement is too broad because courtroom confidence is not a reliable predictor of eyewitness accuracy; confidence can be affected by feedback, repeated questioning, and the passage of time.[https://www.nationalacademies.org/news/report-urges-caution-in-handling-and-relying-upon-eyewitness-identifications-in-criminal-cases-recommends-best-practices-for-law-enforcement-and-courts]
- A peer-reviewed PNAS review states that courtroom statements of confidence are very poor predictors of accuracy, while emphasizing that confidence at the initial identification is more informative.[https://www.pnas.org/doi/10.1073/pnas.1706891114]
- Real-world DNA-exoneration data show that eyewitness misidentification is a major contributor to wrongful convictions: the Innocence Project reports that 69% of U.S. DNA exonerations in its cited dataset involved eyewitness misidentification.[https://innocenceproject.org/news/how-eyewitness-misidentification-can-send-innocent-people-to-prison/]
- The National Academies recommends caution in relying on eyewitness identification and notes that confidence at trial is not a reliable accuracy indicator, contradicting the unqualified claim that extremely confident eyewitnesses are probably accurate.[https://www.nationalacademies.org/read/18891/chapter/1]
How sure this lab was96/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesProceedings of the National Academy of Sciencesjournal · 97 · National Academies of Sciences, Engineering, and Medicinegov · 98 · National Academies of Sciences, Engineering, and Medicineedu · 98 · Innocence Projectother · 88
xAI · EQUAL WEIGHT · 93.1s
grok-4.6
70%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Under fair/pristine lineup procedures with confidence recorded immediately after the ID, very high-confidence identifications are often highly accurate in lab and some field data (commonly cited rates well above chance, sometimes ~90%+).
- There is a positive confidence-accuracy calibration: people who express extremely high confidence (e.g., 90–100%) are, on average, more likely to be correct than those who express moderate or low confidence.
- Some later syntheses argue the older ‘confidence is useless’ view overstated the problem specifically for initial high-confidence IDs made under good conditions.
Evidence against the claim- Many meta-analyses find only a modest overall confidence–accuracy correlation for eyewitness identification, so a large share of extremely confident witnesses are still wrong.
- Confidence is highly malleable: confirmatory feedback, co-witness talk, repeated questioning, and delay inflate confidence without raising accuracy.
- DNA exonerations show mistaken eyewitness identifications—often delivered with high confidence—are the single most common factor in known wrongful convictions.
- The relationship weakens or breaks for event details (not just person ID), poor viewing conditions, long delays, biased lineups, and confidence assessed later rather than immediately.
- Legal decision-makers systematically overweight witness confidence, which is a documented source of error.
How sure this lab was82/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesPsychological Science in the Public Interest / Association for Psychological Sciencejournal · 93 · National Academies of Sciences, Engineering, and Medicineother · 96 · Innocence Projectother · 78 · National Institute of Justice (U.S. Department of Justice)gov · 90
Station 2 · The Masters
Primary Master · drawn at random
gemini-3.1-pro-preview
50% BALONEY
The statement presents a sweeping generalisation that obscures a crucial scientific distinction. Evaluators unanimously point to a psychological consensus that eyewitness confidence is highly predictive of accuracy, but strictly ONLY at the exact moment of the initial identification and under pristine, double-blind lineup conditions. The claim as written is dangerously misleading because it lacks these absolute prerequisites. Human memory is highly malleable; post-identification feedback, repeated questioning, and the passage of time artificially inflate a witness's certainty without improving their accuracy. Because courtroom confidence is often contaminated, treating confidence as a blanket proxy for accuracy is scientifically unsound and has historically been a major driver of wrongful convictions.
The evaluators were in full agreement on the underlying science. They all highlighted the critical distinction between initial pristine identifications (where confidence is informative) and later courtroom testimony (where confidence is manipulated and unreliable). The variance in scores simply reflected how heavily evaluators penalised the claim for omitting its necessary conditional factors.
Blind Validation Master · never saw the Primary’s work
grok-4.6
42% BALONEY
The unqualified claim is misleading. Under narrow 'pristine' conditions (fair double-blind lineup, proper instructions, confidence taken immediately at the first ID), high confidence is strongly calibrated with accuracy. As written, however, it applies to eyewitnesses in general—including later, contaminated, or courtroom statements—where confidence is highly malleable (post-ID feedback, co-witnesses, delay, suggestive procedures) and poorly predictive. DNA-exoneration data and National Academies guidance show highly confident but wrong identifications are common and legally dangerous. Missing those conditions is not a minor caveat; it is the difference between a limited scientific finding and a false everyday implication.
AGREEMENT TEST PASSED — MASTERS SCORED 50% AND 42% · BLIND VALIDATION CONFIRMED THIS SCORE · NO RE-EVALUATION REQUIRED
How the number was reached · scoring model v2 · how our models work
- Median of 5 counted lab scores: 38, 45, 65, 34, 70 → 45.
The Evidence
Evidence offered for the claim
- Predictive at initial test When a lineup is fair, double-blind, and non-suggestive, high confidence recorded immediately after a witness's very first identification is strongly associated with high accuracy.
- Genuine memory signal Under signal-detection theory, statistical models and calibration analyses show that in pristine conditions, extreme confidence reliably reflects a genuinely strong memory match.
Evidence against the claim
- Confidence is highly malleable A witness's certainty can be artificially inflated by post-identification feedback (like a police officer saying 'good job'), repeated questioning, and co-witness discussions, all without any improvement in accuracy.
- Courtroom testimony is not pristine By the time witnesses testify at trial, their initial memories have typically been contaminated, rendering later courtroom confidence a weak or completely unreliable indicator of accuracy.
- Wrongful convictions Eyewitness misidentification is a leading contributor to DNA-based exonerations in the U.S., and a vast number of these wrongful convictions involved witnesses who testified with absolute, unshakeable certainty.
- Conditions matter entirely Poor viewing environments, high-stress events, cross-race identification, long delays, or biased lineups severely weaken the confidence-accuracy relationship, making highly confident errors systemic.
- Jurors overweight confidence Legal decision-makers systematically overweight witness confidence when assessing credibility, making the unqualified claim practically dangerous in real-world legal settings.
Sources · Reliability · Why Accepted or Discounted
Every link was opened when this claim was inspected. A dead link was discarded before publication; “unverified” means the site refused our automated check and the link is the one the panel cited. A source without a URL was never kept.
Serve It Fresh
Ready to Serve11 servings
Serve it the way you want it
The Verdict
“Eyewitnesses who are extremely confident are probably accurate.”45% Rotting Baloney — Throw It Out!Verified blind by 5 frontier AIs · Baloney Inspection Report: baloney.ai/baloney/eyewitnesses-who-are-extremely-confident-are-probably