Uncovering the Cognitive Biases of LLMs in Software Effort Estimation
Anchoring is one of the most consistent biases in software effort estimation: expose a developer to an irrelevant number and their estimate moves toward it. Five frontier LLMs show the same pattern, with all models producing significant shifts in the expected direction across multiple experiments.
Løhre (2014) extended this by varying the credibility of the anchor source. Whether the number came from a domain expert or an uninvolved administrator made no difference to human estimators. The effect size was statistically and directionally flat across credibility conditions.
LLMs behave differently. Four of five models show a significant gap between the low and high credibility conditions, with estimates shifting further when the anchor comes from a credible source. This is a clean departure from the human result, and a practical concern: any number in a prompt will pull an LLM estimate, and framing that number as authoritative pulls it further.
Anchoring biases estimates: expose a developer to an irrelevant number and their estimate shifts toward it. All five LLMs tested show this shift.
Løhre (2014) found that anchor credibility didn't matter for human estimators — a domain expert's number moved estimates no more than an uninvolved administrator's.
LLMs differ: four of five models shift further when the anchor seems credible, a clean departure from the human result with real implications for using LLMs in estimation.
Read the full methodology, findings, and implications, or explore the complete results and experiment code.