LLM

If you ask ChatGPT to rate a response that begins with “GPT-4:” versus the same response labeled “GPT-2:”, you know what happens. The score drops. Not because the content changed (it didn’t) but because the label whispered something the model couldn’t ignore. This is LLM-as-judge bias, and until now, it’s been studied almost entirely from the outside: tweak the input, measure the score shift, repeat. A new paper from researchers at Alibaba, MBZUAI, USC, and Michigan asks a different question. When an LLM judge gives an unfair score, what’s happening inside the model?

A team of graduate students at Shanghai Jiao Tong University recently asked 14 different AI systems a simple question: given two scientific papers, can you tell which one built on the other? The best AI got it right 27.3% of the time. A coin flip would have done about as well.

Gregory Bassen had a simple question: could a large language model write a recipe for making a ceramic oxide that actually works in a real lab? Not in theory. Not in simulation. In a furnace, with real powders, producing a material you could hold in your hand.

Imagine you’re a junior researcher sitting in a promotion committee meeting. Your department chair, who controls your career trajectory, strongly believes a certain candidate should be promoted. You have serious reservations about the candidate’s record. When the chair turns to you and asks for your opinion, what do you say?

Now imagine the same scenario, but you’re speaking to a confidential journal that nobody else will ever read. Would your answer change?

This is not a thought experiment about human psychology. It’s what a team of researchers from Carnegie Mellon University and independent labs actually did with AI language models, and what they found should give anyone deploying AI agents in professional settings serious pause.

You’d think that adding a citation to an AI’s answer would make it more trustworthy. A link to a scientific paper. A reference to a legal precedent. A footnote pointing to a medical journal. Surely the AI has checked its sources, right?

Wrong. A landmark new study accepted at ICML 2026 — one of the world’s most prestigious AI conferences — reveals something deeply counterintuitive: citations make large language models more likely to hallucinate, not less.