AI vs. Human Chemist: The Lab Showdown Where Everyone Won
Gregory Bassen had a simple question: could a large language model write a recipe for making a ceramic oxide that actually works in a real lab? Not in theory. Not in simulation. In a furnace, with real powders, producing a material you could hold in your hand.
So Bassen, a PhD student at Johns Hopkins, and his collaborators set up an unusual competition. They picked a family of materials called Ruddlesden-Popper oxides (layered ceramics used in everything from fuel cells to superconductors) and gave both a human chemist and an LLM the same task: write a synthesis recipe. Then they’d actually make the stuff and see whose recipe worked.
The results, published on July 8, are more interesting than either “AI wins” or “humans win.” The LLM held its own. And when both failed together and had to try again, something unexpected happened: the humans got better, and a brand-new material nobody had ever seen before emerged from the collaboration.
Background & Context
Computational materials science has been on a tear. AI models can now predict millions of hypothetical materials that should, in principle, be stable. The bottleneck isn’t ideas. It’s the lab. Actually making a predicted material requires designing a sequence of chemical reactions: choosing precursors, setting temperatures, picking atmospheres, grinding, heating, cooling. Most predicted materials have never been synthesized because nobody knows the recipe.
LLMs have recently been used to extract synthesis protocols from decades of published literature, and to suggest recipes for new compounds. But there’s a catch: almost all the validation has been computational. Nobody has systematically tested whether LLM-generated recipes work when you actually walk into a lab and follow them. That’s what Bassen’s team set out to do.
What the Researchers Did
The team selected 17 target compounds from the Ruddlesden-Popper family. Some were already known, giving a baseline for success. Others had never been reported. These were the discovery targets. For each one, a human solid-state chemist wrote a synthesis plan based on experience. Separately, an LLM was prompted to generate its own recipe. Both plans were then executed in the lab, in duplicate, by technicians who followed the written instructions exactly.
If both the human and LLM recipes failed, there was a second round. The human chemist studied the failed results and revised their approach. The LLM got the same experimental feedback fed back into its prompt and was asked to try again. This closed-loop design mirrored how real materials discovery works: trial, failure, learning, retry.
What They Found
In the first round, before any feedback, the human and the LLM performed within a whisker of each other. For known materials, the human succeeded 83% of the time, the LLM 75%. For unknown targets, the actual discovery challenge, the human found something new in 17% of attempts, the LLM in 22%. Statistically, these numbers overlap: the AI was, by this measure, as good as the trained chemist at generating workable recipes.
| Agent | Known Materials (Round 1) | Unknown Materials (Round 1) | Known Materials (Round 2) | Unknown Materials (Round 2) |
|---|---|---|---|---|
| Human | 83% ± 8% | 17% ± 9% | 79% ± 8% | 22% ± 7% |
| LLM | 75% ± 9% | 22% ± 10% | 71% ± 9% | 14% ± 6% |
The second round shifted the balance. After feedback, the human’s discovery rate rose to 22%, while the LLM’s dropped to 14%. Both improved at making known materials. The human pulled ahead in the discovery category, and the gap, while still within one standard error, hints at something real: humans got more out of the feedback loop.
There were also qualitative differences in how the two “chemists” wrote their plans. The LLM was verbose, often suggesting multiple options: “grind with a mortar and pestle or a ball mill,” “use an alumina crucible, platinum preferred if available.” It gave temperature ranges rather than specific values. The human’s plans were terse, relying on unwritten assumptions about standard practice. But the LLM’s extra detail sometimes backfired: when it suggested two alternative pathways for one synthesis, the trial that followed the solid-state route worked while the flux route failed.
Why It Matters
Here’s what’s genuinely surprising about this result. When the experiment started, the LLM had no special training on materials synthesis. It was drawing on whatever chemistry knowledge was baked into its training data. And yet it matched a trained chemist on a task that most people in the field consider irreducibly human: knowing, from years of lab experience, what temperature and atmosphere will coax atoms into the right arrangement.
This isn’t about replacing chemists. It’s about the economics of discovery. If an LLM can generate recipes that work three-quarters of the time, a lab can run far more experiments in parallel. A human chemist who might design five synthesis plans in a day could use an LLM to generate 50 candidates, then spend their time on the ones that need expert judgment, like the high-valent nickel compound LaNiO3, which requires a specialized flux technique that the LLM didn’t know about and couldn’t replicate.
How It Could Change Our Lives
The material that came out of this study, Ba3PtO5, won’t be in your phone next year. It’s a platinum-containing oxide that forms one-dimensional chains of corner-sharing octahedra, a structural prototype that fills a gap in the Ruddlesden-Popper family between 2D sheets and 0D isolated units. Its discovery happened because a human chemist, trying a flux growth for a different target, noticed that the barium carbonate had reacted with the platinum crucible. The LLM didn’t plan that. The human followed their nose.
But the system that produced this discovery — AI generating recipes, humans running experiments, both learning from results — is the model that could soon accelerate materials discovery across the board. Better battery cathodes, more efficient solar absorbers, new catalysts for turning CO2 into fuel. Right now, going from a computer prediction to a gram of actual material can take months or years of trial and error. Halving that time with reliable AI-generated starting recipes would change what’s possible.
The Bigger Picture
What makes this study different from most AI-for-science papers is that the LLM didn’t just predict a property or classify a crystal structure. It proposed a physical protocol and that protocol was executed in the real world, with real consequences: powders that either fused into the right phase or didn’t. The gap between predicting materials and making them has been the field’s open secret for years. This work is one of the first to measure that gap and show that AI can narrow it.
Limitations & What’s Next
The study was confined to one family of oxide materials, and the sample size, though admirably large for a lab study, is small in statistical terms. The LLM’s discovery rate in round two fell, possibly because the feedback loop exposed its lack of physical intuition. It knew about standard solid-state synthesis but couldn’t reason about why a reaction failed, the way a human can look at a gray powder and think “needs more oxygen.” The next step, the authors suggest, is building curated databases of synthesis outcomes that LLMs can train on specifically. That, plus models that can reason about chemistry rather than just retrieve recipes, could close the gap with human chemists on the discovery side.
📄 Source: Bassen et al., “Human and LLM Collaboration for Accelerated Materials Synthesis and Discovery,” arXiv:2607.07604v1 (2026).