OpenAI's BioBasicsBench: top model hits 66.7% on wet-lab data reading

OpenAI's top model, GPT-6 Astra, earned 66.7% of available credit on BioBasicsBench, a new benchmark the company published Nov. 6 that tests how well AI models interpret raw experimental data across 100 routine wet-lab tasks. OpenAI built the benchmark with Ligo Biosciences, an enzyme-design startup.

The failures the benchmark exposes matter for any lab manager weighing AI tools. In one cloning task drawn from a Yale study, a model read an agarose gel correctly — distinguishing the band patterns — then reversed the interpretation supported by the controls and recommended colonies still carrying the original plasmid. That kind of error, in a real lab, can cost a researcher a full day culturing cells and purifying a plasmid the lab already had.

What does the benchmark actually test?

BioBasicsBench asks models to interpret messy bench evidence: read a raw result, catch a flawed control, and recognize when evidence is too ambiguous to support a conclusion. "Everyday wet-lab work involves failed experiments, ambiguous images and conflicting observations," OpenAI wrote in its announcement, noting that charts and images selected for publication can leave much of that uncertainty out of view.

To build it, scientists converted data from their own experiments into 100 test questions, each with detailed scoring criteria. Every question passed three reviews: one by another Ligo scientist, one by a PhD-level scientist at an outside wet lab, and a final internal review. Models earned points for accurate measurements, sound reasoning and recognizing uncertainty. Reported scores are the average percentage of available points earned across tasks.

How big was the test run?

Seven OpenAI models from two generations went through the benchmark:

• GPT-5 line: GPT-5.2, GPT-5.4, GPT-5.5 and GPT-5.6 Sol
• GPT-6 line: GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra, the top tier released in early September

Each model ran at several reasoning-effort settings, which control how long the model works through a problem before answering. In total, OpenAI ran 37 model-and-setting combinations, five attempts per task — 18,500 graded answers. The comparison covers only OpenAI's own models, so no external systems were benchmarked.

At "xhigh," the highest setting run on all seven models, Astra earned its 66.7% score while using roughly half the output tokens of GPT-5.6 Sol. OpenAI says stronger native multimodal capabilities "may be key" to the improvement.

Where do the models still fail?

The pattern is consistent: a correct observation paired with a wrong call. In the colony PCR task above, Astra sorted the colony lanes into the right two groups, then assumed the larger PCR product meant successful cloning — and picked the colonies still carrying the old plasmid. It also overcounted one plate's colonies by about 12%, enough to inflate an estimate of DNA uptake efficiency by the same proportion.

One contrast stands out. GPT-5.6 Sol based its conclusions on an empty negative control plate. Astra read the handwritten "LB Kan50" label on the plate's rim, saw the plate contained kanamycin while the experiment used ampicillin, and rejected it as a valid control.

OpenAI's own framing is cautious: "Higher benchmark scores do not yet mean models can make reliable judgments in the lab," the company wrote.

Why this fits OpenAI's science push

The benchmark reflects a broader science focus at OpenAI. In a Nov. 3 podcast interview, CEO Sam Altman said the company's recent math results, including its claimed Navier–Stokes proof, "really made me think that we are closer to this milestone than I expected us to be" — referring to a machine for scientific discovery. The same week, OpenAI said Astra designed the enclosure, circuit board and firmware for a talking Codex desk device, and an MIT team reported Astra could improve other AI models by editing their weights — a result critics said relied on standard machine-learning engineering.

Third-party testing points the same direction, with the same caveats. Benchling, which sells laboratory data software, has built its own tests of AI lab reasoning, including one on planning cloning experiments in which Astra also scored highest. "LLMs can work in the wet lab with the right harness and a human in the loop," Nicholas Larus-Stone, the company's head of AI, told R&D World in August.

For now, the benchmark sets a measured baseline — and a clear gap between reading data and judging it.