BioBasicsBench: check Astra’s 66.7% versus Sol’s 43.8%, not “agents already run wet labs safely”

OpenAI published BioBasicsBench on 6 November 2026 with Ligo, a biotech company focused on enzyme design. The bench asks whether agents can read everyday raw evidence (gels, plates, microscopy, instrument exports), spot errors, and refuse a conclusion when the evidence is not enough.

At the xhigh setting, GPT-6 Astra earned a mean 66.7% of available credit; GPT-5.6 Sol earned 43.8%. The same post shows Astra still letting familiar assumptions override controls and miscounting colonies. The check is those scores and cases, not a claim that AI already staffs a wet bench safely.

Hard facts (https://openai.com/index/measuring-how-ai-agents-handle-wet-lab-science/; published November 6, 2026; cross-checked via public mirror/transcripts; accessed the same calendar day in Asia/Shanghai):

• Partner: Ligo (enzyme-design biotech).
• Tasks: 100 across protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning.
• Data kinds (overlapping): gels and blots 31; protocols, notes and specs 27; tables and instrument exports 20; bench, apparatus and reagent photos 18; microscopy 14; instrument screens and plots 13; sequencing files 12; protocol videos 7; agar plates and colonies 7; reference sheets 21.
• Authoring: scientists used their own experimental data with expert rubrics; each task had a non-author Ligo internal review, an external PhD wet-lab review, and a final internal review.
• Scoring: credit against individual rubric criteria, so a lucky conclusion can score differently from a partially correct analysis. Most tasks combine biology knowledge, image/data reading, error detection, and protocol understanding.
• Evaluation: seven models (GPT-5.2, 5.4, 5.5, 5.6 Sol, 6.1 Sol, 6 Luna, 6 Astra); 37 model and reasoning-effort settings; five attempts per task; 18,500 results. The headline metric is mean reward (percent of available credit) with 95% CIs from resampling whole tasks.
• Xhigh table: Astra 66.7% (60.3-73.1, about 4,704 output tokens per task); 6.1 Sol 62.0%; 5.6 Sol 43.8% (37.6-50.0, about 9,099 tokens); Luna 38.3%; 5.5 35.4%; 5.4 33.2%; 5.2 31.2%.
• Improvement examples: Astra flagged dye-channel overlap instead of calling cells dead from brightness alone; noticed a negative-control plate labeled with the wrong antibiotic; read a microscope series as a z-stack rather than cells “disappearing.”
• Failure examples: on a colony PCR gel Astra chose lanes 4, 6, 7 and 8 while the reference chose 5, 9 and 10; it generalized vacuole contrast from darker examples while lighter ones sat in the same field; colony counting overshot the reference by about 12%.

Evidence strength: an official benchmark note with a model table, confidence intervals, and concrete failure write-ups. The 66.7% versus 43.8% gap, the 18,500-attempt scale, and the three-pass review process can all be checked against the post. The failure cases name reference lanes versus Astra’s lanes, which is checkable negative evidence rather than soft humility.

How to check: open the post and verify the xhigh table and CIs; spot-check the PCR-lane, vacuole-contrast, and colony-count sections for the same reference/model splits. If later independent runs push Astra near full credit and erase those failure modes, then talk about routine wet-lab reading as “cleared.” Right now about one third of available credit is still missing.

Steelmanning the bench: raw, ambiguous experimental evidence is closer to lab troubleshooting than charts selected for papers. Rubric-level credit separates a conclusion that happens to match from an analysis that holds.

Strongest steelman: failed runs, ambiguous images, and conflicting observations are normal in wet labs. Scoring “stop when the evidence cannot support a conclusion” targets a real risk: misreading a gel or missing a control wastes the next experiments.

Steelman risks: tasks and rubrics come from a partner-company process with OpenAI; outsiders cannot yet re-run the full raw pack independently. A leading 66.7% still leaves many criteria unpaid. The post itself shows assumptions overriding controls and counting errors. The bench measures reading and judgment, not pipetting, reagent handling, or a closed biosafety loop.

“Progress” here means the mean-reward gap versus GPT-5.6 Sol, not clearance for unstaffed work.

For labs and biosafety watchers, the next six months are about whether independent teams can reproduce or extend these 100 tasks, whether Astra-class models cut control-check and counting failures, and whether OpenAI ties the same skills to biosafety or biosecurity monitoring with checkable metrics. For readers, the check is public scores, intervals, and failure cases, not a claim that an AI scientist already “reads gels for a living.”

What to check in BioBasicsBench is GPT-6 Astra’s 66.7% (60.3-73.1) versus GPT-5.6 Sol’s 43.8% (37.6-50.0) at xhigh, the 100-task and 18,500-result setup, and the control and counting failures still shown in the post. What not to check is whether agents already run wet labs safely on their own. If those numbers hold, this is a checkable pressure test of everyday reading. Reading 66.7% as a license to staff the bench leaves the source behind.

【No.2】