OpenAI’s New Lab Test Says GPT-6 Astra Handles Messy Science 66.7% of the Time — Here’s What That Test Really Measures

OpenAI gave its best model a lab exam built from the mess scientists actually deal with — blurry gel photos, plates with the wrong antibiotic, colonies too close together to count — and GPT-6 Astra scored 66.7%. That is the best result of the seven models tested, and a big jump from the previous generation. It is also, in OpenAI’s own words, not yet reliable laboratory judgment. The gap between those two sentences is the whole story.

Quick answer: BioBasicsBench scores AI on 100 wet-lab tasks. GPT-6 Astra earned 66.7% of rubric credit — best of seven, but OpenAI says it is not yet reliable lab judgment.

BioBasicsBench, announced by OpenAI on 6 November 2026, is a 100-task benchmark built with Ligo, a biotech company focused on enzyme design, to test whether AI agents can interpret raw evidence from routine wet-lab experiments. Existing benchmarks mostly use charts and images selected for published papers. Everyday lab work is different: failed experiments, ambiguous images, conflicting observations, and troubleshooting. Getting these tasks right matters, OpenAI says, because misreading an image or overlooking a problem with the controls “can lead scientists to draw the wrong conclusion and spend time on experiments that won’t help answer their question.”

What is BioBasicsBench, exactly?

It is 100 tasks across five areas of routine bench work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. The tasks range from simple requests, such as reading the molecular weight of a DNA sample in a gel, to detailed assay analysis — but all of them involve work OpenAI says wet-lab scientists should be able to complete reliably.

What makes the benchmark different is the evidence, not the questions. Agents are handed the raw material scientists actually work with: gel and blot images, protocols and lab notebooks, tables and instrument exports, bench photographs, microscopy images, instrument screens, sequencing files, protocol videos, and agar plates — with reference sheets such as ladder charts and plasmid maps as supporting material. A single task can combine several kinds of evidence at once. Agents return a structured answer — a conclusion, a measurement, or a recommendation, with supporting reasoning where required — and rubrics award credit for individual elements, including numerical results, supporting reasoning, and recognising when the evidence is insufficient to justify a conclusion.

The tasks were written by Ligo scientists from data in their own experiments, each paired with an expert-written grading rubric. Every task went through three reviews: an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review covering the prompt, the evidence, the rubric, the author notes, and a graded model attempt. That review chain is the benchmark’s real claim to seriousness — and it is also why the absence of a released dataset matters, as we explain below.

What GPT-6 Astra actually scored

At the “xhigh” reasoning setting — the highest effort level OpenAI reports for every model — GPT-6 Astra earned a mean reward of 66.7%, against 43.8% for GPT-5.6 Sol. Mean reward is the average task score expressed as a percentage of the available rubric credit, including partial credit. It is not the share of tasks the model solved outright, and a 66.7% does not mean Astra got 66.7 of the 100 tasks right. OpenAI evaluated seven of its own models across 37 combinations of model and reasoning effort, with five attempts on each task per setting, producing 18,500 scored results.

Model (xhigh effort) Mean reward 95% confidence interval Output tokens per task
GPT-6 Astra 66.7% 60.3–73.1% 4,704
GPT-6.1 Sol 62.0% 55.6–68.4% 5,238
GPT-5.6 Sol 43.8% 37.6–50.0% 9,099
GPT-6 Luna 38.3% 32.3–44.5% 13,120
GPT-5.5 35.4% 29.7–41.0% 11,135
GPT-5.4 33.2% 28.3–38.4% 13,717
GPT-5.2 31.2% 26.2–36.3% 27,620

Two details in that table deserve more attention than the headline. First, the confidence intervals: Astra’s interval (60.3–73.1%) does not overlap GPT-5.6 Sol’s (37.6–50.0%), so the generational jump is real. But it does overlap GPT-6.1 Sol’s (55.6–68.4%) — the gap between Astra and its cheaper sibling is inside the noise, and any article claiming Astra is clearly better than 6.1 Sol is overstating the data. Second, Astra used roughly half the output tokens per task of GPT-5.6 Sol (4,704 against 9,099). It was better and, on this measure, more efficient — though output tokens per task describe generated output, not a bill, so do not convert them into a price.

One status note: GPT-6 Astra itself has not been confirmed as publicly launched. The GPT-6 model confirmed as launched so far is GPT-6 Cyber, which we covered in our 6 November piece on its “critical” rating. Treat Astra’s benchmark numbers as a lab result, not a product you can use today.

Where Astra got smarter: three worked examples

The most convincing part of OpenAI’s announcement is not the table — it is three tasks where Astra caught something the older model missed. In a cell-imaging task, GPT-5.6 Sol classified cells as dead from a dye signal. Astra recognised that the detector could also pick up light from a second dye — the detection windows overlap at 525–540 nm — so brightness alone could not justify the conclusion, and it called for controls that could separate the signals. OpenAI reads this as better handling of what data can and cannot show, not just better vision.

In a DNA-cloning task, Sol saw no colonies on a negative-control plate and based its conclusions on that observation. Astra noticed that the plate was labelled “LB Kan50” — kanamycin, a different antibiotic from the ampicillin used in the experiment — and correctly rejected it as a reliable control. That is the kind of detail a working scientist checks before trusting a result, and it is exactly the kind of detail a model that only reads the image would miss.

In a third task, cells appeared to come into and out of focus across a series of microscope images. Sol looked for a biological explanation for their apparent disappearance. Astra correctly recognised a z-stack — a series of images captured at different depths. OpenAI interprets these examples as evidence of stronger native multimodal capability and better use of experimental context. That interpretation is plausible, but it is OpenAI’s interpretation of its own model’s wins, and it has not been independently reproduced.

Where it still fails: the part OpenAI published anyway

To OpenAI’s credit, the announcement spends as much space on Astra’s failures as on its wins — and the failures are more informative than the score. In a colony-PCR task, Astra could distinguish the two groups of colony bands, but assumed a larger PCR product would identify successful cloning. It proposed lanes 4, 6, 7 and 8; the reference interpretation, comparing against the controls, selected lanes 5, 9 and 10. The visual grouping was correct and the experimental identities were reversed — and following Astra’s answer would mean purifying the wrong plasmid.

In another task, Astra concluded that vacuoles — compartments inside cells — are generally darker than the surrounding cell material, even though lighter vacuoles were visible in the same image. Changing the microscope’s focus can reverse the contrast, and Astra’s own intermediate observations noted that contrast varied with focus. It inspected the contradictory evidence and then left it out of the final answer. A correct observation that does not govern the conclusion is, for a scientist, no better than a wrong one.

The third failure is the most routine, and therefore the most worrying. Counting colonies on a plate, Astra failed to separate close neighbours, missed visible colonies, and marked empty agar — an overcount of approximately 12% against the benchmark reference. With plated volume and dilution fixed, that overcount inflates the estimate of how efficiently DNA was introduced into cells by the same proportion. This is not an exotic edge case. It is Tuesday-morning lab work.

“Higher benchmark scores do not yet mean models can make reliable judgments in the lab.” — OpenAI, “Measuring how AI agents handle everyday wet-lab science”, 6 November 2026

OpenAI says progress on these tasks could help scientists spend less time troubleshooting and more time deciding which experiments to pursue next, and that tracking these capabilities will also inform its assessment of their biosafety and biosecurity consequences. Both claims are forward-looking; neither is demonstrated by a 66.7%.

The third “real work” benchmark — and why you cannot rank them

BioBasicsBench is the third benchmark in a month that tests AI on messy, real work instead of clean exam questions — and the three numbers cannot be compared with each other. Our SemiAnalysis GDPx piece covered 206 tasks across 11 domains of real knowledge work, where the best AI in the world scored just 12% on a pass-or-fail basis. Our Google PLE piece covered 85 real engineering tasks, where every agent scored under 50%. BioBasicsBench now reports 66.7% — but that is mean rubric credit with partial credit, not a pass rate, on 100 tasks in one field.

Benchmark What it tests Headline result What the number is
SemiAnalysis GDPx (2 Nov 2026) 206 real knowledge-work tasks, 11 domains Best AI ~12% Pass rate (pass@1, averaged over 3 attempts)
Google PLE (3 Nov 2026) 85 real engineering tasks Every agent under 50% Task pass rate
OpenAI BioBasicsBench (6 Nov 2026) 100 routine wet-lab tasks GPT-6 Astra 66.7% Mean rubric credit, partial credit included

Reading 66.7% as “better than” 12% or 50% would be a category error, and this article will not make it. What the three benchmarks share is a direction, not a league table: the field has stopped asking whether models can pass exams and started asking whether they can do jobs — and on jobs, every vendor’s own numbers still show large, specific failures. The pattern so far is that models are getting much better at judging evidence (BioBasicsBench’s wins are all about interpretation) while still struggling to carry a task through to a reliable conclusion (its failures are all about acting on what they saw). That matches our earlier coverage of Claude’s work on a 40-year-old protein vault mystery: AI is already useful as a scientific reader, and not yet trustworthy as a scientific decider.

Our take: five questions to ask before trusting any AI benchmark

Vendor benchmarks will keep arriving weekly, and most will be less careful than this one — so here is the test we will use on the next one. First, who tested whose model? All seven models in BioBasicsBench are OpenAI’s own, so the leaderboard says nothing about rivals — a benchmark that only ranks one company’s products is a progress report, not a comparison. Second, partial credit or pass/fail? A mean-reward score with partial credit will always look higher than a pass rate, and the two must never be quoted side by side. Third, is there a human baseline? BioBasicsBench has none published, so we do not know whether 66.7% is near a competent scientist or far below one. Fourth, can anyone check the work? At the time of the announcement, no paper, dataset, full rubrics, evaluation code, or raw results were linked from OpenAI’s post — the tasks and the grading stay behind closed doors for now. Fifth, did the vendor publish its failures? OpenAI did, in detail, and that is the single strongest reason to take this benchmark more seriously than most. A benchmark with no published failures is a press release.

What would change our mind: an independent reproduction on the same tasks; a released dataset and rubrics that let outsiders run their own models, including rivals; a human baseline from working wet-lab scientists; and a follow-up showing the failure modes above actually shrinking rather than the headline number merely rising. Until then, the fair summary is OpenAI’s own: real progress on routine lab analysis, with room left in interpreting images, checking controls, and applying experimental context — and a 66.7% that measures credit earned, not judgment you can rely on.

Frequently Asked Questions

What is BioBasicsBench?

A 100-task benchmark from OpenAI, built with biotech company Ligo and announced on 6 November 2026. It tests whether AI agents can interpret raw evidence from routine wet-lab experiments — gel images, colony plates, microscopy, sequencing reads, and instrument exports — across five areas: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning.

What did GPT-6 Astra score on BioBasicsBench?

At the xhigh reasoning setting, GPT-6 Astra earned a mean reward of 66.7% (95% confidence interval 60.3–73.1%), the best of the seven models tested. GPT-6.1 Sol scored 62.0% and GPT-5.6 Sol 43.8%. Mean reward is the average share of available rubric credit, including partial credit — not the share of tasks solved outright.

Is GPT-6 Astra available to use?

Not that OpenAI has confirmed. The GPT-6 model confirmed as launched so far is GPT-6 Cyber, which received a “critical” rating for cyber work. Treat Astra’s benchmark numbers as a lab result, not a product announcement.

Can BioBasicsBench’s 66.7% be compared with GDPx’s 12% or PLE’s under-50%?

No. GDPx and PLE report pass rates on their tasks; BioBasicsBench reports mean rubric credit with partial credit. They measure different things on different tasks, so ranking them would be a category error. What they share is a direction: the field is now testing AI on messy real work, and every vendor’s own numbers still show large, specific failures.

What are the benchmark’s biggest limitations?

All seven models tested are OpenAI’s own, so there is no rival comparison; no human baseline was published; and no paper, dataset, rubrics, or evaluation code were released with the announcement, so outsiders cannot yet reproduce the results. OpenAI also published detailed failure cases — wrong colony selection, a contradicted generalisation, and a 12% colony overcount — which show the gap between a good score and reliable lab judgment.

Sources and methodology

Claims in this article rest on OpenAI’s own research post, “Measuring how AI agents handle everyday wet-lab science” (published 6 November 2026, read in full), cross-checked against Unite.AI’s report of the announcement (6 November 2026) and Kingy.ai’s explainer, which independently re-read the OpenAI post and flagged the absence of a released paper, dataset, or evaluation code. GDPx and Google PLE figures are from our own earlier coverage, linked above, and are quoted as pass rates, never merged with BioBasicsBench’s mean-reward figure. Every figure is OpenAI’s own, with its confidence interval kept beside it; nothing here is hands-on tested — this is a desk-researched explainer, and the article says so where it matters. Verification date: 8 November 2026.

Related reading on OpenAIMaster.ai: The Best AI in the World Scores Just 12% on Real Work · Google’s New Engineering Exam for AI Agents: 85 Real Tasks, and Every Agent Scores Under 50% · OpenAI GPT-6 Cyber Gets ‘Critical’ Rating — and Almost Nobody Can Use It Yet · Claude Mapped a 40-Year-Old Protein Vault Mystery

Leave a Reply

Your email address will not be published. Required fields are marked *