Google has built a test that asks AI agents to do a real engineer’s job — and every frontier model fails more than half of it. Physical-Engineering’s Last Exam (PLE), announced on 3 November 2026 by Google’s AI & Infrastructure team with Mercor, hands agents 85 long, messy engineering assignments and grades them the way a licensed reviewer would: on whether the design is safe, code-compliant and buildable. The best model scored 48.3%.
What is Google’s PLE?
PLE is a benchmark of 85 real engineering tasks, not quiz questions. It covers four disciplines — mechanical engineering (30 tasks), electrical (27), civil (18) and structural (10) — and each task is a complete professional assignment: design a cooling loop, size a switchboard and its cable schedule, lay out a campus with grading and utilities, or design a roof deck.
Instead of answering a single prompt, the agent works inside a containerised project folder built on Mercor’s Archipelago environment, mirroring what a practising engineer actually receives on the job. Each task holds an average of 61 files — 5,170 across the benchmark — including PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models and superseded document revisions. The agent gets no internet access and up to six hours to find the governing building codes and physical limits, run the calculations and produce deliverables a licensed reviewer would review and approve.
Google built the tasks two ways. Practising engineers from Mercor’s network created 44 of them end to end — project folders, prompts and grading rubrics based on real professional work. The other 41 were co-authored: engineers designed the initial task, an agent pipeline assembled the environment and grader from licensable engineering standards, and engineers then validated every task and made it harder.
What the scores actually say
Every frontier model scores below 50% on the private test set — and the gap between models is far bigger than the gap between the tools that run them. Google tested 14 frontier models across seven publicly available agent harnesses (Antigravity, Claude Code, Codex, Hermes, OpenCode, OpenHands and Pi), running each task three times per configuration.
| Model (on its native harness) | Score on the 56-task private set |
|---|---|
| Claude Opus 5.5 (Claude Code) | 48.3% |
| GPT-6 Astra | 45.8% |
| Gemini 4 Argon | 40.6% |
Those three figures come from Mercor’s write-up of the same results. Google’s own announcement reports only the headline: all models below 50%, with the best at 48.3%. One more number from the research paper needs care: the paper reports a strongest-configuration mean of 50.6% — but that is across all 85 tasks, public and private together, while 48.3% is the private 56-task set alone. They are different measurements, not a contradiction, and they should not be quoted interchangeably.
The spread across models runs to roughly 50 points, against a median spread of just three points across harnesses for the same model. In plain terms: which model you use matters far more than which agent harness runs it — though the harness still changes the run cost significantly.
Why 91% and 41.3% can both be true
The most revealing number in PLE is not the headline score — it is the gap between almost-right and actually-buildable. The top-scoring configuration satisfied 91% of individual constraints on the co-authored tasks, yet only 41.3% of its deliverables passed every hard safety and feasibility check required to earn credit.
That is by design. PLE grades safety and code compliance as gates, not as partial credit: an agent earns points for cost efficiency only after its design clears every hard safety check first. A cooling loop that meets 91% of its requirements but gets one load calculation wrong is not a 91% design in the real world — it is a design that cannot be built. PLE scores it that way, which is exactly why its numbers look so much harsher than the 90%+ scores the same models post on textbook benchmarks.
As Google puts it in the announcement, frontier models now score above 90% on many academic benchmarks, yet those benchmarks mostly test textbook questions, single tool calls or small self-contained design problems — none of which show whether an agent can produce a safe, code-compliant, cost-effective design under a licensed engineer’s review.
How PLE compares with the other “real work” test
PLE is the second benchmark in a week to say the best AI in the world is bad at real work — but the two tests measure different things, and their numbers must never be merged. SemiAnalysis’s GDPx, which we covered on 4 November, gives agents 206 multi-hour tasks across 11 domains and reports the best model at 12%. PLE gives agents 85 engineering assignments and reports the best at 48.3%.
| SemiAnalysis GDPx | Google PLE | |
|---|---|---|
| What it tests | 206 tasks, 11 domains of knowledge work | 85 tasks, 4 engineering disciplines |
| Best score | 12.0% (Claude Opus 5.5) | 48.3% (Claude Opus 5.5, private set) |
| What “passing” means | Task completed to a graded standard | Every hard safety and feasibility check cleared |
| Published | 2 November 2026 | 3 November 2026 |
The lesson is not that one number is “right” and the other wrong. GDPx measures breadth across kinds of work; PLE measures depth in one profession where a single missed constraint can make a design unsafe. Both point the same way: agents that look brilliant on academic tests still stumble when the work is long, messy and consequential.
What it costs to run
PLE is also expensive to take — which is part of the point. The best configuration averaged 27 million tokens and 63 minutes per task, costing $17.18 per task at list prices. A full 85-task run at that rate is roughly $1,460 of model calls for one agent, before any human review time. That is why Google is running 13 sponsored models free on its evaluation server for competition entrants — the benchmark would otherwise be out of reach for most independent researchers.
How to try it yourself
Anyone can inspect the test — only 29 of the 85 tasks are public, and that split is deliberate. Google has released the 29 public tasks, their grading tools and the evaluation container from its GitHub repository, and the open competition runs on the hidden 56-task set at ple-bench.com, with prize details still to come. Entrants can bring their own agent setup, use one of the 13 free Google-sponsored models, supply a commercial API key or connect a self-hosted model server.
If you work in engineering, the public tasks are the useful part: they show exactly what an agent is being asked to do, file by file, and where the grading rubrics draw the line between a good-looking design and an approved one.
Our take: five questions before you trust an AI design
PLE does not say AI is useless in engineering — it says the review step is the product, not the generation step. A model that gets 91% of constraints right is a fast draftsperson; it is not a licensed engineer. Until an agent can clear every hard check, the workflow that makes sense is the one Google’s own framing implies: the agent drafts, a licensed human reviews and approves. Before you hand any AI-produced design to a reviewer, ask these five questions:
- Which building code or standard did it follow — and can you name the exact clause?
- Did it check the superseded revisions, or just the newest file in the folder?
- Which single constraint would make the whole design unsafe if it were wrong?
- Has a human re-run the load or sizing calculation independently?
- Would you sign your name to this design as the responsible engineer?
If the answer to any of those is no, the design is a draft, however polished it looks. That is the honest reading of PLE — and, for now, the only safe one.
Frequently asked questions
What is Google’s Physical-Engineering’s Last Exam (PLE)?
PLE is an 85-task benchmark from Google’s AI & Infrastructure team and Mercor, announced 3 November 2026, that tests whether AI agents can complete real mechanical, electrical, civil and structural engineering assignments — including producing safe, code-compliant deliverables a licensed engineer would approve.
What is the best score on PLE?
Claude Opus 5.5 running in Claude Code leads at 48.3% on the 56-task private test set. GPT-6 Astra scores 45.8% and Gemini 4 Argon 40.6%, according to Mercor’s write-up. Every model tested scores below 50% on the private set.
Why do agents score 91% on constraints but only 41.3% on deliverables?
PLE treats safety and feasibility as hard gates: a design earns credit only if it passes every hard check. The top configuration met 91% of individual constraints on co-authored tasks, but only 41.3% of its deliverables cleared all of them — so most designs were almost right and still not buildable.
Is PLE the same as SemiAnalysis GDPx?
No. GDPx tests 206 multi-hour tasks across 11 domains of knowledge work (best score 12%), while PLE tests 85 engineering assignments in four disciplines (best score 48.3%). Different tasks, different grading — the numbers cannot be combined into one figure.
Can I try PLE with my own AI agent?
Yes. Google released 29 public tasks, grading tools and an evaluation container, and runs an open competition on the hidden test set at ple-bench.com, with 13 Google-sponsored models free to use on its servers. Prize details have not been announced yet.
Sources and methodology
Claims in this article rest on Google’s own announcement (Google Cloud Blog, 3 November 2026), Mercor’s companion write-up (3 November 2026), and independent coverage of both (Unite.AI via EuropeSays, Artiverse, Superpower Daily, 3–4 November 2026), cross-checked against each other on 6 November 2026. Scores are reported exactly as each source states them: 48.3% is the private 56-task set, 50.6% is the research paper’s figure across all 85 tasks. PLE is a Google/Mercor product, not an independent audit, and no independent reproduction of its results has been published yet. This article was desk-researched, not hands-on tested — the author did not run an agent on PLE. Links: Google Cloud Blog — Introducing Physical-Engineering’s Last Exam; Mercor — Introducing PLE with Google Cloud AI Research; ple-bench.com. Our testing standards: How we test AI tools. Related: OpenAI’s GPT-6 Cyber gets a ‘Critical’ rating.
Govind Dheda covers AI tools, guides and prompts for OpenAIMaster — what is new, what is worth using, and how to put AI to work.
Related posts
Alibaba Previewed Qwen4: Four Models, One That Runs on Your PC
Alibaba has previewed Qwen4 in four tiers — Max, Plus, Flash and a 27B model built to run on a…
Read more
Thinking Machines’ Inkling 2 Small: a 594B AI That Works Alone for 4 Hours — but You Can’t Download It Yet
Mira Murati's Thinking Machines Lab launched Inkling 2 Small on 5 November 2026 — a 594-billion-parameter model built for long,…
Read more
OpenAI GPT-6 Cyber Gets ‘Critical’ Rating
OpenAI's GPT-6 Cyber is the first of its security models to earn a 'Critical' cybersecurity rating — and almost nobody…
Read more