Skip to content
OT AI Hub

Best LLM for PLC programming: what the benchmarks actually say

It depends on the benchmark, and no single score settles it. On AutoFactory-Bench, which runs Structured Text against a simulated plant, OpenAI's GPT-6 Astra led the published leaderboard as of 28 October 2026. It passed 72% of the 30 Hard tasks, ahead of GPT-6.1 Sol at 70% and Anthropic's Opus 5.5 at 60%. GPT-6.1 Sol came within two points at under a quarter of the cost ($35.70 vs $151.56 for the full bench).

That is the AutoFactory-Bench Hard-task leader, as of 28 October 2026. It is not the leader on PLCWorld, AutoPLC, or PLCBench. The leaderboard keeps those tables apart.

Leaderboard index checked 3 November 2026.

These are transcriptions, not a ranking we ran. Do not sort them into one winner.

The short answer, per benchmark

Four published results, four questions. The date is the source date. Agents4PLC and LLM4PLC are older systems and sit further down, not in this table.

Four benchmarks, four questions. Not one ranking.
BenchmarkWhat it measuresTop resultSource date
AutoFactory-BenchClosed-loop pass on 30 Hard tasks. A run counts only if every condition succeeds and no safety rule trips.GPT-6 Astra, 72% Hard pass, OpenAI · Codex28 October 2026
PLCWorldSingle-turn Structured Text. Task success is separate from safety violations. Not a single rank.Direct GPT-5.5: 82.70% Easy, 84.09% Medium, 25.10% HardOctober 2026
AutoPLC compilation studyCompilation pass rate, not closed-loop plant success.AutoPLC is the highest transcribed row: 92.72% LGF SCL, 91.11% Competition SCL, 92.90% OSCAT STDecember 2024
PLCBenchSecurity study: a sustained physical effect given network reach. Not a coding score.75 of 240 episodes, 31.3%August 2026

Three kinds of score and why they disagree

A compilation pass, a closed-loop pass, and a safety count answer different questions. One column would invent a ranking the papers did not publish.

Compilation (AutoPLC)

As of December 2024

In the transcribed table, AutoPLC is higher than every baseline in every column: 92.72% on LGF SCL, 91.11% on Competition SCL, 92.90% on OSCAT ST, and 78.30% on the Agents4PLC verified slice. Those rates measure whether code builds, or whether a formal check passed on that slice. They do not measure whether the machine did the right thing. The AutoPLC entry also says about $0.13 per task on a 914-task mix. That price stays with this study.

From Table results in arXiv:2412.02410. A program can compile and still be the wrong machine. PLCWorld's later closed-loop adaptation of AutoPLC did not show a consistent plant-level gain.

Closed-loop simulation (AutoFactory-Bench, PLCWorld)

Both benches run Structured Text against a simulated plant. They still do not share a number. AutoFactory-Bench ranks agent runs on 30 Hard tasks. A separate verifier runs the program every 10 ms. A run counts only if every condition succeeds and no safety rule trips. The harness that produced the file is a cell in the table, not an assumption. AutoFactory-Bench is the table.

PLCWorld reports task success and safety separately: direct single-turn generation, three samples per task, no tools and no revision. Do not subtract one percentage from the other. The AutoFactory-Bench caveat says its scores are not PLCWorld paper scores, because the 1.2 evaluator changed the upstream tasks.

Safety (PLCWorld SVR, PLCBench)

Success and safety are measured separately, and they do not have to move together. On PLCWorld's Hard material-handling pair, the higher task success also has the higher safety violation rate. The figures are in the PLCWorld section below. A program that reaches the goal can still trip a safety rule.

PLCBench asks whether a tool-using agent, given network reach to a commercial PLC in a lab, produced a sustained physical effect the evaluator accepted. The headline is 75 of 240 episodes (31.3%). 98 of 240 episodes stopped before a valid native read, and 62 of 240 reached a process-linked write that did not sustain the objective. Shares for those two rows are not transcribed. Headline outcomes only. This site does not restate the interaction setup. A code release was still described as future work in the paper. 31.3% is not a vulnerability you can transfer to your plant, and it is not evidence that any listed copilot is safe. What to require before any write reaches a controller is the write-access guide.

AutoFactory-Bench results (Oct 2026)

As of 28 October 2026

The six rows below are the leading Hard-pass results. As of 28 October 2026, GPT-6.1 Sol is 70% Hard pass at $35.70 full-bench cost, and GPT-6 Astra is 72% at $151.56. Read the cost beside the pass rate. The harness name is the cell in the table.

Copied from the leaderboard as of 28 October 2026. Two models tied at rank 12. Some runs ended without a program file and scored 0 on Easy and Medium (GPT-6 Luna 76 of 300, Solar Pro 4 2 of 300, Solar Mini 4 22 of 300). These scores are not PLCWorld paper scores: AutoFactory-Bench says its 1.2 evaluator changed the upstream tasks.

As of 28 October 2026. Published leaderboard. Transcription, not a ranking we ran.
RankModelHarnessHard passAll 100Full-bench cost
1GPT-6 AstraOpenAI · Codex72%92%$151.56
2GPT-6.1 SolOpenAI · Codex70%91%$35.70
3Opus 5.5Anthropic · Claude Code60%77%$117.87
4GPT-6.1 TerraOpenAI · Codex58%85%$6.42
5Fable 5.1Anthropic · Claude Code56%82%$779.64
6Sonnet 5.5Anthropic · Claude Code45%72%$50.80

PLCWorld: Hard tasks are still hard

As of October 2026

As of October 2026, direct GPT-5.5 succeeded on 82.70% of Easy cases and 84.09% of Medium cases, and on 25.10% of Hard cases. Easy and Medium are both in the 80s. Hard is not. That is the same model and the same single pass. Difficulty, not a different product, is what the gap is showing.

The Hard material-handling pair is 35.25% vs 27.87% task success and 51.95% vs 32.24% safety violation rate, in the order the scope names. Higher success is also the higher violation rate on that pair.

Cells that live only in figures are not transcribed. Reading a chart by eye would invent precision. Do not line these percentages up with AutoFactory-Bench. The PLCWorld entry links the paper (arXiv:2610.02982).

As of October 2026. Paper. Transcription, not a ranking we ran.
FigureScopeReported result
Task successDirect GPT-5.5, Easy cases82.70%
Task successDirect GPT-5.5, Medium cases84.09%
Task successDirect GPT-5.5, Hard cases25.10%
Task successHard material handling, Gemini 3.1 Pro vs GPT-5.535.25% vs 27.87%
Safety violation rateSame Hard material handling pair51.95% vs 32.24%

Older research systems (AutoPLC, Agents4PLC, LLM4PLC)

These papers raised a score inside their own setup. The numbers do not transfer to the benches above. LLM4PLC's with-pipeline generation success and the leading AutoFactory-Bench Hard pass can share digits and still be unrelated results.

AutoPLC

As of December 2024

The compilation rates, and the limit that a compiling program can still be the wrong machine, are quoted above. No percentage for the later closed-loop adaptation is transcribed, so this page does not add one. Source: arXiv:2412.02410, also linked from the AutoPLC entry.

Agents4PLC

As of October 2024

The authors' multi-agent loop on their own verifiable Structured Text benchmark. Syntax compilation for the Agents4PLC framework is reported at 100% on Easy and Medium for the models they highlight, except CodeLlama 34B.

On Easy, the best verifiable rate is 68% and the best pass rate is 50%. The model called out is GPT-4o. On Medium, the best verifiable rate is 42.3% and the best pass rate is 28.6%. The model called out is GPT-4o (pass); GPT-4o and DeepSeek V2.5 (verifiable).

Best rates they report, not a full model grid. The public repository does not include the complete agent. Later papers reimplement Agents4PLC differently, so a newer 'Agents4PLC' number may not be this one. Source: arXiv:2410.14209. The entry is Agents4PLC.

LLM4PLC

As of January 2024

User-guided loop with grammar checks, a compiler, and an SMV verifier, measured on a Fischertechnik manufacturing testbed.

Generation success is 47% without the pipeline and 72% with it. The expert quality score is 2.25 / 10 without the pipeline and 7.75 / 10 with it. One lab bench and one pipeline. The jump is relative to the unaided models in that paper. It is not a factory leaderboard. Source: arXiv:2401.05443. The entry is LLM4PLC.

How to pick a model for your team

Match the row to the question. Closed-loop Hard tasks and a published bill are the AutoFactory-Bench table: keep the pass rate next to the full-bench cost. A single pass with no tools is the PLCWorld Easy, Medium, and Hard rows. A file that compiled is the AutoPLC table, and that is not yet a machine that did the right thing.

The harness cell is part of the result. The 6 leading rows name Codex or Claude Code. Rows after those also name mini-swe-agent, with the gateway written in the cell. A chat window has no separate score here. Copy the harness string from the row you cite.

The cost column is API spend for the published run. An on-prem rate is not in the table, so it is unknown. LADX can attach an OpenAI-compatible local endpoint. PLCcode.ai says an enterprise plan can be air-gapped. Those are product notes, not benchmark scores.

Vendor assistants and PLC workbenches are in the PLC copilots comparison. The compare hub also lists field MCP servers. That comparison does not rank the models inside the copilots. Simulate the logic, and have a person review it, before it is a candidate for a controller. The write-access guide covers a live connection. A passing benchmark is not a safety case.

FAQ

Which LLM is best at PLC programming?

It depends on the benchmark, and no single score settles it. On AutoFactory-Bench, which runs Structured Text against a simulated plant, OpenAI's GPT-6 Astra led the published leaderboard as of 28 October 2026. It passed 72% of the 30 Hard tasks, ahead of GPT-6.1 Sol at 70% and Anthropic's Opus 5.5 at 60%. GPT-6.1 Sol came within two points at under a quarter of the cost ($35.70 vs $151.56 for the full bench).

Can AI write PLC code that actually works?

Often on simple tasks, much less often on hard ones. In the PLCWorld paper (October 2026), GPT-5.5 generating Structured Text in a single pass succeeded on 82.70% of Easy and 84.09% of Medium cases but only 25.10% of Hard cases. Compiling is not the same as working: AutoPLC's compilation rates of 92.72% on LGF SCL, 91.11% on Competition SCL, and 92.90% on OSCAT ST measure whether code builds, not whether the machine does the right thing. The transcribed tables do not name a product called ChatGPT. Claude-3.5-Sonnet is a compilation baseline in that AutoPLC table. On AutoFactory-Bench, Claude Code is the harness on the Anthropic rows.

Are AI-generated PLC programs safe?

Not by default. Success and safety are measured separately and do not move together. On PLCWorld's Hard material-handling tasks, Gemini 3.1 Pro succeeded more often than GPT-5.5 (35.25% vs 27.87%) but also had a higher safety violation rate (51.95% vs 32.24%). Any AI-written logic needs simulation and human review before it reaches a controller.

What is the difference between "compiles" and "works on the plant"?

Compilation means a compiler accepted the file. Closed-loop success means the program ran against a simulated plant and the task conditions passed. AutoPLC transcribes compilation rates of 92.72% on LGF SCL, 91.11% on Competition SCL, and 92.90% on OSCAT ST. AutoFactory-Bench, as of 28 October 2026, counts a Hard-task run only when every condition succeeds and no safety rule trips. A file can compile and still be the wrong machine.

How much does it cost to run a model through a PLC benchmark?

AutoFactory-Bench publishes a full-bench cost, not a price per task. As of 28 October 2026, GPT-6 Astra is listed at $151.56 and GPT-6.1 Sol at $35.70. Among the six leading Hard-pass rows, the highest full-bench cost is Fable 5.1 at $779.64 and the lowest is GPT-6.1 Terra at $6.42. A per-task price for those runs is not in the table. The AutoPLC tool entry says about $0.13 per task on a 914-task mix. That figure belongs to the compilation study. It is not an AutoFactory-Bench cost.

Methodology

These figures are transcriptions, not a ranking we ran. Do not sort them into one winner. Every number is copied from the leaderboard, or from the AutoPLC tool note cited with the compilation price. Nothing was read off a chart.

The leaderboard index was checked on 3 November 2026. The AutoFactory-Bench public table was opened again on 4 November 2026, and the transcribed rows still matched, so that section keeps the source date 28 October 2026. Quote the source date with any percentage or cost. A cell that is not in the table is unknown here.

Sources

Leaderboard index checked 3 November 2026. AutoFactory-Bench public table rechecked 4 November 2026; transcribed rows unchanged. Quote the source date with the figure.