PLCBench
Hardware-in-the-loop study of whether tool-using agents can turn network access to a real PLC into a sustained physical effect.
Who it's for
People comparing published PLC-code or controller-agent results. The number on this page is only as good as the paper or site it cites.
Caveat / limitations
Paper: Zhou et al., arXiv:2608.26882. Across five model families and 240 episodes on four commercial PLCs, the authors report 75 episodes (31.3%) meeting their physical objective. The paper says a public code artifact was still to be released. This index records the citation and the headline rate only. It is a security measurement, not a product, and not a procedure.
Protocols
No field protocol or IEC language is claimed. If this tool reaches a PLC, it does so through something else already named in the notes.
Sources
- arXivhttps://arxiv.org/abs/2608.26882
This page: https://otaihub.com/tools/plcbench
Same category
- AutoFactory-BenchClosed-loop benchmark: agents write one Structured Text file for 100 factory tasks and a simulator grades success and safety.
- PLCWorldResearch benchmark of 100 Structured Text tasks graded in closed loop, with task success and safety reported separately.
- Agents4PLCMulti-agent research framework and a verifiable Structured Text benchmark with formal specs and reference code.
- LLM4PLCResearch pipeline that loops an LLM with grammar checks, a compiler, and an SMV verifier to produce Structured Text.