Professional software becomes the test
The tasks included configuring agreements, approval flows and reusable legal terms in Ironclad’s software. One example is encoding a spending threshold so Finance reviews large purchases while other requests follow a different path. Finishing isolated clicks is not enough: the completed workflow has to behave correctly across the cases it was designed to handle.
Ironclad employees and OpenAI users identified 11 tasks across legal, commercial and procurement work. Each task had between 8 and 50 evaluation criteria. OpenAI estimates that an experienced user would spend 30–40 minutes on a typical task.
A narrow result with a useful signal
OpenAI reports GPT‑6 Astra’s average evaluation score was 32% higher than GPT‑5.6 Sol’s, while estimated time per attempt was 48% lower. The company says it generated synthetic training tasks from representative workflows and used reinforcement learning in hosted software environments.
This is a research collaboration and a limited evaluation, not evidence that the agent can independently run a company’s legal operations. The noteworthy change is that computer-use models are being trained against end-to-end professional tasks with explicit acceptance criteria, rather than judged only on whether they can click through an interface.
Source published 2026-10-06. Coverage is based on the maker’s announcement and demonstration.