From clicks to a complete process
OpenAI and Ironclad built research tasks around work such as setting up a nondisclosure agreement, configuring procurement approvals and changing a reusable legal clause based on jurisdiction. An agent has to keep rules in view across forms, templates and approval steps; a sequence of correct clicks is not enough if the final workflow routes the wrong request.
The evaluation covered 11 tasks, each scored against 8 to 50 criteria. OpenAI reports that GPT-6 Astra averaged 55.0% of criteria met, compared with 41.6% for GPT-5.6 Sol using their best tested reasoning settings.
The time number needs context
OpenAI also reports estimated average times of 19.2 minutes for Astra and 37 minutes for Sol. Those are simulations based on assumed model processing and generation speeds, not measured time saved for Ironclad customers. The percentages describe this small research set, not all contract work or real-world customer performance.
The training tasks were synthetic, created from public SEC EDGAR contracts after filtering designed to remove personal information. OpenAI says it did not use its own customer data, internal contracts or nonpublic Ironclad contracts. That boundary matters when interpreting claims about specialized legal work.
Why this is a meaningful agent test
The more interesting change is the evaluation design. Each task includes detailed success criteria and a hosted software environment where the model can practice and receive feedback. That gives researchers a way to measure whether an agent preserves business rules across a workflow, rather than scoring each isolated action.
For legal operations teams, the demo points toward agent work that still needs review: prepare the intake, set the approval path, then verify the result against the policy. OpenAI itself says human oversight and the full contracting platform remain important. The reported score gain is real evidence of progress in the test, not proof that the agent can run contract operations alone.
Source published October 6, 2026. Coverage is based on the maker’s announcement and demonstration.