Terminal-agent benchmark
Terminal-Bench
Whether an agent can complete work through a real terminal environment.
Key information
- Focus
- Multi-step tasks performed by an agent using a terminal.
- Measure
- Task resolution rate, with model and agent identified separately.
- Version / scope
- Terminal-Bench 4.0
- Update pattern
- Continuously developed tasks with tagged dataset releases. Compare a pinned release, not a moving latest label.
- Publisher update
- Not consistently disclosed
- Editorial verification
- 15 October 2026
In plain English
What makes it different?
Terminal-Bench tests an agent's ability to get work done in an executable environment. The unit being evaluated is a model working through an agent harness, with its tools and limits. This makes it useful for investigating practical agent behaviour without reducing it to short code-completion questions.
Sources and attribution are listed below.
What it measures
Multi-step tasks performed by an agent using a terminal.
Strengths
- Checks outcomes in an executable setting.
- Records the agent as well as the underlying model.
Limitations
- Environment faults, verifier design and available resources can influence success.
- A terminal workload does not cover all software development or autonomous work. Different dataset generations are not interchangeable.
How the evaluation works
An agent receives a task and works in an isolated environment. A verifier checks whether the required outcome was achieved.
The project maintains versioned tasks and uses Harbor to run evaluations. Model, agent, environment and resource constraints all belong to the result.
Configuration matters
Keep model ID, effort, harness, sandbox, task timeouts and repetitions together. A result with one coding agent cannot be attributed to the same model under another agent without evidence.
Primary methods: Current repository and running protocol ↗Official benchmark listing ↗
Results at the publisher
Scores stay with the original source
We have not documented the permissions needed to reproduce this leaderboard here. You can still explore its methods and visit the publisher for results.
Versions & freshness
Verified 15 October 2026The official homepage currently labels Terminal-Bench 4.0. Older 2.x or Hard results remain different benchmarks; this directory does not upgrade their scores.
Official site, current benchmark listing and repository accessible. Continuously developed tasks with tagged dataset releases. Compare a pinned release, not a moving latest label.
Our review date is not an evaluation date. Individual run dates are not consistently available; they remain unknown here. No automatic result collection is enabled.
Publication & reuse
The repository has an Apache 2.0 licence. The scope for redistributing the hosted leaderboard, third-party submissions, retained history and automated updates has not been established separately.
Repository code and individual task assets can have different rights from hosted result tables.
View the scope of the reuse decision
- Result fields
- No result fields cleared
- Public result display
- not enabled
- Storage
- not enabled
- Historical copies
- not enabled
- Updates
- not enabled
- Commercial reuse
- not enabled
- Derived ratings
- No scoped grant documented; not enabled.
- Attribution
- Credit the named publisher and link the primary sources. Attribution alone does not grant reproduction rights.
This decision does not clear mixed-source scientific recommendations. Publisher content, software, datasets and model licences are separate. No partnership is implied.
Primary sources & attribution
This guide is original scor.es commentary about Terminal-Bench team; Stanford, Harbor and Laude Institute's work. The links below support the identity, methods, versions and reuse review.
- Official website ↗Checked 15 October 2026
- Current repository and running protocol ↗Checked 15 October 2026
- Official benchmark listing ↗Checked 15 October 2026
- Repository Apache 2.0 licence ↗Checked 15 October 2026
Editorial edition 2026-10-15.1 · reuse review directory-rights-1. Re-check due 14 November 2026.