A benchmark for language models in construction estimating
The Last 10
90% of estimating is routine. The last 10% decide whether an offer wins and earns money. This benchmark measures how well language models handle them.
- DocsParsing tenders, bills of quantities, specs, and data sheets.
- PlansReading drawings: routes, dimensions, symbols, and quantities.
- CalcDeterministic math: fill ratio, loads, quantities, and sums.
- HoursEstimating labor time per item.
- CostEstimating unit prices and total cost.
- DialogWorking with the user: asking clarifying questions, flagging missing information.
Version 0.1 · Run of 12 November 2026 · 66 tasks · 1 sample per task
Leaderboard
Score runs from 0 to 100, the mean over the six tracks. Reasoning is the effort setting the provider offers (low, medium, high). Time and cost are per task, one full multi-step run. Select a column to sort.
| # | Model | Reasoning | |||||
|---|---|---|---|---|---|---|---|
| 1 | gpt-6.1-terraopenailow | low | 99.0 | 28 s | $0.022 | 4.9 | 48% |
| 2 | gpt-6.1-terraopenaimedium | medium | 99.0 | 35 s | $0.027 | 5.0 | 48% |
| 3 | gpt-6.1-terraopenaihigh | high | 98.8 | 45 s | $0.031 | 4.9 | 46% |
| 4 | gemini-4-hydrogengooglemedium | medium | 97.4 | 74 s | $0.11 | 6.6 | 41% |
Reasoning levels
Each chart shows one model's reasoning levels: score against cost per task, on a log scale. The table below it adds the time.
| ReasoningReasoning | Score | Cost per taskCost | Time per taskTime |
|---|---|---|---|
| low | 99.0 | $0.022 | 28 s |
| medium | 99.0 | $0.027 | 35 s |
| high | 98.8 | $0.031 | 45 s |
| ReasoningReasoning | Score | Cost per taskCost | Time per taskTime |
|---|---|---|---|
| medium | 97.4 | $0.11 | 74 s |
Tracks
Six tracks, each a part of estimating. Select one to see its results.
Docs
| Model | Reasoning | Score | Time per taskTime | Cost per taskCost |
|---|---|---|---|---|
| gpt-6.1-terraopenailow | low | 100.0 | 38 s | $0.030 |
| gpt-6.1-terraopenaimedium | medium | 100.0 | 38 s | $0.031 |
| gpt-6.1-terraopenaihigh | high | 100.0 | 41 s | $0.032 |
| gemini-4-hydrogengooglemedium | medium | 100.0 | 68 s | $0.10 |
By language
Every task runs in German and in English, with the same data, prompts, and grading.
| Model | Reasoning | German | English | Difference |
|---|---|---|---|---|
| gpt-6.1-terraopenailow | low | 99.0 | 99.0 | +0.0 |
| gpt-6.1-terraopenaimedium | medium | 99.0 | 98.9 | +0.2 |
| gpt-6.1-terraopenaihigh | high | 98.5 | 99.2 | −0.7 |
| gemini-4-hydrogengooglemedium | medium | 98.6 | 96.2 | +2.4 |
Method
- Multi-step: every task starts from a short brief. The model reaches the data only through tools and ends by submitting an answer.
- Synthetic ground truth: the tasks are generated, so the correct answer is known by construction.
- Scores are means over tasks, and every task weighs the same. The overall score is the mean over the tracks.
- Cost is what OpenRouter billed, including cache reads and writes.
- Time is the wall time of one full task, from the first model call to the answer.
- Run date
- 12 November 2026
- Tasks
- 66
- Samples per task
- 1
- Models and reasoning levels
- 4