Skip to content

GitHub Copilot: benchmarks site adds harness comparisons

Released · GitHub / Microsoft

Models:Adds GPT-6.2 Sol

What the release notes say

GitHub Copilot benchmarks now includes Terminal-Bench-Science 0.1 and adds a new view that compares agent harnesses, both available in public preview. The site compares models in the GitHub Copilot agent harness, or harnesses using the same model, across established software engineering benchmarks, weighing resolution rate, reasoning effort, and cost per task. Terminal-Bench-Science 0.1 In addition to software engineering benchmarks, the site now includes Terminal-Bench-Science 0.1 , which evaluates AI agents on research workflows across scientific domains. This extends coverage beyond writing code to challenging workflows drawn from scientific research. Tasks are authored and reviewed by domain experts across the life, physical, earth, mathematical, and engineering sciences. Results are available via the benchmark picker. New harness view The new Harnesses view compares GitHub Copilot CLI with Claude Code and Codex CLI using the same model at the same reasoning effort. The view includes:

  • Scatter plot : Each harness result is plotted by resolution rate against mean cost per task. Lines connect matched results, and shaded ellipses show the run-to-run variation observed across published runs.
  • Paired differences : For each matched model, Copilot CLI's resolution advantage (in percentage points) and cost savings (as a percentage) are shown side by side, with approximate 95% bootstrap intervals. Only exact model pairs at medium reasoning effort are shown. Copilot-only configurations, like GitHub Copilot Fusion, and unmatched results are excluded rather than compared against a different model. Much of the work we do to make Copilot better happens at the harness-level, and this view makes those improvements visible. Explore the results on the GitHub Copilot benchmarks site . Table of Contents

Terminal-Bench-Science 0.1

New harness view Menu. Currently selected: Terminal-Bench-Science 0.1

Terminal-Bench-Science 0.1

New harness view copilot Share Copied Shared Related Posts Nov.17 Release OpenAI’s GPT-6.2 Astra and GPT-6.2 Sol now available copilot Nov.17 Improvement GitHub Copilot Fusion updates: more availability, benchmarks, and models copilot Nov.17 Release GitHub Copilot app, Microsoft Ignite releases copilot Nov.17 Release Copilot code review: Adaptive effort, secret detection, and more copilot Nov.17 Release MAI Code 2 and MAI Code 2 Flash in GitHub Copilot copilot Nov.17 Release GitHub Copilot in JetBrains, Ignite releases client apps copilot ... +1 Nov.17 Release <a data-targets="reactive-line-state.l