Benchmark·Public sample

GBA Eval

A long-horizon coding eval in which agents build a Game Boy Advance emulator in Rust within 24 hours.

Company-reportedConfidence: high·Last verified: 2026-08-16·How verification works
Access
Public / Open
Profile completeness 100%

What it is

GBA Eval measures whether a coding agent can implement a Game Boy Advance emulator from scratch and compile it to WebAssembly. Mechanize describes it as one carefully graded task rather than a general-purpose software-engineering benchmark.

Editorial note: Mechanize explicitly cautions that this single task is not a complete measure of general coding capability.

Capabilities

Only evidenced capabilities are shown as Yes. Missing evidence remains Unknown.

StatefulUnknown
Long horizonYes
Multi-turnUnknown
Multi-agentUnknown
AdversarialUnknown
MultimodalUnknown
Tool useUnknown
Browser useUnknown
Computer useUnknown
Code executionYes
File manipulationUnknown
API interactionUnknown
MCP supportUnknown
SandboxedUnknown
Deterministic resetUnknown
Stochastic behaviorUnknown
Human in the loopUnknown
Expert authoredUnknown
SyntheticUnknown
Production derivedUnknown
Real application replicaUnknown
Partial observabilityUnknown
Persistent stateUnknown
Cross-applicationUnknown

Sources & provenance

  1. Mechanize · official · accessed 2026-08-16 · company-reported

Related artifacts

Benchmark

CoreCraft

Public sample

A simulated enterprise environment where agents complete customer-support and operational tasks inside a fictional PC retailer.

Surge · 100% completeCompany-reported