Technical guide
HVAC-Bench: protocol, scope, results, limits
HVAC-Bench is an internal protocol on sampled real fault codes (PAC, splits, boilers). Scores are scenario-level — not a blanket guarantee on every OEM code worldwide.
Updated 30 August 2026
Protocol
Each scenario fixes brand, model and fault code. We score whether the answer matches the manufacturer procedure with valid manual reference — not stylistic similarity.
Scope
Lookup scenarios use the global corpus; multi-step diagnostics use Harness Beta allowlist (5 anchor brands). Do not extrapolate Beta coverage to 400k+ lookup.
Limits
Bench dates and rows live on /preuve. We publish deltas LLM-only vs sourced — not competitor benchmarks or Bluon-style parts coverage.