AI Safety
Evaluation
Merged
inspect_evals contributions
Upstream fixes to the open evaluation suite used by the UK AI Security Institute. A seeded answer-choice shuffle so every GPQA build is the same exam, and an IFEval install fix.
Highlights
- fix(gpqa): seed the answer-choice shuffle so every build is the same exam (merged 10 Sep 2026)
- docs(ifeval): drop the pip install line that cannot resolve (merged 10 Sep 2026)
Overview
Two merged pull requests to inspect_evals, the open collection of evaluations for Inspect AI maintained by the UK AI Security Institute.
- GPQA: the answer choices were shuffled without a fixed seed, so two builds of the benchmark were not the same exam. The shuffle is now seeded, making scores comparable across runs.
- IFEval: removed an install instruction that could not resolve, so the documented setup works.
