SealedBench: A Protocol for Freezing AI Benchmarks Before the Questions Are Known
A frozen model manifest, a generator sealed with drand, and an Arweave record anyone can use to rebuild the exam and check its scores.
I knew the question types but not the actual questions
On 2026-10-17, a bundle carrying my encrypted generator reached an Arweave block. The generator was sealed to a drand round that had not been published yet. I wrote that program, so I knew it would produce integer word problems, short Python programs, and shortest-path questions. I did not know which ones. The questions depended on a beacon the drand network would publish later, at round 33170212, on 2026-10-18T09:00:00Z.
When the round arrived, the same beacon value unlocked the program and seeded it. Four open-weight models answered 360 completions each: 240 on newly generated questions and 120 on a frozen set of public GSM8K items.
But the first two batches I ran should not be cited, and...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
