I Shipped My Code Gate to a Stranger's Machine. Its First Bug Report Fixed the Evaluator Itself.
TLDR: I build a structural gate that refuses AI coding agents’ writes before they land — hard budgets on file size and imports, no human override. Last week it ran for the first time on someone else’s machine, on a real project, with six evaluation questions attached. The report came back with three correct refusals, one false positive, and a fix list. Every fix shipped within a day. The next day, the same machine re-verified every one of them against the code that surfaced them. This is what the loop looked like, including the part where the tool sat dark for two days and nobody noticed.
Three weeks ago I wrote about making coding agents work under a constitution: files capped at 500 lines, no more than 7 imports per module, adversarial cross-vendor review, done-means-proven. The system is private while it hardens. But the smallest piece of it — the...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
