Secure Code Benchmark Finds Top Coding Agents Write Secure Code Only 25% of the Time

https://hackernoon.imgix.net/images/n5ld9fKE5xhTwn7KxPGA8cNuAi42-y383awj.png

Last Secure Code Benchmark: Is the Code You Vibe Secure?

Authors: Tianyi Wu, Chao Ge, Peng Chen, Fengji Zhang, Yongxing Wu, Yuchong Xie, Jian Li, Chaoyuan Peng, Kaikai Zhang, Dong Huang, Mingzhe Du, Yukai Huang, Jiaheng Zhang, Terry Yue Zhuo, See-Kiong Ng (Qwen)


Abstract

AI coding agents now write much of the code that ships, yet whether that code is secure remains unclear. We introduce the Last Secure Code Benchmark (LSCBench), a comprehensive evaluation of secure code generation by coding agents. Built from 150 real vulnerabilities, its 450 tasks span seven widely used languages (Java, C, C++, JavaScript, Python, PHP, and Go), 24 CWEs including 21 of the 2025 CWE Top 25 and 8 of the Top 10 KEV weaknesses, more than any benchmark we compare with, and three vibe-coding scenarios: filling in a function, writing a file, and building a repository’s source tree from scratch. Every task asks only...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE

Read more