🎉 Excited to share that our paper "VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities” has been accepted to EMNLP 2026! This work is a collaboration between @PurdueCS and multiple teams from @RedHat (AI Engineering, Ecosystem Engineering, and Product Security). Software supply chains are an increasingly important security challenge. While tools can identify vulnerable dependencies, determining whether a vulnerability is actually exploitable in a downstream application often requires substantial cross-repository reasoning and security expertise. In this work, we introduce VEX-Bench, a benchmark for evaluating LLM agents on this problem: - 75 real-world vulnerability cases mined from GitHub - Expert-labeled across Python, Java, and Go - Evaluation of 9 models across 3 agent harnesses - Going beyond binary “exploitable / not exploitable” decisions to evaluate the reasoning behind exploitability Our results show that frontier LLM agents can perform well on binary vulnerability assessment, but fine-grained exploitability reasoning remains significantly more challenging. This points to an important direction for AI-assisted software security: it’s not enough for an agent to identify that something is vulnerable. We need to understand why it is exploitable, whether it actually matters in context, and how confidently we can trust the assessment. 🙌 Huge thanks to all the co-authors and collaborators who made this possible! @tian_yi_zhang, Jiahao Shi, Yifeng Di, Ilan Pinto, Edward Tsien, Hongjiao Zhang, Ronit Dey, Ilona Shishov, Gal Netanel, Zvi Grinberg, Vladimir Belousov, Bat-Zion Rotman 📄 Paper: #EMNLP2026# #AI# #LLM# #LLMAgents# #Cybersecurity# #SoftwareSupplyChain# #OpenSource# #PurdueCS# #RedHat#