Keynote Speaker
Denys Poshyvanyk
Towards an Interpretable Science of Deep Learning for Secure Software Engineering: A Causal Inference View
Abstract: Neural Code Models (NCMs) are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models arrive at decisions or whether practitioners trust NCMs’ outcomes. In this talk, I will introduce doCode, a post hoc interpretability framework specific to NCMs that can explain model predictions. doCode is based upon causal inference to enable programming language-oriented explanations. While the theoretical underpinnings of doCode are extensible to exploring different model properties, we provide a concrete instantiation that aims to mitigate the impact of spurious correlations by grounding explanations of model behavior in properties of programming languages. doCode can generate causal explanations based on Abstract Syntax Tree information and software engineering-based interventions. To demonstrate the practical benefit of doCode, I will present empirical results of using doCode for detecting confounding bias in NCMs as well as detecting code smells in the code generated by state-of-the-art NCMs, such as CodeLlama and Mistral. Finally, I will showcase results of our recent work on how causal inference can be used in the context of secure software engineering research.
Biography: Denys Poshyvanyk is a Chancellor Professor and a Graduate Director in the Computer Science Department at William & Mary. He currently serves as a Senior Associate Editor at the ACM Transactions on Software Engineering and Methodology (TOSEM). He recently served as a Program Co-Chair for FSE’25 and FORGE’25. He is a recipient of multiple ACMSIGSOFT Distinguished paper awards, most influential paper awards, and the NSF CAREER award (2013). He is an IEEE Fellow and an ACM distinguished member.