Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
9-2024
Abstract
Neural code generation systems have recently attracted increasing attention to improve developer productivity and speed up software development. Typically, these systems maintain a pre-trained neural model and make it available to general users as a service (e.g., through remote APIs) and incorporate a feedback mechanism to extensively collect and utilize the users' reaction to the generated code, i.e., user feedback. However, the security implications of such feedback have not yet been explored. With a systematic study of current feedback mechanisms, we find that feedback makes these systems vulnerable to feedback data injection (FDI) attacks. We discuss the methodology of FDI attacks and present a pre-attack profiling strategy to infer the attack constraints of a targeted system in the black-box setting. We demonstrate two proof-of-concept examples utilizing the FDI attack surface to implement prompt injection attacks and backdoor attacks on practical neural code generation systems. The attacker may stealthily manipulate a neural code generation system to generate code with vulnerabilities, attack payload, and malicious and spam messages. Our findings reveal the security implications of feedback mechanisms in neural code generation systems, paving the way for increasing their security.
Keywords
Code generation, Data poisoning, User feedback, Security and privacy, Feedback data injection
Discipline
Artificial Intelligence and Robotics | Information Security
Research Areas
Cybersecurity; Intelligent Systems and Optimization
Publication
Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024) : Vienna, Austria, September 16-20
First Page
528
Last Page
540
Identifier
10.1145/3650212.3680300
Publisher
Association for Computing Machinery
City or Country
New York, NY, USA
Citation
SUN, Zhensu; DU, Xiaoning; LUO, Xiapu; SONG, Fu; LO, David; and LI, Li.
FDI : Attack neural code generation systems through user feedback channel. (2024). Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024) : Vienna, Austria, September 16-20. 528-540.
Available at: https://ink.library.smu.edu.sg/sis_research/9885
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3650212.3680300
