Steer your model: Secure code generation with contrastive decoding

Publication Type

Journal Article

Publication Date

1-2026

Abstract

Large Language Models (LLMs) specialized in code have demonstrated impressive capabilities in various programming tasks such as code generation. However, these models often generate vulnerable code due to inherent flaws in training datasets derived from large-scale, unfiltered open-source repositories. Existing methods like SVEN (prefix tuning) and CoSec (supervised co-decoding) attempt to address these risks but face challenges with transferability or inflexible security constraints. To mitigate these issues, we propose SCoDE, a two-stage approach for secure and functionally correct code generation. After an initial functional tuning phase, we integrate a plug-and-play security steering matrix at the model’s output embedding layer. This matrix can be transferred across models without modifying their original weights. During inference, we introduce a novel contrastive decoding mechanism that adaptively balances the base model’s functional logits with positive and negative security steering signals. Extensive experiments on 60 security scenarios and two standard benchmarks (HumanEval, MBPP) using StarCoder, Qwen2.5-Coder, and CodeLlama demonstrate that SCoDE enhances security while maintaining functional correctness. On average, SCoDE improves security by 28.09% over the original models, 12.07% over CoSec, and 4.82% over SVEN. For functional correctness, it achieves average gains of 55.03% on HumanEval and 41.81% on MBPP over the original models.

Keywords

Codes, Security, Adaptation Models, Tuning, Electronic Mail, Standards, Decoding, Training, Software Development Management, Vectors, Large Language Models, Code Generation, Code Security

Discipline

Information Security

Research Areas

Intelligent Systems and Optimization

Publication

IEEE Transactions on Software Engineering

Volume

52

Issue

3

First Page

809

Last Page

834

ISSN

0098-5589

Identifier

10.1109/TSE.2025.3650127

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TSE.2025.3650127

This document is currently not available here.

Share

COinS