Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

4-2025

Abstract

Large language models (LLMs) often exhibit hallucinations, producing incorrector outdated knowledge. Hence, model editing methods have emerged to enabletargeted knowledge updates. To achieve this, a prevailing paradigm is the locatingthen-editing approach, which first locates influential parameters and then edits themby introducing a perturbation. While effective, current studies have demonstrated that this perturbation inevitably disrupt the originally preserved knowledge within LLMs, especially in sequential editing scenarios. To address this, we introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters. We theoretically prove that this projection ensures the output of post-edited LLMs remains unchanged when queried about the preserved knowledge, thereby mitigating the issue of disruption. Extensive experiments on various LLMs, including LLaMA3, GPT2XL, and GPT-J, show that AlphaEdit boosts the performance of most locatingthen-editing methods by an average of 36.7% with a single line of additional code for projection solely. Our code is available at: https://github.com/jianghoucheng/AlphaEdit.

Discipline

Artificial Intelligence and Robotics | Programming Languages and Compilers

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Digital transformation

Publication

Proceedings of the 13th International Conference on Learning Representations (ICLR 2025), Singapore, April 24-28

First Page

1

Last Page

32

Publisher

ICLR

City or Country

Singapore

Download

Share

COinS