Publication Type

Journal Article

Version

acceptedVersion

Publication Date

1-2026

Abstract

Recently, combining the strength of large language models (LLMs) and Evolutionary Computation (EC) has shown promising results for addressing optimization problems. It typically involves either iterative next-step solution seeking or directly prompting LLMs to generate critical optimization codes. However, these methods often suffer from low computational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. LLaMoCo features a comprehensive instruction set that includes code-style problem descriptions as input prompts and robust optimization codes from expert EC optimizers as target outputs. We then develop a novel two-phase learning strategy with a contrastive learning-based warm-up to enhance convergence during instruction tuning. Extensive experiments demonstrate that a CodeGen (350M) model tuned by our LLaMoCo yields a powerful domain-specific model for generating high-performance optimizers, achieving superior performance compared to GPT-4 family and other competitors on both synthetic and realistic problem sets.

Keywords

Black-Box Optimization, Code Generation, Evolutionary Computation, Large Language Model

Discipline

Artificial Intelligence and Robotics

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Sustainability

Publication

IEEE Transactions on Evolutionary Computation

Volume

14

Issue

8

First Page

1

Last Page

21

ISSN

1089-778X

Identifier

10.1109/TEVC.2026.3656374

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TEVC.2026.3656374

Download

Share

COinS