Publication Type

Journal Article

Version

acceptedVersion

Publication Date

3-2026

Abstract

In this study, we propose a reinforcement learning-based adaptive variable neighborhood search (RL-AVNS) method designed for effectively solving the Vehicle Routing Problem with Multiple Time Windows (VRPMTW). Unlike traditional adaptive approaches that rely solely on historical operator performance, our method integrates a reinforcement learning framework to dynamically select neighborhood operators based on real-time solution states and learned experience. We introduce a fitness metric that quantifies customers’ temporal flexibility to improve the shaking phase, and employ a transformer-based neural policy network to intelligently guide operator selection during the local search. Extensive computational experiments are conducted on realistic scenarios derived from the replenishment of unmanned vending machines, characterized by multiple clustered replenishment windows. Results demonstrate that RL-AVNS significantly outperforms traditional variable neighborhood search (VNS), adaptive VNS (AVNS), and state-of-the-art learning-based heuristics, achieving substantial improvements in solution quality and computational efficiency across various instance scales and time window complexities. Particularly notable is the algorithm’s capability to generalize effectively to problem instances not encountered during training, underscoring its practical utility for complex logistics scenarios.

Keywords

Multiple time windows, Reinforcement learning, Unmanned vending machine replenishment, Vehicle routing

Discipline

Artificial Intelligence and Robotics

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Sustainability

Publication

Computers & Industrial Engineering

Volume

213

First Page

1

Last Page

14

ISSN

0360-8352

Identifier

10.1016/j.cie.2025.111760

Publisher

Elsevier

Additional URL

https://doi.org/10.1016/j.cie.2025.111760

Download

Share

COinS