Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

8-2024

Abstract

High-frequency trading (HFT) that executes algorithmic trading in short time scales, has recently occupied the majority of cryptocurrency market. Besides traditional quantitative trading methods, reinforcement learning (RL) has become another appealing approach for HFT due to its terrific ability of handling high-dimensional financial data and solving sophisticated sequential decision-making problems, e.g., hierarchical reinforcement learning (HRL) has shown its promising performance on second-level HFT by training a router to select only one sub-agent from the agent pool to execute the current transaction. However, existing RL methods for HFT still have some defects: 1) standard RL-based trading agents suffer from the overfitting issue, preventing them from making effective policy adjustments based on financial context; 2) due to the rapid changes in market conditions, investment decisions made by an individual agent are usually one-sided and highly biased, which might lead to significant loss in extreme markets. To tackle these problems, we propose a novel Memory Augmented Context-aware Reinforcement learning method On HFT, a.k.a. MacroHFT, which consists of two training phases: 1) we first train multiple types of sub-agents with the market data decomposed according to various financial indicators, specifically market trend and volatility, where each agent owns a conditional adapter to adjust its trading policy according to market conditions; 2) then we train a hyper-agent to mix the decisions from these sub-agents and output a consistently profitable meta-policy to handle rapid market fluctuations, equipped with a memory mechanism to enhance the capability of decision-making. Extensive experiments on various cryptocurrency markets demonstrate that MacroHFT can achieve state-of-the-art performance on minute-level trading tasks. Code has been released in https://github.com/ZONG0004/MacroHFT.

Keywords

Reinforcement learning, High-frequency trading, Dynamic programming, Markov decision processes, Electronic commerce

Discipline

Artificial Intelligence and Robotics | Management Information Systems

Research Areas

Intelligent Systems and Optimization

Publication

Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2024) : Barcelona, Spain, August 25-29

First Page

4712

Last Page

4721

Identifier

10.1145/3637528.3672064

Publisher

Association for Computing Machinery

City or Country

New York

Additional URL

https://doi.org/10.1145/3637528.3672064

Download

Share

COinS