Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
6-2023
Abstract
Restless multi-armed bandits (RMAB) is a popular framework for optimizing performance with limited resources under uncertainty. It is an extremely useful model for monitoring beneficiaries (arms) and executing timely interventions using health workers (limited resources) to ensure optimal benefit in public health settings. For instance, RMAB has been used to track patients’ health and monitor their adherence in tuberculosis settings, ensure pregnant mothers listen to automated calls about good pregnancy practices, etc. Due to the limited resources, typically certain individuals, communities, or regions are starved of interventions, which can potentially have a significant negative impact on the individual/community in the long term. To that end, we first define a soft fairness objective which entails an algorithm never probabilistically favors one arm over another if the long-term cumulative reward of choosing the latter arm is higher. Then we provide a scalable approach to ensure longterm optimality while satisfying the proposed fairness constraints in RMAB. Our method, referred to as SoftFair, can balance the tradeoff between the goal of having resources uniformly distributed and maximizing cumulative rewards. SoftFair also provides theoretical performance guarantees and is asymptotically optimal. Finally, we demonstrate the utility of our approaches on simulated benchmarks and show that the soft fairness objective can be handled without a significant sacrifice on the optimal value.
Keywords
Automated calls, Community OR, Fairness, Optimizing performance, Patient health, Restless multi-armed bandit, Softmax, Uncertainty, Whittle indexs, Workers'
Discipline
Databases and Information Systems | Theory and Algorithms
Research Areas
Data Science and Engineering; Intelligent Systems and Optimization
Areas of Excellence
Digital transformation
Publication
Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems, London, Great Britain, 2023 May 29-June 02
First Page
1303
Last Page
1311
Publisher
IFAAMAS
City or Country
Taipei
Citation
LI, Dexun and VARAKANTHAM, Pradeep.
Avoiding starvation of arms in restless multi-armed bandit. (2023). Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems, London, Great Britain, 2023 May 29-June 02. 1303-1311.
Available at: https://ink.library.smu.edu.sg/sis_research/9096
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.