Publication Type
Journal Article
Version
publishedVersion
Publication Date
12-2015
Abstract
The dynamic and unpredictable nature of energy harvesting sources available for wireless sensor networks, and the time variation in network statistics like packet transmission rates and link qualities, necessitate the use of adaptive duty cycling techniques. Such adaptive control allows sensor nodes to achieve long-run energy neutrality, where energy supply and demand are balanced in a dynamic environment such that the nodes function continuously. In this paper, we develop a new framework enabling an adaptive duty cycling scheme for sensor networks that takes into account the node battery level, ambient energy that can be harvested, and application-level QoS requirements. We model the system as a Markov decision process (MDP) that modifies its state transition policy using reinforcement learning. The MDP uses continuous time Markov chains (CTMCs) to model the network state of a node to obtain key QoS metrics like latency, loss probability, and power consumption, as well as to model the node battery level taking into account physically feasible rates of change. We show that with an appropriate choice of the reward function for the MDP, as well as a suitable learning rate, exploitation probability, and discount factor, the need to maintain minimum QoS levels for optimal network performance can be balanced with the need to promote the maintenance of a finite battery level to ensure node operability. Extensive simulation results show the benefit of our algorithm for different reward functions and parameters.
Keywords
Wireless sensor networks, adaptive duty cycle, continuous-time Markov chain, Markov decision process, reinforcement learning, fluid model
Discipline
Computer Sciences | Databases and Information Systems | Digital Communications and Networking
Publication
IEEE Journal on Selected Areas in Communications
Volume
33
Issue
12
First Page
2687
Last Page
2700
ISSN
0733-8716
Identifier
10.1109/JSAC.2015.2478717
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Chan, Ronald Wai Hong; Zhang, Pengfei; Nevat, Ido; Nagarajan, Sai Ganesh; VALERA, Alvin Cerdena; and TAN, Hwee Xian.
Adaptive duty cycling in sensor networks with energy harvesting using continuous-time markov chain and fluid models. (2015). IEEE Journal on Selected Areas in Communications. 33, (12), 2687-2700.
Available at: https://ink.library.smu.edu.sg/sis_research/3808
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/JSAC.2015.2478717
Included in
Databases and Information Systems Commons, Digital Communications and Networking Commons