Publication Type

Journal Article

Version

publishedVersion

Publication Date

12-2015

Abstract

The dynamic and unpredictable nature of energy harvesting sources available for wireless sensor networks, and the time variation in network statistics like packet transmission rates and link qualities, necessitate the use of adaptive duty cycling techniques. Such adaptive control allows sensor nodes to achieve long-run energy neutrality, where energy supply and demand are balanced in a dynamic environment such that the nodes function continuously. In this paper, we develop a new framework enabling an adaptive duty cycling scheme for sensor networks that takes into account the node battery level, ambient energy that can be harvested, and application-level QoS requirements. We model the system as a Markov decision process (MDP) that modifies its state transition policy using reinforcement learning. The MDP uses continuous time Markov chains (CTMCs) to model the network state of a node to obtain key QoS metrics like latency, loss probability, and power consumption, as well as to model the node battery level taking into account physically feasible rates of change. We show that with an appropriate choice of the reward function for the MDP, as well as a suitable learning rate, exploitation probability, and discount factor, the need to maintain minimum QoS levels for optimal network performance can be balanced with the need to promote the maintenance of a finite battery level to ensure node operability. Extensive simulation results show the benefit of our algorithm for different reward functions and parameters.

Keywords

Wireless sensor networks, adaptive duty cycle, continuous-time Markov chain, Markov decision process, reinforcement learning, fluid model

Discipline

Computer Sciences | Databases and Information Systems | Digital Communications and Networking

Publication

IEEE Journal on Selected Areas in Communications

Volume

33

Issue

12

First Page

2687

Last Page

2700

ISSN

0733-8716

Identifier

10.1109/JSAC.2015.2478717

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Additional URL

https://doi.org/10.1109/JSAC.2015.2478717

Share

COinS