Publication Type

Journal Article

Version

acceptedVersion

Publication Date

7-2017

Abstract

With the proliferation of sensors, such as accelerometers,in mobile devices, activity and motion tracking has become a viable technology to understand and create an engaging user experience. This paper proposes a fast adaptation and learning scheme of activity tracking policies when user statistics are unknown a priori, varying with time, and inconsistent for different users. In our stochastic optimization, user activities are required to be synchronized with a backend under a cellular data limit to avoid overcharges from cellular operators. The mobile device is charged intermittently using wireless or wired charging for receiving the required energy for transmission and sensing operations. Firstly, we propose an activity tracking policy by formulating a stochastic optimization as a constrained Markov decision process (CMDP). Secondly, we prove that the optimal policy of the CMDP has a threshold structure using a Lagrangian relaxation approach and the submodularity concept.We accordingly present a fast Q-learning algorithm by considering the policy structure to improve the convergence speed over that of conventional Q-learning. Finally, simulation examples are presented to support the theoretical findings of this paper.

Keywords

Activity tracking, fast adaptation, Internet of Things, Markov decision processes, wireless charging

Discipline

Computer Sciences | Software Engineering

Research Areas

Software and Cyber-Physical Systems

Publication

IEEE Transactions on Vehicular Technology

Volume

66

Issue

7

First Page

5995

Last Page

6008

ISSN

0018-9545

Identifier

10.1109/TVT.2016.2628966

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1109/TVT.2016.2628966

Download

Share

COinS