Publication Type
Journal Article
Version
acceptedVersion
Publication Date
8-2025
Abstract
Autonomous rehabilitation support solutions, such as virtual coaches, should provide real-time feedback to improve motor function and maintain patient engagement. However, fully annotated dataset collection for real-time exercise assessment is time-consuming and costly, posing a barrier to evaluating proposed methods. In this work, we present a novel framework that learns a frame-level classifier using weakly annotated videos for real-time assessment of compensatory motions in stroke rehabilitation exercises by generating pseudo-labels at a frame level. We consider three approaches: 1) a baseline approach that uses a source dataset to train a frame-level classifier, 2) a transfer learning approach that uses target dataset video-level labels and parameters learned from a source dataset with frame-level labels, and 3) a semi-supervised approach that leverages a target dataset video-level labels and a small set of frame-level labels. We intend to generalize to a weakly labeled target dataset with new exercises and patients. To validate the approach, we use two datasets annotated on compensatory motions: TULE, an existing video and frame-level labeled dataset of 15 post-stroke patients and three exercises, and SERE, a new dataset of 20 post-stroke patients and five exercises, created by the authors, with video-level labels and a small amount of frame-level labels. We show that a frame-level classifier trained on TULE does not generalize well on SERE ( f1=72.87% ), but our semi-supervised and transfer learning approaches achieve, respectively, f1=78.93% and f1=80.47% . Generating pseudo-labels leads to better frame-level classification results for the target dataset than training a classifier with the source dataset (baseline). Thus, the proposed approach can simplify the customization of virtual coaches to new patients and exercises with low data annotation efforts.
Discipline
Databases and Information Systems
Research Areas
Intelligent Systems and Optimization
Areas of Excellence
Sustainability
Publication
IEEE Transactions on Neural Systems and Rehabilitation Engineering
Volume
33
First Page
3334
Last Page
3345
ISSN
1534-4320
Identifier
10.1109/TNSRE.2025.3602548
Publisher
Institute of Electrical and Electronics Engineers
Citation
CÓIAS, Ana Rita; LEE, Min Hun; BERNARDINO, Alexandre; SMAILAGIC, Asim; MATEUS, Mariana; FERNANDES, David; and TRAPOLA, Sofia.
Learning frame-level classifiers for video-based real-time assessment of stroke rehabilitation exercises from weakly annotated datasets. (2025). IEEE Transactions on Neural Systems and Rehabilitation Engineering. 33, 3334-3345.
Available at: https://ink.library.smu.edu.sg/sis_research/10616
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/TNSRE.2025.3602548