Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

5-2026

Abstract

Generating realistic 3D hand motion from natural language is vital for VR, robotics, and human-computer interaction. Existing methods either focus on full-body motion, overlooking detailed hand gestures, or require explicit 3D object meshes, limiting generality. We propose TSHaMo, a model-agnostic teacher-student diffusion framework for text-driven hand motion generation. The student model learns to synthesize motions from text alone, while the teacher leverages auxiliary signals (e.g., MANO parameters) to provide structured guidance during training. A co-training strategy enables the student to benefit from the teacher’s intermediate predictions while remaining text-only at inference. Evaluated using two diffusion backbones on GRAB and H2O, TSHaMo consistently improves motion quality and diversity. Ablations confirm its robustness and flexibility in using diverse auxiliary inputs without requiring 3D objects at test time.

Keywords

Hand motion generation, Teacher-student model, Diffusion model, Auxiliary learning

Discipline

Artificial Intelligence and Robotics | Graphics and Human Computer Interfaces

Advisors/Committee Chairs

NA

Degree Awarded

PhD in Computer Science

First Page

1

Last Page

5

ISBN

9798331567026

Identifier

10.1109/ICASSP55912.2026.11464505

Publisher

IEEE

Additional URL

https://doi.org/10.1109/ICASSP55912.2026.11464505

Download

Share

COinS