Abstract
Assessing human motion similarity from video is challenging because viewpoint changes, scale ambiguity, and pose-estimation noise entangle motion dynamics with subject-specific anatomy. Pose-based metrics can therefore penalize semantically identical motions performed by people with different body scales or limb proportions, a serious limitation for rehabilitation assessment. We propose Body-Scale-Invariant Motion Embeddings, a framework for learning motion representations that reduce anatomical bias while preserving action semantics. Rather than treating intra-skeletal retargeting only as a generation task, we use it as a self-supervised objective for scale-invariant representation learning. Our Conditional Autoencoder conditions the decoder on target skeletal parameters so that the latent code is encouraged to encode motion rather than morphology. We evaluate the approach on synthetic video data and a clinical rehabilitation dataset, using ablations and downstream similarity classification to analyze its utility.