'LATENT' teaches tennis skills to humanoid robots using incomplete human motion data.

A research team at Tsinghua University in China has announced ' LATENT ,' a system that teaches tennis skills to highly athletic humanoids using incomplete human motion data. LATENT provides a humanoid learning pipeline for pre-training , knowledge distillation , and high-level policy learning, and uses the general-purpose physics engine MuJoCo for simulation.

LATENT
https://zzk273.github.io/LATENT/

Human athletes utilize versatile and highly dynamic tennis skills to return fast-moving tennis balls. However, replicating such movements with a humanoid is extremely difficult. This is because there is a lack of perfect humanoid motion data or human kinematic motion data to use as a reference.

Therefore, a research team at Tsinghua University has developed 'LATENT,' a system that learns the tennis skills of highly athletic humanoids from incomplete human motion data. The incomplete human motion data consists only of motion fragments that capture the basic skills used when playing tennis, rather than accurate and complete human tennis motion sequences obtained from actual tennis matches, which significantly reduces the difficulty of data collection.

GitHub - GalaxyGeneralRobotics/LATENT: Official implementation of Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data · GitHub
https://github.com/GalaxyGeneralRobotics/LATENT

Despite the incompleteness of this data, the research team argues it can be helpful in providing prior information about fundamental human skills in tennis. Furthermore, by modifying and synthesizing the data, it may be possible to train a humanoid that can maintain a natural movement style, consistently returning incoming balls under a wide range of conditions and hitting them to a target location.

In addition, by proposing multiple design options to achieve robust data transfer from simulation to the real world, we have successfully made Unitree G1 actually play tennis.

You can see in the following video how LATENT was used to train Unitree G1 to perform tennis movements and how it actually played tennis with a human.

Results of training a Unitree G1 robot to play tennis using 'LATENT,' a system that teaches tennis skills to humanoid robots from incomplete human motion data - YouTube