Meet SwimXYZ: A Synthetic Dataset of Swimming Motions and Videos Containing 3.4M Frames Annotated with Ground Truth 2D and 3D Joints

Human motion capture has emerged as a key tool in various industries, including sports, medical, and character animation for the entertainment sector. Motion capture is utilized in sports for multiple purposes, including injury prevention, injury analysis, video game industry animations, and even generating informative visualization for TV broadcasters. Traditional motion capture systems provide solid results in the majority of circumstances. Still, they are expensive and time-consuming to set up, calibrate, and post-process, making them difficult to utilize on a broad scale. These concerns are made worse for aquatic activities like swimming, which bring up unique problems such as marker reflections or the installation of underwater cameras. 

Recent developments have enabled capturing motion from RGB photos and films using simple, affordable devices. These real-time, single-camera systems might open the door for the widespread application of motion capture during sporting events by utilizing existing live video data. It might be used in small structures to enhance amateur athletes’ training programs. However, because of a need for more data, they face several obstacles when using computer vision-based motion capture for swimming. Every Human Pose and Shape (HPS) estimate approach, whether 2D (2D joints, body segmentation) or 3D (3D joints, virtual markers), must extract information from the image. However, computer-vision algorithms trained on traditional datasets need help handling aquatic data since it differs greatly from the training pictures. 

Recent advancements in HPS estimation demonstrated that synthetic data might replace or supplement actual pictures. They introduce SwimXYZ to broaden the application of image-based motion capture techniques in swimming. SwimXYZ is an artificial dataset featuring swimming-specific films annotated with 2D and 3D joints from real swimming pools. The 3.4 million frames of the 11520 movies that make up SwimXYZ vary in camera perspective, subject and water look, lighting, and action. Along with 240 synthetic swimming motion sequences in SMPL format, SwimXYZ offers a variety of body forms and swimming motions. 

Researchers from CentraleSupélec, IETR UMR, Centrale Nantes and Université Technologique de Compiègne established SwimXYZ in this study, a sizable collection of artificial swimming movements and films that will be made available online when the paper is accepted.SwimXYZ’s trials demonstrate the potential for motion capture in swimming, and their goal is to help make it more widely used. Future studies may employ movements in the SMPL format for training pose and motion priors or swimming stroke classifiers in addition to the films given by SwimXYZ for training 2D and 3D pose estimation models. SwimXYZ’s lack of variety in subjects (gender, body type, and swimming suit look) and locations (outside environment, pool floor) may be rectified in future works. Other enhancements can include other annotations (such as segmentation and depth maps) or the addition of additional swimming motions, such as dives and turnarounds.

Check out the Paper and Project Page. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 31k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

We are also on WhatsApp. Join our AI Channel on Whatsapp..