Researchers at NTU Singapore Propose PointHPS: An AI Framework for Accurate Human Pose and Shape Estimation from 3D Point Clouds

With several advancements in the field of Artificial Intelligence, human pose and shape estimation (HPS) has become an increasingly important research area in recent years. With several practical applications, including motion capture, virtual try-on, and mixed reality, recovering 3D human bodies has become a significant challenge. Estimating poses and how the body is arranged, along with analyzing the shapes and the physical properties of the body of individuals in 3D space, is a step in this process. One example is using parametric human models, like the SMPL model, which depict human bodies with shape and position characteristics.

Predicting these parametric models from 2D photos has become significantly easier in recent years. However, in some circumstances, 2D images offer drawbacks, such as depth ambiguity and privacy issues. This is the situation when 3D point cloud data is useful. Accurately estimating human poses and shapes from 3D point clouds has become possible thanks to the advancement of depth sensors and the accessibility of large-scale datasets.

In recent research, a team of researchers has introduced a methodical framework termed PointHPS for precise 3D HPS from point clouds acquired in real-world environments. PointHPS uses a cascaded design in which point characteristics are repeatedly refined at each iteration. It uses an iterative refinement process in which the input point cloud data is subjected to a number of downsampling and upsampling techniques at various stages. These processes seek to extract from the data both local and global cues.

Two cutting-edge modules have been included in PointHPS to improve the feature extraction procedure. First is Cross-stage Feature Fusion (CFF), which is a module that enables multi-scale feature propagation, enabling efficient information transfer between the various network stages. It helps in context preservation and information capture. Second is IFE (Intermediate Feature Enhancement), which concentrates on collecting characteristics in a manner that is conscious of the structure of the human body. After each stage, the quality of the features is increased, making them better suited for precise posture and form estimation.

The team has run tests on two substantial benchmarks to provide a thorough evaluation under varied conditions –

1. Real-world dataset: This dataset contains a variety of participants and actions that were recorded in a lab setting using genuine commercial sensors. It represents a more difficult and realistic environment.

2. Dataset generation: This dataset was meticulously created taking into account genuine conditions, such as dressed people in busy outdoor settings. Control over a variety of environmental parameters was also provided.

Extensive testing has revealed that PointHPS beats state-of-the-art techniques across all assessment measures with its robust approach to point feature extraction and processing. The effectiveness of the suggested cascaded architecture, which is improved by the CFF and IFE modules, is further supported by ablation investigations. The team intends to release their pretrained models, code, and data for use in additional HPS from point cloud research. Future research in this area should be made easier, which will also increase the ability to accurately estimate 3D human position and shape from real-world point cloud data.

Check out the Paper, Project Page, and Github. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 29k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..