The University of Texas Austin Researchers Propose HM3D-ABO: A Photo-realistic Dataset for Object-Centric Multi-View 3D Reconstruction

Since the rise in popularity of AR/VR applications, researchers have been studying the process of reconstructing 3D objects. Researchers can create data-driven algorithms for object reconstruction using multi-view renderings from object model collecting datasets. However, the clean backdrops create a gap with real-world photos.

Things3D has made an effort to provide those object renderings as alternative backdrops by inserting CAD models into fabricated 3D indoor environments. Their simulations of synthetic interior spaces, nevertheless, are still a long way from being realistic. On the other side, a number of efforts, like Pix3D and Scan2CAD, have attempted to produce 3D object annotations from actual 2D pictures.

However, these works either don’t offer a precise 3D model or just offer a single-view image setup. Redwood includes multi-view 2D photos and 3D depth scans, although it only occasionally offers reconstructed 3D models. Recent releases of the Google Objectron dataset include a selection of quick, object-focused video clips with 3D object-level annotations, but no reconstructed 3D models are included.

Researchers at the University of Texas recently presented a method to produce a photo-realistic object-centric dataset based on current high-quality scene and object models in response to the difficulty of producing correct annotations for real-world item scans.

Researchers used two different sorts of elements to generate the dataset: scene and object. The scene assets were taken from the just-released Habitat-Matterport 3D dataset. These scenes were captured in high definition from actual indoor settings. For the object assets, the team selected the Amazon-Berkeley Object dataset since it has precise geometry and high-quality texture.

The group selects a scene and an item at random. Then, by sampling the navigable position in the scene, they determine the placement location of such an object in that scenario. By ensuring that the area around the selected place is empty, a physical collision between the inserted object and the rest of the scene is prevented. By performing a few physical simulation stages, they also ensure that the object is positioned on the scene’s floor rather than floating in midair.

Conclusion

The University of Texas proposed a method to produce the photo-realistic object-centric dataset HM3D-ABO in a recent study. In contrast to datasets based on ShapeNet and synthetic scenes, the generated photos include realistic objects and backgrounds. Additionally, it includes top-notch 3D models, which are missing from several real-captured object-centric datasets. The team contends that the HM3D-ABO dataset is more realistic than the earlier synthetic object-centric datasets, but they also highlight a number of drawbacks. First, there are only a few categories available for the object assets. The existing system for item placement is also constrained. The team expects that the dataset will advance the field and that further work on it will help get beyond its drawbacks.

This Article is written as a summary article by Marktechpost Staff based on the paper 'HM3D-ABO: A Photo-realistic Dataset for Object-centric Multi-view 3D Reconstruction Technical Report'. All Credit For This Research Goes To Researchers on This Project. Checkout the paper and github.

Please Don't Forget To Join Our ML Subreddit