Researchers at Apple Develop Texturify: A GAN-based Approach for Generating Textures on 3D Shape Surfaces

The development of 3D content for visual consumption in movies, video games, and mixed reality environments is one of many application areas where it is essential. A significant improvement in geometric fidelity has been achieved in recent years thanks to new generative learning techniques applied to large-scale 3D shape datasets. While modeling geometry has shown significant potential, creating fully textured 3D objects has received less attention.

The creation of textured 3D content necessitates time-consuming hand labor. The scarcity of high-quality textured 3D data presents a significant obstacle to learning how to automatically generate textured 3D material. Large-scale shape datasets like ShapeNet have contributed to the success of 3D geometric shape modeling, although they frequently feature uniform and basic texturing for the objects. Furthermore, since textures are only well-defined on geometric surfaces, existing texture generation approaches primarily mimic popular generative geometric representations that implicitly define surfaces over a volume in space. This leads to inefficient learning and frequently yields blurry results.

In a recent study, researchers at Apple proposed Texturify to address these issues in the process of autonomous texture synthesis for 3D shape collections. In other words, Texturify learns to automatically create a range of various textures on a shape when sampling from a latent texture space for a given shape geometry. Researchers used merely a set of photos and a collection of 3D shape geometry from the same class category instead of depending on supervision from 3D textured objects. They did not require any relationship between image and geometry or any semantic part information of the shapes.

To make sure that generated textures on the 3D shapes presented realistic imagery from a range of viewpoints during training, the researchers used differentiable rendering with an adversarial loss. The researchers suggested tying texture production directly to the surface of the 3D shape rather than generating textures for 3D shapes defined over a volume in space as has been done with implicit representations or volumetric representations.

The group developed a generative adversarial network to operate on the faces of a 4-way rotationally symmetric quad mesh by creating convolutional face operators for texture generation, conditioned on the 3D shape geometry and a latent texture coding. The method permits producing potential shape textures with awareness of 3D structural neighborhood relations and little distortion, in contrast to the conventional 2D texture parameterization with UV maps.

The researchers conducted numerous tests to demonstrate Texturify’s efficacy in texturing ShapeNet automobiles and chairs that were trained using real-world footage. The experiments revealed that the method performed, on average, 22% FID scores better than state-of-the-art.

Conclusion

In a recent publication, researchers at Apple developed Texturify, a new method for creating textures on mesh surfaces using different collections of 2D picture collections and 3D shape geometry, i.e., without the need for any explicit 3D color supervision or 2D to 3D correspondences. The texture creation method creates high-quality, cohesive textures by working directly on a given mesh surface. Experiments demonstrate that the state-of-the-art techniques are outperformed both statistically and qualitatively when the 4-RoSy parameterization is combined with face convolutions that use geometric characteristics as input. The Texturify team thinks that by automatically generating textures for 3D objects that can be used in conventional computer graphics pipelines, Texturify is a crucial step in the creation of 3D content.

This Article is written as a research summary article by Marktechpost Staff based on the research paper 'Texturify: Generating Textures on
3D Shape Surfaces'. All Credit For This Research Goes To Researchers on This Project. Check out the paper and project link.

Please Don't Forget To Join Our ML Subreddit