Disney Researchers Have Developed An Artificial Intelligence (AI) Tool That Instantly Makes An Actor Appear Younger Or Older In A Scene

There is a growing trend in the entertainment and advertising industries to use photorealistic digital re-aging of faces in video. However, even for experienced painters, the standard 2D painting approach might involve hours or even days of tedious frame-by-frame manual labor. Although studies in facial image re-aging have attempted to automate the answer to this issue, the present methodologies could be more effective due to problems including loss of face identity, low resolution, and inconsistent outcomes over many video frames.

To demonstrate the potential of artificial intelligence in photo-realistically changing footage, Disney researchers have unveiled a revolutionary age-shifting device that can make an actor seem convincingly older or younger without weeks of costly, specialized CGI labor. 

Proposed Methods

Disney’s R&D department has developed an artificial intelligence system to alter an actor’s age in a given scene. To ensure the result is as lifelike as possible, artists can still make tweaks by hand, but the AI tool might do much of the work for them. An individual frame’s worth of aging effects is believed to take the AI no more than five seconds to complete. Disney’s researchers developed FRAN (which stands for face re-aging network) as a neural network trained using a large database containing pairs of randomly generated synthetic faces at varying ages to avoid the need to find thousands of images of real people at different (documented) ages depicting the same facial expression, pose, lighting, and background.

Synthesizing high-quality, longitudinal aging data is another technique utilized for face re-aging visual effects with FRAN. The basic idea here is to devise a method to avoid the seemingly impossible challenge of collecting annotated. These longitudinal picture collections show people of diverse ages, races, and genders from various angles. The goal is to produce several input-output image pairs showing the same (arbitrary) identity at two distinct ages with consistent facial expression, stance, lighting, and backdrop.

How does it work?

Researchers took the time to adapt the tried-and-true U-Net architectural design for FRAN to improve translation quality and re-aging control, given the image-to-image translation problem. Using L1, perceptual, and adversarial losses during training, FRAN is built from paired, synthetic data. Each pixel in the picture to be re-aged is given an input and output age in the form of a single-channel age map, which FRAN receives as part of the 5-channel tensor that makes up the input to FRAN. To construct the final re-aged image, the U-Net predicts per-pixel RGB deltas (offsets) that are superimposed on top of the original image. The network may anticipate the re-aging output as RGB offsets on top of the input picture, preventing severe loss of the input identity rather than learning how to produce faces of multiple identities under diverse expressions, views, and lighting. Good temporal consistency in FRAN’s output is a natural byproduct of the input video frames’ smoothness across time. These features work together to create FRAN, an ideal, production-ready method for adjusting the ages of people’s faces in videos.

 Performance results

• Applying FRAN on video frames yields reliable re-aging results, and it can adapt fluidly to changes in head attitude, lighting, and depth of field.
• FRAN reliably and convincingly re-ages supplied images while preserving the target identity. When looking at the timeline, the aging process is continuous and linear.
• Compared to HRFAE, whose results reveal considerable attenuation of skin detail, FRAN is superior in preserving the input identification and the specific skin detail of the given person.
• Real-world photographs perform nearly as well as simulated ones for FRAN’s processing.

 FRAN is not anticipated to displace many industry jobs for quite some time given that manual VFX work and even actual prosthetic makeup application do not have these limitations. However, there are a few limitations, and studies like this are seldom novel. Disney found that FRAN wasn’t ideal for drastic changes like re-aging to and from extremely early ages and that the greying of scalp hair wasn’t reflected when aging up an actor since it wasn’t included in the dataset used to train the tool.

 To sum it up

There are several benefits to making visual effects simpler, including reducing the workload of already overworked and underpaid artists and making the tools available to filmmakers who do not have Hollywood-sized budgets. Even for huge studios, the ability to automate this labor has a commercial motivation. As a result, businesses like Disney fund studies aimed at improving visual effects’ state of the art; in recent years, some of these studies have focused on how AI may streamline the process.

Disney has plenty of justifications for wanting to create such a device. It could help visual effects artists save time by reducing the work they must put into their projects. It might help keep rising production costs in check and assist low-budget movies in artificially aging their performers. The results of using neural networks and ML to age or de-age a person are convincing enough when applied to still images. Still, they are far from photorealistic when applied to moving video, with temporal artifacts that occur and vanish from frame to frame and the appearance of a person sometimes becoming unrecognizable as the altered video plays. These hundreds of fictitious people were aged and re-aged using pre-existing machine learning aging methods. The resulting data was utilized for training a new neural network termed FRAN (face re-aging network).

Check out the Paper. All Credit For This Research Goes To Researchers on This Project. Also, don’t forget to join our Reddit page and discord channel, where we share the latest AI research news, cool AI projects, and more.

References:

• https://studios.disneyresearch.com/2022/11/30/production-ready-face-re-aging-for-visual-effects/
• https://www.engadget.com/disney-ai-actor-younger-older-reaging-154509639.html
• https://www.theverge.com/2022/12/1/23488336/disneys-ai-tool-aging-actors-vfx-announcement
• https://gizmodo.com/disney-ai-art-vfx-visual-effects-de-age-younger-older-1849835548