Latest Machine Learning Research at MIT Presents a Novel ‘Poisson Flow’ Generative Model (PFGM) That Maps any Data Distribution into a Uniform Distribution on a High-Dimensional Hemisphere

Deep generative models are a popular data generation strategy used to generate high-quality samples in pictures, text, and audio and improve semi-supervised learning, domain generalization, and imitation learning. Current deep generative models, however, have shortcomings such as unstable training objectives (GANs) and low sample quality (VAEs, normalizing flows ). Although recent developments in diffusion and scored-based models attain equivalent sample quality to GANs without adversarial training, the stochastic sampling procedure in these models is sluggish. New strategies for securing the training of CNN-based or ViT-based GAN models are presented.

They suggest backward ODEsamplers (normalizing flow) accelerate the sampling process. However, these approaches have yet to outperform their SDE equivalents. We introduce a novel “Poisson flow” generative model (PFGM) that takes advantage of a surprising physics fact that extends to N dimensions. They interpret N-dimensional data items x (say, pictures) as positive electric charges in the z = 0 plane of an N+1-dimensional environment filled with a viscous liquid like honey. As shown in the figure below, motion in a viscous fluid converts any planar charge distribution into a uniform angular distribution.

A positive charge with z > 0 will be repelled by the other charges and will proceed in the opposite direction, ultimately reaching an imaginary globe of radius r. They demonstrate that, in the r limit, if the initial charge distribution is released slightly above z = 0, this rule of motion will provide a uniform distribution for their hemisphere crossings. They reverse the forward process by generating a uniform distribution of negative charges on the hemisphere, then tracking their path back to the z = 0 planes, where they will be dispersed as the data distribution.

A Poisson flow is a sort of continuous normalizing flow that continuously maps between an arbitrary distribution and an easily sampled one. In practice, the Poisson flow is implemented by solving a pair of forwarding/backward ordinary differential equations (ODEs) caused by the electric field (see figure below) supplied by the N-dimensional version of Coulomb’s law (the gradient of the solution to the Poisson’s equation with the data as sources).

Because electric fields generally relate to the exceptional situation N = 3, they will interchangeably refer to this gradient as the Poisson field. The proposed generative model PFGM has a stable training goal and outperforms prior state-of-the-art continuous flow approaches experimentally. PFGM, as a distinct iterative approach, has two benefits over score-based systems. The PFGM ODE technique delivers quicker sampling speeds than the SDE samplers while maintaining equivalent performance. Second, its backward ODE outperforms reverse-time ODEs of VE/VP/sub-VPSDEs in terms of generation performance and stability on a weaker architecture NSCNv2.

The time variables in these ODE baselines are substantially linked with the sample norms throughout the training period, leading to a less error-tolerant inference. In PFGM, however, the link between the anchoring variable and the sample norm is substantially weaker. The CIFAR-10 dataset demonstrates that PFGM reaches current state-of-the-art performance in the normalizing flow family, with FID/Inception scores of 2.549.62 (w/ DDPM++) and 2.48.68 (w/ DDPM++ deep). It outperforms existing state-of-the-art SDE samplers and gives a 10 to 20 speedup across datasets.

Notably, the backward ODE in PFGM is the only ODE-based sampler that can provide good samples on NCSNv2 without adjustments, while other ODE baselines fail. Furthermore, using a variable number of function evaluations (NFE) ranging from 10 to 100, PFGM proves the resilience to step size in the Euler method. They also demonstrate the applicability of the Poisson field’s invertible forward/backward ODEs for probability evaluation and picture modification, as well as their scaling to higher-resolution photos on the LSUNbedroom 256*256 dataset.

On CIFAR-10, PFGM achieves current best-in-class performance among normalized flow models, with an Inception score of 9.68 and a FID score of 2.48. It also outperforms cutting-edge SDE techniques (e.g., score-based SDEs or Diffusion models) while providing 10x to 20x acceleration on picture production tasks. Furthermore, given a weaker network design, PFGM appears to be more tolerant of estimate mistakes, resilient to step size in the Euler technique, and capable of scaling up to higher-resolution datasets. The code implementation can be found on GitHub.

This Article is written as a research summary article by Marktechpost Staff based on the research paper 'Poisson Flow Generative Models'. All Credit For This Research Goes To Researchers on This Project. Check out the paper and github link.

Please Don't Forget To Join Our ML Subreddit