CMU Research Introduces CoVO-MPC (Covariance-Optimal MPC): A Novel Sampling-based MPC Algorithm that Optimizes the Convergence Rate

Model Predictive Control (MPC) has become a key technology in a number of fields, including power systems, robotics, transportation, and process control. Sampling-based MPC has shown effectiveness in applications such as path planning and control, and it is useful as a subroutine in Model-Based Reinforcement Learning (MBRL), all because of its versatility and parallelizability, 

Despite its strong performance in practice, thorough theoretical knowledge is lacking, particularly with regard to features like convergence analysis and hyperparameter adjustment. In a recent research, a team of researchers from Carnegie Mellon University offered a detailed description of the convergence characteristics of a popular sampling-based MPC technique called Model Predictive Path Integral Control (MPPI).

Understanding MPPI’s convergence behavior is the main goal of the analysis, especially in situations where the optimization is quadratic. This includes cases like time-varying linear quadratic regulator (LQR) systems. The study has proved that, in certain circumstances, MPPI shows at least linear convergence rates. Based on this foundation, the study has expanded to include nonlinear systems that are more broadly defined.

The convergence study from CMU has theoretically led to the creation of a new sampling-based maximum probability correction method called CoVariance-Optimal MPC (CoVO-MPC). CoVO-MPC is unique in optimally scheduling the sampling covariance to maximize the convergence rate. This method, driven by the theoretical results of convergence qualities, constitutes a substantial divergence from the conventional MPPI.

The research has presented empirical data from simulations and real-world quadrotor agile control challenges to validate the efficiency of CoVO-MPC. A significant improvement was seen upon comparing the performance of CoVO-MPC with normal MPPI. CoVO-MPC demonstrated its practical efficiency by outperforming regular MPPI by 43-54% in both simulated environments and real quadrotor control tasks.

The team has summarized their primary contributions as follows. 

1. MPPI Convergence Analysis: The study has introduced the Model Predictive Path Integral Control (MPPI) convergence analysis. In particular, the team has proved that MPPI shrinks towards the ideal control sequence when the total cost is quadratic with respect to the control sequence.

2. The exact relationship between the contraction rate and important parameters, such as sampling covariance (Σ), temperature (λ), and system characteristics, has been established. Beyond the quadratic context, scenarios like strongly convex total cost, linear systems with nonlinear residuals, and general systems have been covered in the research.

3. CoVO-MPC, or Covariance-Optimal MPC: The study has presented a unique sampling-based MPC algorithm called CoVariance-Optimal MPC (CoVO-MPC), which builds on the theoretical conclusions. With the use of offline approximations or real-time computation of the ideal covariance Σ, this approach is intended to maximize the rate of convergence.

4. CoVO-MPC Empirical Evaluation – The suggested CoVO-MPC method has been thoroughly tested on a range of robotic systems, from real-world situations to simulations of Cartpole and quadrotor dynamics. A comparison with the typical MPPI algorithm has shown a significant improvement in performance, ranging from 43% to 54% on various jobs.

In conclusion, this study advances the theoretical knowledge of sampling-based MPC, particularly MPPI, and presents a unique technique that shows notable gains in real-world applications.

Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our Telegram Channel