Model Predictive Control (MPC) has change into a key technology in a variety of fields, including power systems, robotics, transportation, and process control. Sampling-based MPC has shown effectiveness in applications akin to path planning and control, and it is beneficial as a subroutine in Model-Based Reinforcement Learning (MBRL), all due to its versatility and parallelizability, 

Despite its strong performance in practice, thorough theoretical knowledge is lacking, particularly with regard to features like convergence evaluation and hyperparameter adjustment. In a recent research, a team of researchers from Carnegie Mellon University offered an in depth description of the convergence characteristics of a preferred sampling-based MPC technique called Model Predictive Path Integral Control (MPPI).

Understanding MPPI’s convergence behavior is the important goal of the evaluation, especially in situations where the optimization is quadratic. This includes cases like time-varying linear quadratic regulator (LQR) systems. The study has proved that, in certain circumstances, MPPI shows no less than linear convergence rates. Based on this foundation, the study has expanded to incorporate nonlinear systems which are more broadly defined.

The convergence study from CMU has theoretically led to the creation of a brand new sampling-based maximum probability correction method called CoVariance-Optimal MPC (CoVO-MPC). CoVO-MPC is exclusive in optimally scheduling the sampling covariance to maximise the convergence rate. This method, driven by the theoretical results of convergence qualities, constitutes a considerable divergence from the standard MPPI.

The research has presented empirical data from simulations and real-world quadrotor agile control challenges to validate the efficiency of CoVO-MPC. A major improvement was seen upon comparing the performance of CoVO-MPC with normal MPPI. CoVO-MPC demonstrated its practical efficiency by outperforming regular MPPI by 43-54% in each simulated environments and real quadrotor control tasks.

The team has summarized their primary contributions as follows. 

1. MPPI Convergence Evaluation: The study has introduced the Model Predictive Path Integral Control (MPPI) convergence evaluation. Particularly, the team has proved that MPPI shrinks towards the best control sequence when the full cost is quadratic with respect to the control sequence.

2. The precise relationship between the contraction rate and necessary parameters, akin to sampling covariance (Σ), temperature (λ), and system characteristics, has been established. Beyond the quadratic context, scenarios like strongly convex total cost, linear systems with nonlinear residuals, and general systems have been covered within the research.

3. CoVO-MPC, or Covariance-Optimal MPC: The study has presented a novel sampling-based MPC algorithm called CoVariance-Optimal MPC (CoVO-MPC), which builds on the theoretical conclusions. With the usage of offline approximations or real-time computation of the best covariance Σ, this approach is meant to maximise the speed of convergence.

4. CoVO-MPC Empirical Evaluation – The suggested CoVO-MPC method has been thoroughly tested on a variety of robotic systems, from real-world situations to simulations of Cartpole and quadrotor dynamics. A comparison with the everyday MPPI algorithm has shown a major improvement in performance, starting from 43% to 54% on various jobs.

In conclusion, this study advances the theoretical knowledge of sampling-based MPC, particularly MPPI, and presents a novel technique that shows notable gains in real-world applications.

Take a look at the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.

In the event you like our work, you’ll love our newsletter..

Don’t Forget to affix our Telegram Channel