MuJoCo 3 simulator a result of unified efforts at Google

Listen to this article

It’s been two-plus years since Google DeepMind acquired the MuJoCo physics engine for robotics research and development purposes. DeepMind released the open-source version in May 2022 and recently added some major upgrades to MuJoCo 3, including support for accelerator hardware, improved scalability on CPU and more versatile collision primitives.

MuJoCo said it recently joined forces with colleagues from Brax and Everyday Robots, fellow groups at Google developing another physics engine and robotics platform, respectively. This collaboration led to a “unified robotics simulation team” and, ultimately, MuJoCo 3. Its name stands for “Multi-Joint dynamics with Contact.”

One of the main upgrades, the company said, is that MuJoCo 3 supports accelerated simulation via the new MuJoCo XLA (MJX) module. The team said users can now run simulations at millions of steps per second on Google Cloud TPU or their own accelerator hardware.

MuJoCo said this is especially useful for data-hungry learning regimes such as reinforcement learning, evolution strategies, optimization for model predictive control and more.

Submit a session abstract now to be an event speaker. Submission Deadline: December 15, 2023

MuJoCo adds Python support

DeepMind also said Python users can now better transition between running MuJoCo on CPU, GPU, or TPU. The MJX API is nearly identical to MuJoCo, the company said, employing the same data model and simulation algorithms. Simulations in MJX will generally run in the same way as they do in MuJoCo, and machine learning models trained with MJX will also operate the same, DeepMind claimed.

To speed up a single, large scene, MuJoCo 3 can detect “contact islands” or sub-components of a scene that do not interact. For example, in a scene with two robots, they are assigned to separate islands as long as the robots don’t touch each other.

Islands can be solved for independently, for example, on separate threads. Using a new thread-pool API designed to exploit such parallelization opportunities, a model with 22 humanoids runs three times faster, said the company.

More collision geometries accounted for

Finally, MuJoCo 3 adds support for collision geometries defined via signed distance functions (SDFs), allowing users to create new primitives by specifying the distance from any given location to the closest point on a surface. These geometries are not constrained to be convex. 

The company said it provides examples of what is possible including tori, gears, nuts and bolts. Additionally, users can import any mesh and generate a multiresolution voxelized SDF at model compilation time.

In contrast to meshes, SDFs allow the number of contacts to be independent from the mesh resolution, making the cost of collision detection cheaper and more predictable. MuJoCo 3 SDFs can collide with all existing primitives and meshes.

You can learn about all the changes in MuJoCo 3 in the release page and in its documentation.

Software has history with DeepMind, OpenAI

DeepMind’s robotics team was a MuJoCo customer before the acquisition in 2021. DeepMind won an 2023 RBR50 Robotics Innovation Award for its decision to acquire and make the technology open-source. This makes the software freely available to anyone, which enables more robotics engineers to use an accurate, realistic physics engine and ease some of the challenges of developing complex robotics systems.

It’s easy to forget with the soap opera that has transpired in the past week, but there was a time when OpenAI focused heavily on robotics research. In 2018, OpenAI trained a human-like robot hand to manipulate physical objects with unprecedented dexterity.

The system, called Dactyl, was trained entirely in simulation and transferred its knowledge to reality. Some of the simulation work was done in MuJoCo. OpenAI abandoned its robotics research in early 2021 to focus on artificial intelligence.