← All projects
AIPublishedJul 2026

PhysMent

AI for MATH Workshop, ICML 2026 · PMLR 306

Static benchmarks only test what a model has read. PhysMent makes language models run experiments in a live physics simulator to find the answer.

First page of the PhysMent paper
105classical mechanics scenes
26simulation tools the models call
25–67%accuracy across seven models

The idea

Models apply forces, query object states, advance time and change the scene in MuJoCo before answering, and the physics engine checks the result. Much of what a task needs isn't visible up front. It can only be found by experimenting, watching what happens and adapting.

The finding

Models reached up to 80% on qualitative single-concept tasks, but most fell below 30% on the hardest. The failures were mostly about procedure: answering too early, exploring inefficiently, and not grounding in what the simulator actually showed.