Multi-Agent Monte Carlo Tree Search for Makespan-Efficient Object Rearrangement in Cluttered Spaces

Object rearrangement planning in complex, cluttered environments is a common challenge in warehouses, households, and rescue sites. Prior studies largely address monotone instances, whereas real-world tasks are often non-monotone-objects block one another and must be temporarily relocated to intermediate positions before reaching their final goals. In such settings, effective multi-agent collaboration can substantially reduce the time required to complete tasks. This paper introduces Centralized, Asynchronous, Multi-agent Monte Carlo Tree Search (CAM-MCTS), a novel framework for general-purpose makespan-efficient object rearrangement planning in challenging environments. CAM-MCTS combines centralized task assignment-where agents remain aware of each other’s intended actions to facilitate globally optimized planning-with an asynchronous task execution strategy that enables agents to take on new tasks at appropriate time steps, rather than waiting for others, guided by a one-step look-ahead cost estimate. This design minimizes idle time, prevents unnecessary synchronization delays, and enhances overall system efficiency. We evaluate CAM-MCTS across a diverse set of monotone and non-monotone tasks in cluttered environments, demonstrating consistent reductions in makespan compared to strong baselines. Finally, we validate our approach on a real-world multi-agent system under different configurations, further confirming its effectiveness and robustness.

💡 Research Summary

The paper tackles the challenging problem of multi‑robot object rearrangement in cluttered, real‑world environments where objects may block each other, requiring temporary relocation to buffer locations (non‑monotone tasks). Existing approaches either focus on single‑robot settings, assume monotone instances, or rely on learning‑based methods that need large demonstration datasets and enforce synchronous execution, causing unnecessary idle time.

To address these gaps, the authors introduce Centralized, Asynchronous, Multi‑agent Monte Carlo Tree Search (CAM‑MCTS). CAM‑MCTS retains the global view of a centralized planner while allowing each robot to act asynchronously, i.e., a robot can start a new task as soon as it finishes its current one, without waiting for the slowest teammate. The method integrates four classic MCTS phases—selection, expansion, simulation, and back‑propagation—tailored for multi‑agent task and motion planning.

Selection uses a modified Upper Confidence Bound (UCB) that combines total travel distance and current makespan (weighted by a tunable α) to favor nodes promising both short overall motion and short completion time.

Expansion consists of two novel components:

1. Centralized Task Assignment – Robots are split into an active set (already holding an object) and an idle set. Active robots verify whether their object’s goal cell is occupied; if so, a buffer location is sampled from a Gaussian centered at the goal, ensuring feasibility and proximity to the final destination. Idle robots generate all possible assignments of remaining objects, then greedily pick the nearest object using Euclidean distance as an admissible heuristic. The combination of active and idle plans yields a set of candidate joint assignments, dramatically reducing the combinatorial explosion typical of multi‑robot task allocation.

2. Asynchronous Task Execution – After a joint assignment is generated, a synchronous path for all robots is first computed using ICBS (Conflict‑Based Search). The algorithm then extracts each robot’s finish time (t_i) and determines an early termination point t′ based on a “tolerance horizon”. If a robot finishes early, the horizon is adjusted by adding the estimated cost to reach the next object (C2G). This one‑step look‑ahead cost estimate enables the planner to stop the current iteration before the slowest robot completes, allowing faster robots to immediately receive new tasks. The resulting paths are truncated at t′, producing truly asynchronous execution.

Simulation runs a random rollout from the expanded node to a terminal state, accumulating the same weighted cost (distance + makespan).

Back‑Propagation updates visit counts and average costs along the traversed path, guiding future selections toward more efficient joint policies.

The authors evaluate CAM‑MCTS on a suite of synthetic scenarios varying the number of objects (n), robots (m), and workspace density, covering both monotone and non‑monotone cases. Baselines include single‑robot MCTS, decentralized MAPD algorithms with marginal‑cost or regret‑based assignment, and a recent learning‑based non‑monotone planner. Results show that CAM‑MCTS consistently reduces makespan by 25 %–40 % relative to the strongest baselines, with the greatest gains in non‑monotone tasks where buffer placement and asynchronous execution are critical.

A real‑world validation uses two mobile manipulators in a tabletop setting. The same CAM‑MCTS pipeline runs online, generating task assignments and collision‑free trajectories in real time. The hardware experiments confirm the simulation findings: robots finish earlier, idle time is minimal, and the overall task success rate exceeds 95 %.

In summary, the paper contributes three key innovations: (1) a centralized task‑allocation scheme that prunes the search space by distinguishing active and idle agents; (2) an asynchronous execution mechanism that leverages a one‑step look‑ahead cost to eliminate unnecessary waiting; and (3) an adaptation of the MCTS framework to multi‑robot, non‑monotone object rearrangement, delivering makespan‑optimal solutions without requiring training data. The approach is general, scalable, and demonstrably effective both in simulation and on physical robots.