A Fairness-Oriented Multi-Objective Reinforcement Learning approach for Autonomous Intersection Management

This study introduces a novel multi-objective reinforcement learning (MORL) approach for autonomous intersection management, aiming to balance traffic efficiency and environmental sustainability across electric and internal combustion vehicles. The proposed method utilizes MORL to identify Pareto-optimal policies, with a post-hoc fairness criterion guiding the selection of the final policy. Simulation results in a complex intersection scenario demonstrate the approach’s effectiveness in optimizing traffic efficiency and emissions reduction while ensuring fairness across vehicle categories. We believe that this criterion can lay the foundation for ensuring equitable service, while fostering safe, efficient, and sustainable practices in smart urban mobility.

💡 Research Summary

The paper tackles the emerging challenge of managing heterogeneous traffic—specifically electric (EV) and internal‑combustion (IC) vehicles—at unsignalized urban intersections. While most reinforcement‑learning (RL) approaches for autonomous intersection management (AIM) focus solely on global efficiency (e.g., minimizing total delay) or safety, they ignore broader societal concerns such as environmental impact and fairness among different vehicle classes. To address this gap, the authors propose a fairness‑oriented multi‑objective reinforcement learning (MORL) framework that simultaneously optimizes three potentially conflicting objectives: traffic efficiency, CO₂ emissions, and safety, while later selecting the most equitable policy across vehicle categories.

Technical Contributions

1. Graph‑Based State Augmentation – Building on prior graph representations of AIM, each vehicle is a graph node whose feature vector now includes a binary type indicator (k = 0 for petrol, k = 1 for electric). Edge types are also expanded to encode the fuel combination of the two incident vehicles (pp, pe, ep, ee). This enriched representation enables the RL agent to perceive and exploit heterogeneity directly.

2. Multi‑Objective Reward Design – The reward consists of three components:

   • Efficiency (R_eff) rewards sustained speeds relative to the speed limit using a piecewise nonlinear function that encourages high speeds without excessive overshoot.
   • Environmental (R_env) penalizes the total CO₂ emitted by petrol vehicles each timestep.
   • Safety (R_saf) imposes a large penalty for collisions (‑10), a small penalty for a complete stand‑still (‑1), and zero otherwise. Crucially, R_saf is not scaled by the trade‑off weight, reflecting safety as a non‑negotiable objective.

3. Unified MORL via Randomized Trade‑off Parameter – Instead of training separate agents for each weighting of efficiency versus emissions, a single TD3‑based actor‑critic network learns a continuum of policies. At every training step a weight ω ∈