In automated warehouses, efficient management and economic benefits hinge on the effective scheduling of automated guided vehicles (AGVs) to transport diverse packets. Emerging technologies such as artificial intelligence and automation control have greatly contributed to the development of packet transport schemes for AGVs. However, the development of the logistics industry results in a massive amount of packets with diverse deadlines, which brings new challenges for the AGV scheduling system. To address this, this paper treats each AGV as an agent and designs a novel hierarchical transformer reinforcement learning (HTRL) framework to generate efficient AGV scheduling policies. Specifically, this framework consists of one encoder and two decoders to produce the packet selection and path improvement actions. These two decoders are equipped with masked self-attention mechanisms to learn efficient packet selection and path improvement policies, facilitating AGV transport efficiency to meet the deadlines of packets. Moreover, we consider the kinetic features of AGVs and design a model predictive control (MPC)-based speed control method for AGVs to prevent frequent stop-and-wait of AGVs and enhance their transport efficiency. We build up a simulated warehouse environment containing packets with different deadlines and conduct extensive experiments. Experimental results validate that the proposed HTRL framework increases the delivered packets within expiration by up to 36.6% compared to other baselines.

Effective multi-intersection collaboration is crucial for mitigating urban traffic congestion through reinforcement learning (RL)-based traffic signal control (TSC). Existing work mainly considers scenarios involving a single vehicle type, where cooperation is typically limited to neighboring intersections. However, in urban traffic scenarios where high priority vehicles coexist with ordinary vehicles, considering only a limited number of neighboring nodes may be insufficient to ensure the swift passage of high priority vehicles while minimizing the impact on overall traffic efficiency. Therefore, we formulate the multiple intersections' decision-making process in urban scenarios as a Markov game and propose a novel centralized cooperative RL framework called MATLIT to solve the game. Specifically, we adopt a multi-agent transformer (MAT)-based architecture that facilitates efficient global cooperation among intersections. The attention mechanism and auto-regressive process of the MAT effectively mitigate the curse of the dimensionality problem, which guarantees MATLIT to tackle large-scale traffic scenarios. Meanwhile, the stability and sequence action generation capacity of the MAT-based architecture is further enhanced by incorporating MAT with a gated mechanism. Furthermore, considering the inherent topological constraints in urban traffic scenarios, we utilize graph attention networks (GATs) to capture graph-structured mutual influences. Additionally, in response to the urban traffic scenarios with various types of high priority vehicles that have time-varying priorities, we integrate the soft actor-critic (SAC) algorithm to enhance the exploration capabilities of our framework, allowing it to learn robust strategies in heterogeneous traffic conditions. Extensive experiments demonstrate that our proposed MATLIT framework outperforms all baselines and can reduce high priority vehicles' waiting time by 24.57% while reducing the average waiting time of all vehicles by 18.51% in realistic urban scenarios.