Joint Optimization of Task Offloading and Image–Container Caching Based on Hierarchical Multi-Agent Reinforcement Learning in Containerized MEC Networks

Future Internet applications such as intelligent transportation, immersive services, and edge-assisted artificial intelligence require latency-sensitive service provisioning at the network edge. In containerized mobile edge computing (MEC), service orchestration is not only a task-offloading problem, but also a task–container–image constrained decision problem: an offloaded task can be executed only when the required runtime container is active, and a newly activated container must be supported by a locally cached service image. This dependency couples task placement, runtime container caching, and persistent image caching under limited RAM and ROM resources. To address this challenge, this paper proposes HAM-MADDPG, a dependency-aware hierarchical action-masked multi-agent reinforcement learning algorithm for joint task offloading and image–container caching in containerized MEC networks. HAM-MADDPG decomposes the monolithic orchestration decision into three causally ordered policy layers: task offloading, runtime container caching, and persistent image caching. Each layer learns a structured subproblem conditioned on upstream realized decisions, while dynamic action masking and feasibility-aware action realization guide the learned policies toward executable decisions satisfying task–container and container–image constraints. Extensive simulations under dynamic service demands and heterogeneous edge resources show that HAM-MADDPG achieves more stable convergence than non-hierarchical reinforcement learning baselines and reduces long-term system latency by approximately 14–25% compared with representative heuristic and flat DRL baselines.