Alessio Pitteri — Fondazione Bruno Kessler - Università di Bologna #
Ant Swarm Functional Control via Stigmergic Reinforcement Learning Agents #
In this work, we investigate the functional controllability of a well-known model of collective behavior: the Ant Swarm Model. We introduce a population of controlling agents, trained via Reinforcement Learning (RL), to influence the dynamics of the system and promote the emergence of ordered behavior. Smart-agents are optimized with Proximal Policy Optimization in a centralized-training decentralized-execution setting, and interact with ants only through the shared pheromone field. The reward design promotes trail pheromone structures and alignment of ant positions with high-pheromone paths, without requiring control of specific microscopic configurations. Our results demonstrate that the learned policies effectively shift the phase transition line that characterizes the global behavior of the system, enabling the formation of organized trails in regimes that are typically dominated by randomness. Compared with both the baseline system and an enhanced-hybrid setup, with equal pheromone reinforcement but no learned policy, our approach yields consistently higher ordering and quality metrics across the phase diagram. This study provides insights into the potential of RL based control strategies for stigmergic systems and contributes to the general understanding of functional controllability in complex systems.