Multi-Agent Reinforcement Learning System (Taxi-MARL)

Overview

Taxi-MARL is a multi-agent reinforcement learning system that started with single-agent Taxi problems and evolved to implement coordination strategies across multiple agents. The project shows the progression from basic single-agent learning to multi-agent scenarios using parameter sharing and Independent Q-Learning.

Project Evolution

The system was developed through progressive stages:

Single-Agent Foundation: Established baseline performance with traditional Q-Learning on the Taxi problem
Multi-Agent Expansion: Extended the framework to support multiple coordinating agents
Parameter Sharing: Implemented shared learning across agents to improve sample efficiency
Independent Q-Learning: Developed decentralized learning approach allowing agents to learn independently while coordinating implicitly

Key Achievements

Scalability Analysis: Evaluated performance across 2-5 agents
Utilization Metrics: Analyzed resource utilization and coordination efficiency
Consistent Performance: Achieved stable learning across different agent configurations
Parameter Sharing: Improved learning efficiency by sharing parameters across agents

Technical Implementation

Key components:

Q-Learning Algorithms: Traditional and independent Q-learning variants
State Space Design: Multi-agent environment representation
Coordination: Implicit coordination through parameter sharing
Evaluation: Metrics for learning curves and utilization tracking

Research Insights

Key findings:

Centralized vs decentralized trade-offs in multi-agent learning
How parameter sharing affects learning efficiency in cooperative settings
Scalability challenges as agent count increases
Coordination patterns that emerge from implicit agent communication

Technologies Used

Python • Reinforcement Learning Frameworks • Multi-Agent Systems • Q-Learning • NumPy • Matplotlib

Shyam Krishna Sateesh