MADDPGAgent<T>

Models & Types · AiDotNet.ReinforcementLearning.Agents.MADDPG

Multi-Agent Deep Deterministic Policy Gradient (MADDPG) agent for cooperative and competitive multi-agent reinforcement learning with continuous action spaces.

Inheritance: ReinforcementLearningAgentBase<T> → DeepReinforcementLearningAgentBase<T> → MADDPGAgent<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

MADDPG enables multiple agents to learn together in shared environments. During training, critics can “see” all agents’ actions (centralized), but during execution, each agent acts independently (decentralized).

Key features:

  • Centralized Critics: Observe all agents during training
  • Decentralized Actors: Independent policies per agent
  • Continuous Actions: Based on DDPG
  • Cooperative or Competitive: Handles both settings

Think of it like: Team sports where players practice together seeing everyone’s moves, but during games each makes independent decisions.

Examples: Robot swarms, traffic control, multi-player games

How It Works

MADDPG extends DDPG to multi-agent settings with centralized training and decentralized execution.

Example

// Create a multi-agent DDPG system with 3 cooperative agents
var options = new MADDPGOptions<double> { NumAgents = 3, ActorLearningRate = 0.001 };
var agent = new MADDPGAgent<double>(options);
            
// Each agent selects a continuous action from its observation
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2, 0.8, -0.1, 0.4, 0.6 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
MADDPGAgent()
MADDPGAgent(MADDPGOptions<T> options) Initializes a new instance with the given options.

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
T Train()
Task TrainAsync()
Task<Vector<T>> PredictAsync(Vector<T> input)
Vector<T> Predict(Vector<T> input)
Vector<T> SelectAction(Vector<T> state, bool training)
Vector<T> SelectActionForAgent(int agentId, Vector<T> state, bool training) Selects an action for a specific agent from its own actor network.
byte[] Serialize() Auto-generated common model serialization surface.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath) Loads a trained model from a file.
void ResetEpisode()
void SaveModel(string filepath) Saves the trained model to a file.
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)
void StoreMultiAgentExperience(List<Vector<T>> states, List<Vector<T>> actions, List<T> rewards, List<Vector<T>> nextStates, bool done) Store multi-agent experience with per-agent reward tracking.
void Train(Vector<T> state, Vector<T> target) Performs a one-shot supervised update for the training/test harness.