DeepQNetwork<T>

Models & Types · AiDotNet.NeuralNetworks

Represents a Deep Q-Network (DQN), a reinforcement learning algorithm that combines Q-learning with deep neural networks.

Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → VectorModelLayoutBase<T> → DeepQNetwork<T>

For Beginners

A Deep Q-Network is like a smart decision-maker that learns through trial and error.

Imagine you’re teaching a robot to play a video game:

  • The robot needs to learn which actions (button presses) are best in each situation (game screen)
  • At first, the robot makes many random moves to explore the game
  • Over time, it remembers which moves led to high scores and which led to game over
  • The “Deep” part means it uses a neural network to recognize patterns in complex situations
  • The “Q” refers to “Quality” - how good an action is in a specific situation

For example, in a maze game, the network learns that moving toward the exit is usually better than moving away from it, even if it hasn’t seen that exact maze position before.

How It Works

A Deep Q-Network (DQN) is a reinforcement learning algorithm that uses a neural network to approximate the Q-function, which represents the expected future rewards for taking specific actions in specific states. DQNs overcome the limitations of traditional Q-learning by using neural networks to generalize across states, allowing them to handle problems with large or continuous state spaces. Key features of DQNs include experience replay (storing and randomly sampling past experiences) and the use of a separate target network to stabilize learning.

Example

var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var state = Tensor<float>.CreateRandom(new[] { 1, 4 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new DeepQNetwork<float>(architecture))
    .Build(trainX, trainY);
var qValues = result.Predict(state);

Constructors

Constructor Summary
DeepQNetwork() Initializes a new instance of the DeepQNetwork class with the specified architecture and exploration rate.
DeepQNetwork(NeuralNetworkArchitecture<T> architecture, ILossFunction<T> lossFunction, DeepQNetworkOptions options) Creates a Deep Q-Network using the published RMSProp settings.
DeepQNetwork(NeuralNetworkArchitecture<T> architecture, ILossFunction<T> lossFunction, DeepQNetworkOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata() Gets metadata about this Deep Q-Network model.
Tensor<T> GetQValues(Tensor<T> state) Gets the Q-values for all possible actions in the given state.
int GetAction(Tensor<T> state) Gets an action to take in the given state, balancing exploration and exploitation.
int GetBestAction(Tensor<T> state) Gets the best action to take in the given state based on current Q-values.
void AddExperience(Tensor<T> state, int action, T reward, Tensor<T> nextState, bool done) Adds a new experience to the replay buffer.