RT2ActionTokenizer<T>
Models & Types · AiDotNet.VisionLanguage.Robotics
RT-2’s action-as-text tokenizer.
How It Works
Implements the RT-2 paper’s action representation: each continuous action dimension is
uniformly discretized into NumBins bins (default 256) over its range, and each
bin is associated with one of the NumBins least frequently used tokens in the
vision-language model’s vocabulary. The model emits these tokens autoregressively just
as it would emit ordinary text tokens; downstream decoding maps each token back to a
continuous action value in the centre of its bin.
References:
Constructors
| Constructor | Summary |
|---|---|
RT2ActionTokenizer(int actionDim, int numBins, int vocabSize, double[] minRange, double[] maxRange) |
Initialises the tokenizer. |
Properties
| Property | Summary |
|---|---|
int ActionDim { get; } |
Number of continuous action dimensions emitted per timestep. |
int NumBins { get; } |
Number of discrete bins per action dimension (RT-2 paper: 256). |
int TokenIdEndExclusive { get; } |
One past the last action-bin token ID. |
int TokenIdOffset { get; } |
Vocabulary token-ID where the first action bin lives. |
int VocabSize { get; } |
Total vocabulary size — equals TokenIdEndExclusive because the action bins live in the LAST NumBins token IDs of the vocab per paper §3.2. |
Methods
| Method | Summary |
|---|---|
Tensor<T> DecodeAction(int[] tokenIds) |
Decodes vocabulary token IDs (one per action dimension) into a continuous action tensor of length ActionDim. |
Tensor<T> DecodeHorizon(int[] tokenIds, int horizon) |
Decodes a flat token stream of length horizon * ActionDim into a horizon-by-dim action tensor. |
bool IsActionToken(int tokenId) |
Returns true when the supplied token ID falls in this tokenizer’s action-bin range. |
int ActionDimOfPosition(int positionInStream) |
Given a position in the emitted action-token stream, returns which continuous action dimension it controls. |
int GreedyActionToken(Tensor<T> logits) |
Selects the action-bin token with the highest logit (greedy argmax over the tokenizer’s vocabulary slice). |
int[] EncodeAction(Tensor<T> continuousAction) |
Encodes a per-dimension continuous action vector into vocabulary token IDs. |
int[] EncodeAction(Vector<T> continuousAction) |
Encodes a per-dimension continuous action vector into vocabulary token IDs. |
int[] EncodeHorizon(Tensor<T> horizonAction, int horizon) |
Encodes a multi-step horizon action (shape [horizon, ActionDim] or [horizon * ActionDim]) into a flat token stream of length horizon * ActionDim. |