TokenFlowModel<T>

Models & Types · AiDotNet.Diffusion.Video.VideoEditing

TokenFlow consistent video editing via token flow propagation.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → TokenFlowModel<T>

For Beginners

TokenFlow enables consistent video editing by propagating visual changes through time using token flow, tracking how image features move between frames. This ensures edits look natural and consistent across the entire video.

How It Works

References:

  • Paper: “TokenFlow: Consistent Diffusion Features for Consistent Video Editing” (Bar-Tal et al., 2023)

TokenFlow achieves consistent video editing by explicitly enforcing semantic correspondences of diffusion features across video frames. The method propagates edited tokens from keyframes to all frames using the natural token flow, maintaining temporal consistency without per-frame processing. Works with any text-guided image editing method.

Technical specifications:

  • Architecture: Token Flow Propagation + DDIM Inversion + Keyframe Editing
  • Latent channels: 4
  • Default: 24 frames at 8 FPS
  • Supports I2V: No | T2V: Yes | V2V: Yes

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 50 };
var input = Tensor<float>.CreateRandom(new[] { 1, 4, 24, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new TokenFlowModel<float>(options: options))
    .Build(trainX, trainY);
var edited = result.Predict(input);

Constructors

Constructor Summary
TokenFlowModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, VideoUNetPredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS) Initializes a new instance of TokenFlowModel with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> TemporalVAE { get; }
IVAEModel<T> VAE { get; }
bool SupportsImageToVideo { get; }
bool SupportsTextToVideo { get; }
bool SupportsVideoToVideo { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()