SEEDX<T>
Models & Types · AiDotNet.VisionLanguage.Unified
SEED-X: multi-granularity comprehension and generation model.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → SEEDX<T>
Implements: IUnifiedVisionModel<T>
For Beginners
SEED-X is a unified model for multi-granularity visual understanding and generation from Tencent. Default values follow the original paper settings.
How It Works
SEED-X (Tencent, 2024) is a multimodal model with unified multi-granularity comprehension and generation. It processes visual information at multiple granularity levels from global scene understanding to fine-grained detail recognition, using a SEED visual tokenizer that captures both high-level semantics and low-level visual details for generation tasks.
References:
- Paper: “SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation” (Tencent, 2024)
Example
// Create a SEED-X model for multi-granularity comprehension and generation
// with unified visual understanding at multiple detail levels
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new SEEDX<double>(architecture, "seedx.onnx");
// Training mode with native layers
var trainModel = new SEEDX<double>(architecture, new SEEDXOptions());
Constructors
| Constructor | Summary |
|---|---|
SEEDX(NeuralNetworkArchitecture<T> architecture, SEEDXOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
SEEDX(NeuralNetworkArchitecture<T> architecture, string modelPath, SEEDXOptions options) |
Properties
| Property | Summary |
|---|---|
bool SupportsGeneration { get; } |
|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text from image using SEED-X’s multi-granularity visual comprehension. |
Tensor<T> GenerateImage(string textDescription) |
Generates an image from text using SEED-X’s multi-granularity generation pipeline. |