Florence2Options
Options & Configuration · AiDotNet.VisionLanguage.Encoders
Configuration options for Florence-2, Microsoft’s unified vision foundation model.
Inheritance: ModelOptions → VisionEncoderOptions → Florence2Options
How It Works
Florence-2 (Xiao et al., 2024) is a sequence-to-sequence vision foundation model of 0.23B to 0.77B parameters. It handles captioning, object detection, grounding and OCR through task prompts. A DaViT image encoder feeds projected image tokens, followed by the prompt, to a BART encoder-decoder. The output is generated text whose regions are written as location tokens.
The inherited options mean the following here:
EmbeddingDimis the projected image-token width. It must equal
TextDim, because image tokens and text share the BART encoder.
the BART encoder’s depth and heads.
FfnMultiplieris DaViT’s MLP ratio.DropoutRateis BART’s dropout.PatchSizeis DaViT’s total stride, 32 (4 x 2 x 2 x 2). The
architecture fixes it.
Constructors
| Constructor | Summary |
|---|---|
Florence2Options() |
Creates Florence-2-base options. |
Florence2Options(Florence2ModelSize size) |
Copies every option from other. |
Florence2Options(Florence2Options other) |
Copies every option from other. |
Properties
| Property | Summary |
|---|---|
Florence2ModelSize ModelSize { get; } |
The published size these options were created from. |
Florence2Task DefaultTask { get; set; } |
The task Predict runs when no task is named. |
int MaxOutputTokens { get; set; } |
Most tokens one generation produces before it stops. |
int MaxTextPositions { get; set; } |
BART’s learned positions; bounds both the encoder input and the decoder output. |
int NumDecoderLayers { get; set; } |
Number of BART decoder layers (6 or 12). |
int NumLocationBins { get; set; } |
Number of location tokens <loc_0>..<loc_N-1>, each a quantised 0-1 coordinate. |
int TextDim { get; set; } |
BART model width, shared by encoder and decoder (768 for base, 1024 for large). |
int TextFeedForwardDim { get; set; } |
BART feed-forward width (3072 or 4096). |
int VisionBaseDim { get; set; } |
DaViT stage-0 width. |
int VisionBaseHeads { get; set; } |
DaViT stage-0 heads, also its channel-attention group count. |
int VisionThirdStageDepth { get; set; } |
Depth of DaViT’s third stage. |
int VocabSize { get; set; } |
Vocabulary size: BART’s 50265 BPE tokens plus the added task and location tokens. |
int WindowSize { get; set; } |
Side of DaViT’s spatial attention windows. |
Methods
| Method | Summary |
|---|---|
void Validate() |
Throws if the configuration cannot build a Florence-2 model. |