Florence2Options

Options & Configuration · AiDotNet.VisionLanguage.Encoders

Configuration options for Florence-2, Microsoft’s unified vision foundation model.

Inheritance: ModelOptions → VisionEncoderOptions → Florence2Options

How It Works

Florence-2 (Xiao et al., 2024) is a sequence-to-sequence vision foundation model of 0.23B to 0.77B parameters. It handles captioning, object detection, grounding and OCR through task prompts. A DaViT image encoder feeds projected image tokens, followed by the prompt, to a BART encoder-decoder. The output is generated text whose regions are written as location tokens.

The inherited options mean the following here:

  • EmbeddingDim is the projected image-token width. It must equal

TextDim, because image tokens and text share the BART encoder.

the BART encoder’s depth and heads.

architecture fixes it.

Constructors

Constructor Summary
Florence2Options() Creates Florence-2-base options.
Florence2Options(Florence2ModelSize size) Copies every option from other.
Florence2Options(Florence2Options other) Copies every option from other.

Properties

Property Summary
Florence2ModelSize ModelSize { get; } The published size these options were created from.
Florence2Task DefaultTask { get; set; } The task Predict runs when no task is named.
int MaxOutputTokens { get; set; } Most tokens one generation produces before it stops.
int MaxTextPositions { get; set; } BART’s learned positions; bounds both the encoder input and the decoder output.
int NumDecoderLayers { get; set; } Number of BART decoder layers (6 or 12).
int NumLocationBins { get; set; } Number of location tokens <loc_0>..<loc_N-1>, each a quantised 0-1 coordinate.
int TextDim { get; set; } BART model width, shared by encoder and decoder (768 for base, 1024 for large).
int TextFeedForwardDim { get; set; } BART feed-forward width (3072 or 4096).
int VisionBaseDim { get; set; } DaViT stage-0 width.
int VisionBaseHeads { get; set; } DaViT stage-0 heads, also its channel-attention group count.
int VisionThirdStageDepth { get; set; } Depth of DaViT’s third stage.
int VocabSize { get; set; } Vocabulary size: BART’s 50265 BPE tokens plus the added task and location tokens.
int WindowSize { get; set; } Side of DaViT’s spatial attention windows.

Methods

Method Summary
void Validate() Throws if the configuration cannot build a Florence-2 model.