MusicGenOptions
Options & Configuration · AiDotNet.Audio.MusicGen
Options for MusicGen (Copet et al. 2023, “Simple and Controllable Music Generation”): a single-stage transformer language model over the four codebooks of a 32 kHz EnCodec, interleaved by the “delay” pattern and conditioned on T5 text embeddings through cross-attention.
Inheritance: ModelOptions → MusicGenOptions
For Beginners
MusicGen writes music the way a language model writes text, one short frame of compressed audio at a time, guided by a description such as “an upbeat 80s synth-pop track”.
How It Works
From the paper (§2–3): EnCodec at 32 kHz with stride 640 (50 Hz), four RVQ codebooks of 2048; the delay pattern; pre-norm layers of causal self-attention, cross-attention and a D → 4D → D ReLU block; a frozen T5 text encoder; AdamW (β = 0.9, 0.95), decoupled weight decay 0.1, gradient clipping 1.0, a cosine schedule with 4000 warmup steps, 1M steps, D-Adaptation for the 300M model, an EMA of decay 0.99; condition merging 0.25, description dropout 0.5 and word dropout 0.3; condition dropout 0.2 and classifier-free guidance 3.0; top-k 250 at temperature 1.0.
What the paper leaves open follows audiocraft (config/model/lm/musicgen_lm.yaml, its model scales,
conditioner/text2music.yaml, solver/musicgen): the widths and depths of the three sizes, t5-base as
the text encoder, bias-free linear layers, the sinusoidal step embedding, the learning rate 1e-4 of the
AdamW-trained sizes, the EMA taken every 10 updates, the dropout of other metadata fields 0.5 and 30-second
generation.
Constructors
| Constructor | Summary |
|---|---|
MusicGenOptions() |
Creates the paper’s 1.5B (“medium”) configuration. |
MusicGenOptions(MusicGenOptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
EnCodecOptions Codec { get; set; } |
Gets or sets the audio tokenizer (EnCodec 32 kHz, §3.1). |
MusicGenModelSize ModelSize { get; set; } |
Gets or sets the size preset (Medium: 1.5B parameters). |
MusicGenStereoPattern StereoPattern { get; set; } |
Gets or sets the stereo codebook pattern (partial delay: the better rated, Table 3). |
bool FreezeTextEncoder { get; set; } |
Gets or sets whether the T5 encoder stays fixed during training (true: “pretrained”, §2.3, and the reference’s finetune: false). |
bool MelodyConditioning { get; set; } |
Gets or sets whether the model is conditioned on a melody as well as text (§2.3, the melody model): the chromagram of the conditioning audio, quantized to its dominant pitch class, and the text are given as a prefix of the sequence instead… |
bool Stereo { get; set; } |
Gets or sets whether the model generates stereo (§4.3): EnCodec codes each channel separately, giving twice the codebooks, interleaved left, right per level (audiocraft InterleaveStereoCompressionModel). |
bool UseSampling { get; set; } |
Gets or sets whether generation samples (true) or takes the most likely code. |
bool? UseDAdaptation { get; set; } |
Gets or sets whether training uses D-Adaptation; null uses it for Small only (§3.1). |
double Beta1 { get; set; } |
Gets or sets AdamW’s β₁ (0.9). |
double Beta2 { get; set; } |
Gets or sets AdamW’s β₂ (0.95). |
double ConditionDropout { get; set; } |
Gets or sets the probability of training a step without the condition (0.2, §3.1). |
double DropDescriptionProbability { get; set; } |
Gets or sets the probability of dropping the description when metadata is merged (0.5, §3.1). |
double DropoutRate { get; set; } |
Gets or sets the dropout inside the transformer (0, the reference’s). |
double DurationSeconds { get; set; } |
Gets or sets the default generation length in seconds (30, the reference’s). |
double EmaDecay { get; set; } |
Gets or sets the EMA decay (0.99, §3.1). |
double GradientClipNorm { get; set; } |
Gets or sets the gradient-norm clip (1.0). |
double GuidanceScale { get; set; } |
Gets or sets the classifier-free guidance scale (3.0, §3.1). |
double KeepMetadataFieldProbability { get; set; } |
Gets or sets the probability of keeping each metadata field in a merge (0.5, the reference’s drop_other_p). |
double MelodyDropout { get; set; } |
Gets or sets the probability of training a step without the melody (0.5, audiocraft’s attribute_dropout.wav.self_wav; the paper does not say). |
double MergeMetadataProbability { get; set; } |
Gets or sets the probability of merging the metadata into the description (0.25, §3.1). |
double Temperature { get; set; } |
Gets or sets the sampling temperature (1.0, §3.1). |
double TopP { get; set; } |
Gets or sets nucleus sampling’s p; 0 uses top-k (0). |
double WeightDecay { get; set; } |
Gets or sets the decoupled weight decay (0.1). |
double WordDropout { get; set; } |
Gets or sets the probability of dropping each word of the description (0.3, §3.1). |
double? LearningRate { get; set; } |
Gets or sets the learning rate; null takes the size’s (1 for the D-Adaptation-trained Small, the D-Adaptation multiplier; 1e-4 otherwise). |
int ChromaHop { get; set; } |
Gets or sets the chromagram hop (2¹² = 4096, §3.1). |
int ChromaWindow { get; set; } |
Gets or sets the chromagram window and FFT size (2¹⁴ = 16384, §3.1). |
int EmaEveryUpdates { get; set; } |
Gets or sets how many optimizer updates pass between EMA updates (10, the reference’s). |
int FeedForwardScale { get; set; } |
Gets or sets the feed-forward width as a multiple of D (4, §2.4). |
int MaxTextLength { get; set; } |
Gets or sets the most text tokens read (512, T5’s). |
int ModelDim { get; set; } |
Gets or sets the model width D (the preset’s: 1024, 1536, 2048); 0 also takes the preset’s. |
int NumHeads { get; set; } |
Gets or sets the attention heads (the preset’s: 16, 24, 32); 0 also takes the preset’s. |
int NumLayers { get; set; } |
Gets or sets the transformer layers L (the preset’s: 24, 48, 48); 0 also takes the preset’s. |
int TextFeedForwardDim { get; set; } |
Gets or sets the T5 feed-forward width (3072, t5-base). |
int TextHeads { get; set; } |
Gets or sets the T5 attention heads (12, t5-base). |
int TextKeyValueDim { get; set; } |
Gets or sets the T5 key/value width per head (64, t5-base). |
int TextLayers { get; set; } |
Gets or sets the T5 encoder layers (12, t5-base). |
int TextModelDim { get; set; } |
Gets or sets the T5 encoder width (768, t5-base). |
int TextVocabularySize { get; set; } |
Gets or sets the T5 embedding rows (32128, t5-base; its tokenizer uses the first 32100). |
int TopK { get; set; } |
Gets or sets top-k sampling’s k (250, §3.1). |
int TrainingSteps { get; set; } |
Gets or sets the training steps the cosine schedule spans (1,000,000). |
int WarmupSteps { get; set; } |
Gets or sets the learning-rate warmup steps (4000). |
int[] Delays { get; set; } |
Gets or sets each modelled codebook’s delay; null takes the pattern’s ([0, 1, 2, 3], Eq. |
Methods
| Method | Summary |
|---|---|
EnCodecOptions EnCodec32kHz() |
The paper’s 32 kHz EnCodec (§3.1; audiocraft encodec_large_nq4_s640): stride 640 (ratios 8, 5, 4, 4), 64 initial filters doubling over the encoder, non-causal with constant padding, four quantizers of 2048 codes. |
ValueTuple<int, int, int> PresetDimensions(MusicGenModelSize size) |
The width, heads and depth of a size preset (audiocraft’s model scales). |
int[] ResolveDelays() |
The delays in effect: Delays, or the pattern’s. |