MusicGenOptions

Options & Configuration · AiDotNet.Audio.MusicGen

Options for MusicGen (Copet et al. 2023, “Simple and Controllable Music Generation”): a single-stage transformer language model over the four codebooks of a 32 kHz EnCodec, interleaved by the “delay” pattern and conditioned on T5 text embeddings through cross-attention.

Inheritance: ModelOptions → MusicGenOptions

For Beginners

MusicGen writes music the way a language model writes text, one short frame of compressed audio at a time, guided by a description such as “an upbeat 80s synth-pop track”.

How It Works

From the paper (§2–3): EnCodec at 32 kHz with stride 640 (50 Hz), four RVQ codebooks of 2048; the delay pattern; pre-norm layers of causal self-attention, cross-attention and a D → 4D → D ReLU block; a frozen T5 text encoder; AdamW (β = 0.9, 0.95), decoupled weight decay 0.1, gradient clipping 1.0, a cosine schedule with 4000 warmup steps, 1M steps, D-Adaptation for the 300M model, an EMA of decay 0.99; condition merging 0.25, description dropout 0.5 and word dropout 0.3; condition dropout 0.2 and classifier-free guidance 3.0; top-k 250 at temperature 1.0.

What the paper leaves open follows audiocraft (config/model/lm/musicgen_lm.yaml, its model scales, conditioner/text2music.yaml, solver/musicgen): the widths and depths of the three sizes, t5-base as the text encoder, bias-free linear layers, the sinusoidal step embedding, the learning rate 1e-4 of the AdamW-trained sizes, the EMA taken every 10 updates, the dropout of other metadata fields 0.5 and 30-second generation.

Constructors

Constructor Summary
MusicGenOptions() Creates the paper’s 1.5B (“medium”) configuration.
MusicGenOptions(MusicGenOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
EnCodecOptions Codec { get; set; } Gets or sets the audio tokenizer (EnCodec 32 kHz, §3.1).
MusicGenModelSize ModelSize { get; set; } Gets or sets the size preset (Medium: 1.5B parameters).
MusicGenStereoPattern StereoPattern { get; set; } Gets or sets the stereo codebook pattern (partial delay: the better rated, Table 3).
bool FreezeTextEncoder { get; set; } Gets or sets whether the T5 encoder stays fixed during training (true: “pretrained”, §2.3, and the reference’s finetune: false).
bool MelodyConditioning { get; set; } Gets or sets whether the model is conditioned on a melody as well as text (§2.3, the melody model): the chromagram of the conditioning audio, quantized to its dominant pitch class, and the text are given as a prefix of the sequence instead…
bool Stereo { get; set; } Gets or sets whether the model generates stereo (§4.3): EnCodec codes each channel separately, giving twice the codebooks, interleaved left, right per level (audiocraft InterleaveStereoCompressionModel).
bool UseSampling { get; set; } Gets or sets whether generation samples (true) or takes the most likely code.
bool? UseDAdaptation { get; set; } Gets or sets whether training uses D-Adaptation; null uses it for Small only (§3.1).
double Beta1 { get; set; } Gets or sets AdamW’s β₁ (0.9).
double Beta2 { get; set; } Gets or sets AdamW’s β₂ (0.95).
double ConditionDropout { get; set; } Gets or sets the probability of training a step without the condition (0.2, §3.1).
double DropDescriptionProbability { get; set; } Gets or sets the probability of dropping the description when metadata is merged (0.5, §3.1).
double DropoutRate { get; set; } Gets or sets the dropout inside the transformer (0, the reference’s).
double DurationSeconds { get; set; } Gets or sets the default generation length in seconds (30, the reference’s).
double EmaDecay { get; set; } Gets or sets the EMA decay (0.99, §3.1).
double GradientClipNorm { get; set; } Gets or sets the gradient-norm clip (1.0).
double GuidanceScale { get; set; } Gets or sets the classifier-free guidance scale (3.0, §3.1).
double KeepMetadataFieldProbability { get; set; } Gets or sets the probability of keeping each metadata field in a merge (0.5, the reference’s drop_other_p).
double MelodyDropout { get; set; } Gets or sets the probability of training a step without the melody (0.5, audiocraft’s attribute_dropout.wav.self_wav; the paper does not say).
double MergeMetadataProbability { get; set; } Gets or sets the probability of merging the metadata into the description (0.25, §3.1).
double Temperature { get; set; } Gets or sets the sampling temperature (1.0, §3.1).
double TopP { get; set; } Gets or sets nucleus sampling’s p; 0 uses top-k (0).
double WeightDecay { get; set; } Gets or sets the decoupled weight decay (0.1).
double WordDropout { get; set; } Gets or sets the probability of dropping each word of the description (0.3, §3.1).
double? LearningRate { get; set; } Gets or sets the learning rate; null takes the size’s (1 for the D-Adaptation-trained Small, the D-Adaptation multiplier; 1e-4 otherwise).
int ChromaHop { get; set; } Gets or sets the chromagram hop (2¹² = 4096, §3.1).
int ChromaWindow { get; set; } Gets or sets the chromagram window and FFT size (2¹⁴ = 16384, §3.1).
int EmaEveryUpdates { get; set; } Gets or sets how many optimizer updates pass between EMA updates (10, the reference’s).
int FeedForwardScale { get; set; } Gets or sets the feed-forward width as a multiple of D (4, §2.4).
int MaxTextLength { get; set; } Gets or sets the most text tokens read (512, T5’s).
int ModelDim { get; set; } Gets or sets the model width D (the preset’s: 1024, 1536, 2048); 0 also takes the preset’s.
int NumHeads { get; set; } Gets or sets the attention heads (the preset’s: 16, 24, 32); 0 also takes the preset’s.
int NumLayers { get; set; } Gets or sets the transformer layers L (the preset’s: 24, 48, 48); 0 also takes the preset’s.
int TextFeedForwardDim { get; set; } Gets or sets the T5 feed-forward width (3072, t5-base).
int TextHeads { get; set; } Gets or sets the T5 attention heads (12, t5-base).
int TextKeyValueDim { get; set; } Gets or sets the T5 key/value width per head (64, t5-base).
int TextLayers { get; set; } Gets or sets the T5 encoder layers (12, t5-base).
int TextModelDim { get; set; } Gets or sets the T5 encoder width (768, t5-base).
int TextVocabularySize { get; set; } Gets or sets the T5 embedding rows (32128, t5-base; its tokenizer uses the first 32100).
int TopK { get; set; } Gets or sets top-k sampling’s k (250, §3.1).
int TrainingSteps { get; set; } Gets or sets the training steps the cosine schedule spans (1,000,000).
int WarmupSteps { get; set; } Gets or sets the learning-rate warmup steps (4000).
int[] Delays { get; set; } Gets or sets each modelled codebook’s delay; null takes the pattern’s ([0, 1, 2, 3], Eq.

Methods

Method Summary
EnCodecOptions EnCodec32kHz() The paper’s 32 kHz EnCodec (§3.1; audiocraft encodec_large_nq4_s640): stride 640 (ratios 8, 5, 4, 4), 64 initial filters doubling over the encoder, non-causal with constant padding, four quantizers of 2048 codes.
ValueTuple<int, int, int> PresetDimensions(MusicGenModelSize size) The width, heads and depth of a size preset (audiocraft’s model scales).
int[] ResolveDelays() The delays in effect: Delays, or the pattern’s.