FederatedPromptTuning<T>

Models & Types · AiDotNet.FederatedLearning.Adapters

Federated Prompt Tuning — soft prompt aggregation for foundation model personalization.

Inheritance: FederatedLearningComponentBase<T> → FederatedPromptTuning<T>

Implements: IFederatedAdapterStrategy<T>

For Beginners

Instead of modifying the model itself, prompt tuning adds a few learned “instructions” in front of each input. These instructions tell the model how to adapt to the specific task. In federated prompt tuning, each device learns its own instructions and they’re combined at the server — only a few thousand parameters need to be shared, even for billion-parameter models.

How It Works

Prompt tuning (Lester et al., 2021) prepends learnable “soft prompt” tokens to the input. In federated settings, only these prompt embeddings (typically 10-100 tokens × embedding dim) are communicated, offering even higher compression than LoRA for very large models.

References: Lester et al. (2021), “The Power of Scale for Parameter-Efficient Prompt Tuning”. Zhao et al. (2024), “FedPSF-LLM: Dual Prompt Personalization for Federated Foundation Models”.

Constructors

Constructor Summary
FederatedPromptTuning(int modelDim, int numPromptTokens, int embeddingDim) Creates a new federated prompt tuning strategy.

Properties

Property Summary
double CompressionRatio { get; }
int AdapterParameterCount { get; }

Methods

Method Summary
Vector<T> AggregateAdapters(Dictionary<int, Vector<T>> clientAdapters, Dictionary<int, double> clientWeights)
Vector<T> ExtractAdapterParameters(Vector<T> fullModelParameters)
Vector<T> MergeAdapterParameters(Vector<T> fullModelParameters, Vector<T> aggregatedAdapters)