[Paper] Federated Customization of Large Models: Approaches, Experiments, and Insights

Source: arXiv - 2601.00526v1

Overview

The paper investigates how to customize massive pretrained models (LLMs, vision‑transformers, etc.) in a federated learning (FL) setting, where data never leaves the device or organization that owns it. By adapting a range of fine‑tuning and prompting techniques to FL, the authors show that it is possible to reap the benefits of large‑model personalization without sacrificing privacy or incurring prohibitive communication costs.

Key Contributions

• Systematic survey of six major large‑model customization strategies (full fine‑tuning, efficient fine‑tuning, prompt engineering, prefix‑tuning, knowledge distillation, retrieval‑augmented generation) and how each maps onto federated learning constraints.
• First implementation of federated prefix‑tuning, extending a lightweight prompting method to the FL paradigm.
• Empirical benchmark comparing federated prefix‑tuning against three other federated customization approaches on standard NLP/vision tasks.
• Performance‑vs‑efficiency analysis demonstrating that federated prefix‑tuning attains accuracy close to a centralized baseline while using far fewer communication rounds and less client‑side compute.
• Robustness evaluation showing consistent behavior across heterogeneous client data distributions and varying network conditions.

Methodology

1. Problem framing – The authors define “federated customization” as the process of adapting a shared large model to the local data of many clients while keeping raw data private.
2. Technique adaptation – For each of the six customization methods, they outline the required modifications to the FL workflow (e.g., which parameters are sent, how gradients are aggregated, whether a server‑side prompt pool is needed).
3. Federated prefix‑tuning design
   • Each client maintains a small set of prefix vectors (learnable embeddings that prepend to every transformer layer).
   • The large backbone model stays frozen on the client; only the prefix vectors are updated locally.
   • After each local training epoch, clients upload the delta of their prefix vectors; the server aggregates them (FedAvg) and broadcasts the updated prefix back.
4. Experimental setup
   • Datasets: GLUE‑style text classification and a vision benchmark (e.g., CIFAR‑100) to illustrate cross‑modal applicability.
   • Baselines: Federated full fine‑tuning, federated efficient fine‑tuning (e.g., LoRA), and federated prompt engineering.
   • Metrics: Task accuracy, communication volume (MB per round), client‑side FLOPs, and robustness to non‑IID data splits.

Results & Findings

*Measured as additional FLOPs beyond a forward pass of the frozen backbone.

• Accuracy: Federated prefix‑tuning trails the centralized (non‑FL) baseline by only 0.2 %, outperforming the other FL methods.
• Efficiency: Communication overhead is reduced by ~90 % compared with full fine‑tuning, and client compute stays comparable to a simple forward‑only pass plus a tiny gradient update.
• Robustness: Across highly skewed (non‑IID) client partitions, performance degradation remains under 0.5 % for prefix‑tuning, whereas full fine‑tuning drops >2 %.

Practical Implications

• Edge AI & Mobile Apps – Developers can ship a massive pretrained model (e.g., a 7B LLM) to smartphones and let each device learn a personalized “prompt prefix” without ever uploading user text. This enables on‑device assistants that adapt to a user’s vocabulary while preserving privacy.
• Enterprise SaaS – Companies offering AI‑powered services can fine‑tune a shared model across many tenants using federated prefix‑tuning, achieving tenant‑specific behavior with minimal bandwidth and compute budgets.
• Regulated Industries – In healthcare or finance, where data residency is mandatory, federated prefix‑tuning provides a compliant path to leverage state‑of‑the‑art models without moving PHI or PII.
• Rapid Prototyping – Because only a few hundred kilobytes of prefix parameters need to be exchanged, developers can iterate on personalization cycles in minutes rather than hours, making A/B testing of model tweaks feasible at scale.

Limitations & Future Work

• Scope of tasks – Experiments focus on classification and simple generation; more complex multi‑turn dialogue or vision‑language tasks may expose hidden bottlenecks.
• Security considerations – While data stays local, the exchanged prefix vectors could still leak information; the paper does not explore differential privacy or secure aggregation for these parameters.
• Scalability to billions of parameters – The study uses models up to a few hundred million parameters; extending to truly massive LLMs (tens of billions) may require additional compression or hierarchical aggregation strategies.
• Future directions suggested include: integrating secure multi‑party computation for prefix updates, exploring adaptive prefix lengths per client, and evaluating the approach on heterogeneous hardware (IoT, AR glasses).

Authors

• Yuchuan Ye
• Ming Ding
• Youjia Chen
• Peng Cheng
• Dusit Niyato

Paper Information

• arXiv ID: 2601.00526v1
• Categories: cs.LG, cs.DC
• Published: January 2, 2026
• PDF: Download PDF