# Federated meta-learning

Federated meta-learning is a machine learning approach that combines federated learning across decentralized clients with meta-learning, so that a shared model or algorithm learned without exchanging raw data can be adapted to each client with only a few local gradient steps. It addresses a weakness of plain federated learning: when client data are non-IID (client data not independently and identically distributed), a single global model fits no client well, whereas a meta-learned initialization or update rule personalizes quickly. Per-FedAvg, for example, optimizes a shared initialization \( w \) through the objective \( \min_{w} F(w) := (1/n) \sum_{i} f_{i}(w - \alpha \nabla f_{i}(w)) \), where each client's loss is evaluated after one inner gradient step.<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup> FedMeta instead shares a parameterized algorithm (a meta-learner) rather than a model<sup>[2](https://arxiv.org/pdf/1802.07876)</sup>, and FedAvg itself can be interpreted as a meta-learning algorithm.<sup>[3](https://arxiv.org/pdf/1909.12488)</sup>

| Key fact | Value |
|---|---|
| What is learned globally | An initialization (Per-FedAvg, federated MAML), a parameterized update rule (FedMeta, Meta-SGD), or hyperparameters of fine-tuning (FedL2P); plain FL is the special case with inner stepsize zero<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup><sup> • </sup><sup>[4](https://arxiv.org/pdf/2310.02420)</sup> |
| Communication savings | FedMeta reduces required communication by 2.82–4.33× versus FedAvg<sup>[2](https://arxiv.org/pdf/1802.07876)</sup> |
| Accuracy gains | FedMeta reports 3.23%–14.84% higher accuracy than FedAvg on LEAF and production datasets<sup>[2](https://arxiv.org/pdf/1802.07876)</sup> |
| Computation overhead | A Hessian-vector product costs the same order as a gradient under automatic differentiation, while constructing the full Hessian would scale quadratically in \( d \); on Shakespeare, FedAvg's compute is about 5× less than MAML's<sup>[5](https://openreview.net/pdf?id=H5UUzau7r6)</sup><sup> • </sup><sup>[2](https://arxiv.org/pdf/1802.07876)</sup><sup> • </sup><sup>[24](https://iclr-blogposts.github.io/2024/blog/bench-hvp/)</sup> |
| Convergence rate | \( O(\varepsilon^{-3/2}) \) rounds to an \( \varepsilon \)-approximate stationary point for non-convex Per-FedAvg<sup>[6](https://arxiv.org/abs/2609.22833)</sup> |
| Adaptation advantage | The FedAvg-trained model overfits when fine-tuned on few samples, while the meta-model improves with additional fine-tuning steps without overfitting<sup>[7](https://ar5iv.labs.arxiv.org/html/2001.03229)</sup> |

## How it works

The method rests on the bi-level structure of Model-Agnostic Meta-Learning (MAML), which trains initial parameters so that one or a few gradient steps on a new task yield good performance.<sup>[8](https://proceedings.mlr.press/v70/finn17a/finn17a.pdf)</sup> For each task (in federation, each client), adapted parameters are computed as \( \theta'_{i} = \theta - \alpha \nabla_{\theta} L_{T_{i}}(f_{\theta}) \) with inner stepsize \( \alpha \), and the meta-parameters are then updated with meta stepsize \( \beta \) using the losses of the adapted models.<sup>[8](https://proceedings.mlr.press/v70/finn17a/finn17a.pdf)</sup> In the federated setting the tasks are clients' local datasets, and the outer loop becomes a server aggregation. What is learned globally depends on the variant: an initialization whose adaptation is left to the client (Per-FedAvg)<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup>, a parameterized algorithm (FedMeta with Meta-SGD)<sup>[2](https://arxiv.org/pdf/1802.07876)</sup>, or fine-tuning hyperparameters produced by a meta-network (FedL2P).<sup>[4](https://arxiv.org/pdf/2310.02420)</sup> A useful boundary case: federated learning is federated meta-learning with the inner stepsize set to zero, and the meta-learning local update uses a biased gradient estimate containing high-order information that plain FL's unbiased estimates lack.<sup>[9](https://arxiv.org/pdf/2108.06453v4.pdf)</sup>

Per-FedAvg established non-convex convergence with heterogeneity measured by Total Variation and 1-Wasserstein distances between users' data distributions<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup>; later work restates this as \( O(\varepsilon^{-3/2}) \)-round convergence under uniformly bounded gradients and bounded gradient dissimilarity, a bound matched by the policy-gradient extension Per-FedAvg-PG.<sup>[6](https://arxiv.org/abs/2609.22833)</sup> In the edge framework, an error term grows with task dissimilarity and with the number of local steps \( T_{0} \) when \( T_{0} \) is large, and vanishes at \( T_{0} = 1 \), letting a platform trade communication against computation.<sup>[7](https://ar5iv.labs.arxiv.org/html/2001.03229)</sup>

## How it is done

A round proceeds as follows. The server sends the current global model to a fraction \( r \cdot n \) of users chosen uniformly at random; each selected user performs \( \tau \geq 1 \) steps of stochastic gradient descent locally with respect to its meta-function \( F_{i} \), and the server aggregates.<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup> In FedMeta's workflow, sampled clients receive the algorithm parameters, train a task-specific model on a support set, evaluate on a disjoint query set, and upload the test losses; the server then performs the outer (algorithm) update, so only algorithm parameters and losses travel, never raw data.<sup>[2](https://arxiv.org/pdf/1802.07876)</sup> In an edge-oriented framework, each node applies one gradient step on its training split and evaluates on its test split, with \( T_{0} \) local update steps between aggregations.<sup>[7](https://ar5iv.labs.arxiv.org/html/2001.03229)</sup> Hyperparameters matter: with \( \alpha = 0.001 \) and \( \tau = 10 \), both Per-FedAvg variants beat FedAvg, but at \( \alpha = 0.01 \) the first-order variant drops sharply while the Hessian-approximating variant improves.<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup>

## Origin

The method combines two precursors. McMahan and colleagues reported Federated Averaging (FedAvg) in 2016 on arXiv<sup>[10](https://doi.org/10.48550/arxiv.1602.05629)</sup>, and Finn, Abbeel, and Levine reported MAML in 2017 on arXiv.<sup>[11](https://doi.org/10.48550/arxiv.1703.03400)</sup> Two 2018–2020 works brought them together. Chen and colleagues shared the FedMeta framework in 2018 on arXiv, integrating MAML into federated learning with a parameterized algorithm as the shared object.<sup>[12](https://doi.org/10.48550/arxiv.1802.07876)</sup> Fallah, Mokhtari, and Ozdaglar presented Per-FedAvg at NeurIPS 2020 with convergence theory for the MAML formulation of personalized FL<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup>; they note that the same formulation had been proposed independently in another work and studied numerically, their contribution being the theoretical analysis.<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup> Separately, a 2019 paper on arXiv established the FedAvg–MAML connection and the FedAvg–Reptile equivalence under equal client data sizes<sup>[3](https://arxiv.org/pdf/1909.12488)</sup>, so attribution of the canonical reference is split between the framework paper (FedMeta), the theory paper (Per-FedAvg), and the interpretation paper (Jiang et al.).

## Variants

**Per-FedAvg** comes in two implementations: Per-FedAvg (FO), which drops the Hessian term as in FO-MAML, and Per-FedAvg (HF), which approximates the Hessian-vector product by a difference of gradients, \( \nabla^{2}\phi(w)u \approx (\nabla\phi(u + \delta v) - \nabla\phi(u - \delta v))/\delta \).<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup> **FedMeta** integrates MAML, FOMAML, and Meta-SGD, the last converging faster than MAML in its experiments.<sup>[2](https://arxiv.org/pdf/1802.07876)</sup><sup> • </sup><sup>[13](https://doi.org/10.48550/arxiv.1707.09835)</sup> **pFedMe** replaces the MAML objective with Moreau envelopes, \( F_{i}(w) = \min_{\theta_{i}} f_{i}(\theta_{i}) + (\lambda/2)\|\theta_{i} - w\|^{2} \), decoupling personalized from global optimization in a bi-level problem, with a server parameter \( \beta \) that reduces to FedAvg averaging at \( \beta = 1 \).<sup>[14](https://proceedings.neurips.cc/paper/2020/file/f4f1f13c8289ac1b1ee0ff176b56fc60-Paper.pdf)</sup> **ARUBA** meta-learns a per-coordinate learning rate for MAML, Reptile, and FedAvg, giving tuning-free personalization.<sup>[15](https://proceedings.neurips.cc/paper/2019/file/f4aa0dd960521e045ae2f20621fb4ee9-Paper.pdf)</sup> Hypernetwork methods such as PeFLL and FedL2P learn an update rule or hyperparameter mapping rather than an initialization.<sup>[16](https://proceedings.iclr.cc/paper_files/paper/2024/file/8957cfb29ae633d6e8c3615be038d4d6-Paper-Conference.pdf)</sup><sup> • </sup><sup>[4](https://arxiv.org/pdf/2310.02420)</sup> Air-meta-pFL (Wen, Xing, and Simeone, 2024) frames MAML-based meta-personalization as federated pre-training and fine-tuning over a shared wireless channel, proving convergence bounds under smooth non-convex losses and a generalization-error bound, and showing that channel noise during pre-training can act as a regularizer while larger transmit power speeds convergence but may degrade generalization.<sup>[17](https://arxiv.org/pdf/2406.11569v6.pdf)</sup>

## Applications

Against the costs of the inner loop, FedMeta reports 2.82–4.33× lower communication and 3.23%–14.84% higher accuracy than FedAvg.<sup>[2](https://arxiv.org/pdf/1802.07876)</sup> On wireless benchmarks after fifty rounds with 20–40 devices and one local step, NUFM reached 68.04% ([Fashion-MNIST](https://www.edgepedia.org/fashion-mnist)), 58.80% (CIFAR-10), 23.95% (CIFAR-100), and 34.04% (ImageNet), versus Per-FedAvg at 62.75%, 58.22%, 21.49%, and 30.98%, and FedAvg at 61.04%, 54.31%, 10.13%, and 12.14%.<sup>[9](https://arxiv.org/pdf/2108.06453v4.pdf)</sup> Per-FedAvg (HF) outperforms FedAvg in all tested cases, with the gap growing under increased heterogeneity; on MNIST the gain is marginal, on CIFAR-10 more significant.<sup>[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)</sup> PeFLL achieves the best test accuracy in almost all benchmark cases, with the largest gains on previously unseen clients.<sup>[16](https://proceedings.iclr.cc/paper_files/paper/2024/file/8957cfb29ae633d6e8c3615be038d4d6-Paper-Conference.pdf)</sup>

## Limitations and alternatives

**Failure modes.** Meta-learning-based FL suffers from inner-loop fluctuation: for the same client across sampling loops, adaptation goals are unstable because of the non-convex loss and SGD randomness, hindering convergence.<sup>[18](https://arxiv.org/pdf/2306.16703)</sup> Bi-level optimization instability is a recognized challenge of MAML-style methods generally.<sup>[19](https://arxiv.org/pdf/2307.04722)</sup> [Performance](https://www.edgepedia.org/performance) deteriorates as dissimilarity among tasks increases, indicating a globally shared set of meta-parameters may not capture task heterogeneity.<sup>[19](https://arxiv.org/pdf/2307.04722)</sup> Greater client heterogeneity and larger inner learning rate \( \alpha \) increase the convergence error of the global meta-model.<sup>[5](https://openreview.net/pdf?id=H5UUzau7r6)</sup> Under non-IID data, FedAvg itself suffers client drift, where local models move away from the optimal global model, causing unstable and slow convergence.<sup>[20](https://arxiv.org/html/2210.13111v2)</sup> A practical asymmetry favors meta-learning at adaptation time: the FedAvg-trained model overfits when fine-tuned with few samples, whereas the meta-model improves with additional fine-tuning steps.<sup>[7](https://ar5iv.labs.arxiv.org/html/2001.03229)</sup>

**Alternatives.** FedProx adds proximal regularization penalizing updates that deviate from the global model, SCAFFOLD introduces [control variates](https://www.edgepedia.org/control-variates) to correct drift, and clustered FL groups similar clients and aggregates per cluster before global fusion.<sup>[21](https://www.mdpi.com/2073-431X/15/3/155)</sup><sup> • </sup><sup>[22](https://doi.org/10.48550/arxiv.1812.06127)</sup> Model-based personalization approaches, including meta-learning, assume a single global model and architecture, making them unsuitable under large client distribution differences.<sup>[23](https://ar5iv.labs.arxiv.org/html/2103.00710)</sup>

## References

1. [Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach (Per-FedAvg, NeurIPS 2020)](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)
2. [Federated Meta-Learning with Fast Convergence and Efficient Communication (FedMeta, arXiv:1802.07876)](https://arxiv.org/pdf/1802.07876)
3. [Improving Federated Learning via Model Aggregation (Jiang et al., arXiv:1909.12488, 2019)](https://arxiv.org/pdf/1909.12488)
4. [FedL2P: Federated Learning to Personalize (arXiv:2310.02420)](https://arxiv.org/pdf/2310.02420)
5. [First-order Personalized Federated Meta-Learning via Over-the-Air Computations (OpenReview)](https://openreview.net/pdf?id=H5UUzau7r6)
6. [Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients (Per-FedAvg-PG, arXiv:2609.22833)](https://arxiv.org/abs/2609.22833)
7. [Real-Time Edge Intelligence in the Making: A Collaborative Learning Framework via Federated Meta-Learning (FedML, arXiv:2001.03229)](https://ar5iv.labs.arxiv.org/html/2001.03229)
8. [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML, ICML 2017)](https://proceedings.mlr.press/v70/finn17a/finn17a.pdf)
9. [Efficient Federated Meta-Learning over Wireless Networks (NUFM/URAL, arXiv:2108.06453)](https://arxiv.org/pdf/2108.06453v4.pdf)
10. [McMahan, H. Brendan and colleagues (2016). Communication-Efficient Learning of Deep Networks from Decentralized Data. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1602.05629)
11. [Finn, Chelsea, Abbeel, Pieter, Levine, Sergey (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1703.03400)
12. [Chen, Fei and colleagues (2018). Federated Meta-Learning with Fast Convergence and Efficient Communication. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1802.07876)
13. [Li, Zhenguo and colleagues (2017). Meta-SGD: Learning to Learn Quickly for Few-Shot Learning. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1707.09835)
14. [Personalized Federated Learning with Moreau Envelopes (pFedMe, NeurIPS 2020)](https://proceedings.neurips.cc/paper/2020/file/f4f1f13c8289ac1b1ee0ff176b56fc60-Paper.pdf)
15. [Adaptive Gradient-Based Meta-Learning Methods (ARUBA, NeurIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/f4aa0dd960521e045ae2f20621fb4ee9-Paper.pdf)
16. [PeFLL: Personalized Federated Learning by Learning to Learn (ICLR 2024)](https://proceedings.iclr.cc/paper_files/paper/2024/file/8957cfb29ae633d6e8c3615be038d4d6-Paper-Conference.pdf)
17. [Pre-Training and Personalized Fine-Tuning via Over-the-Air Federated Meta-Learning: Convergence-Generalization Trade-Offs (Air-meta-pFL, arXiv:2406.11569)](https://arxiv.org/pdf/2406.11569v6.pdf)
18. [FedEC: An Elastic-Constrained Meta-Learner for Federated Learning (arXiv:2306.16703, 2023)](https://arxiv.org/pdf/2306.16703)
19. [A Comprehensive Technical Overview of Meta-learning (survey, arXiv:2307.04722)](https://arxiv.org/pdf/2307.04722)
20. [Federated Learning and Meta Learning: Approaches, Applications, and Directions (tutorial/survey, arXiv:2210.13111)](https://arxiv.org/html/2210.13111v2)
21. [Federated Learning: A Survey of Core Challenges, Current Methods, and Opportunities (MDPI Computers, 2025)](https://www.mdpi.com/2073-431X/15/3/155)
22. [Li, Tian and colleagues (2018). Federated Optimization in Heterogeneous Networks. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1812.06127)
23. [Towards Personalized Federated Learning (survey, arXiv:2103.00710)](https://ar5iv.labs.arxiv.org/html/2103.00710)
24. [Bench hvp (iclr-blogposts.github.io)](https://iclr-blogposts.github.io/2024/blog/bench-hvp/)

---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Machine learning methods › Ensemble, boosting, and transfer methods › Transfer learning and domain adaptation*

*Initially written Sep 29, 2026 · Reviewed: Sep 30, 2026 · Edited: Sep 30, 2026 · Last review: Sep 30, 2026*

*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*

License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license