{ "id": "epmwrt484v", "slug": "federated-meta-learning", "title": "Federated meta-learning", "updated": "2026-09-30", "topic_path": [ { "id": "technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/topics/technology" }, { "id": "technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing" }, { "id": "technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai" }, { "id": "technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_computation" }, { "id": "technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ml_methods" }, { "id": "technology.computing.ai.ensembles_boosting", "label": "Ensemble, boosting, and transfer methods", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ensembles_boosting" }, { "id": "technology.computing.ai.ensembles_boosting.transfer_learning", "label": "Transfer learning and domain adaptation", "api_url": "https://www.edgepedia.org/api/v1/topics/technology.computing.ai.ensembles_boosting.transfer_learning" } ], "geo": [ { "id": "geo.nongeo.t2001.technology.computing.ai.ensembles_boosting", "label": "Non-geographic · 2001 to 2020: Ensemble, boosting, and transfer methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ensembles_boosting", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t2001", "label": "Non-geographic · 2001 to 2020", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001" }, { "id": "geo.nongeo.t2001.technology", "label": "Technology and the built world", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology" }, { "id": "geo.nongeo.t2001.technology.computing", "label": "Computing and digital systems", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing" }, { "id": "geo.nongeo.t2001.technology.computing.ai", "label": "Artificial intelligence and data", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_computation", "label": "Machine learning and neural computation", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_computation" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ml_methods", "label": "Machine learning methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ml_methods" }, { "id": "geo.nongeo.t2001.technology.computing.ai.ensembles_boosting", "label": "Ensemble, boosting, and transfer methods", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t2001.technology.computing.ai.ensembles_boosting" } ] } ], "excerpt": "Federated meta-learning is a machine learning approach combining federated learning with meta-learning, so a shared model learned without exchanging raw data adapts to each client with a few local gradient steps.", "snippet": "Federated meta-learning is a machine learning approach combining federated learning with meta-learning, so a shared model learned without exchanging raw data adapts to each client with a few local gradient steps.", "node": "technology.computing.ai.ensembles_boosting.transfer_learning", "markdown": "# Federated meta-learning\n\nFederated meta-learning is a machine learning approach that combines federated learning across decentralized clients with meta-learning, so that a shared model or algorithm learned without exchanging raw data can be adapted to each client with only a few local gradient steps. It addresses a weakness of plain federated learning: when client data are non-IID (client data not independently and identically distributed), a single global model fits no client well, whereas a meta-learned initialization or update rule personalizes quickly. Per-FedAvg, for example, optimizes a shared initialization \\( w \\) through the objective \\( \\min_{w} F(w) := (1/n) \\sum_{i} f_{i}(w - \\alpha \\nabla f_{i}(w)) \\), where each client's loss is evaluated after one inner gradient step.[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf) FedMeta instead shares a parameterized algorithm (a meta-learner) rather than a model[2](https://arxiv.org/pdf/1802.07876), and FedAvg itself can be interpreted as a meta-learning algorithm.[3](https://arxiv.org/pdf/1909.12488)\n\n| Key fact | Value |\n|---|---|\n| What is learned globally | An initialization (Per-FedAvg, federated MAML), a parameterized update rule (FedMeta, Meta-SGD), or hyperparameters of fine-tuning (FedL2P); plain FL is the special case with inner stepsize zero[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf) • [4](https://arxiv.org/pdf/2310.02420) |\n| Communication savings | FedMeta reduces required communication by 2.82–4.33× versus FedAvg[2](https://arxiv.org/pdf/1802.07876) |\n| Accuracy gains | FedMeta reports 3.23%–14.84% higher accuracy than FedAvg on LEAF and production datasets[2](https://arxiv.org/pdf/1802.07876) |\n| Computation overhead | A Hessian-vector product costs the same order as a gradient under automatic differentiation, while constructing the full Hessian would scale quadratically in \\( d \\); on Shakespeare, FedAvg's compute is about 5× less than MAML's[5](https://openreview.net/pdf?id=H5UUzau7r6) • [2](https://arxiv.org/pdf/1802.07876) • [24](https://iclr-blogposts.github.io/2024/blog/bench-hvp/) |\n| Convergence rate | \\( O(\\varepsilon^{-3/2}) \\) rounds to an \\( \\varepsilon \\)-approximate stationary point for non-convex Per-FedAvg[6](https://arxiv.org/abs/2609.22833) |\n| Adaptation advantage | The FedAvg-trained model overfits when fine-tuned on few samples, while the meta-model improves with additional fine-tuning steps without overfitting[7](https://ar5iv.labs.arxiv.org/html/2001.03229) |\n\n## How it works\n\nThe method rests on the bi-level structure of Model-Agnostic Meta-Learning (MAML), which trains initial parameters so that one or a few gradient steps on a new task yield good performance.[8](https://proceedings.mlr.press/v70/finn17a/finn17a.pdf) For each task (in federation, each client), adapted parameters are computed as \\( \\theta'_{i} = \\theta - \\alpha \\nabla_{\\theta} L_{T_{i}}(f_{\\theta}) \\) with inner stepsize \\( \\alpha \\), and the meta-parameters are then updated with meta stepsize \\( \\beta \\) using the losses of the adapted models.[8](https://proceedings.mlr.press/v70/finn17a/finn17a.pdf) In the federated setting the tasks are clients' local datasets, and the outer loop becomes a server aggregation. What is learned globally depends on the variant: an initialization whose adaptation is left to the client (Per-FedAvg)[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf), a parameterized algorithm (FedMeta with Meta-SGD)[2](https://arxiv.org/pdf/1802.07876), or fine-tuning hyperparameters produced by a meta-network (FedL2P).[4](https://arxiv.org/pdf/2310.02420) A useful boundary case: federated learning is federated meta-learning with the inner stepsize set to zero, and the meta-learning local update uses a biased gradient estimate containing high-order information that plain FL's unbiased estimates lack.[9](https://arxiv.org/pdf/2108.06453v4.pdf)\n\nPer-FedAvg established non-convex convergence with heterogeneity measured by Total Variation and 1-Wasserstein distances between users' data distributions[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf); later work restates this as \\( O(\\varepsilon^{-3/2}) \\)-round convergence under uniformly bounded gradients and bounded gradient dissimilarity, a bound matched by the policy-gradient extension Per-FedAvg-PG.[6](https://arxiv.org/abs/2609.22833) In the edge framework, an error term grows with task dissimilarity and with the number of local steps \\( T_{0} \\) when \\( T_{0} \\) is large, and vanishes at \\( T_{0} = 1 \\), letting a platform trade communication against computation.[7](https://ar5iv.labs.arxiv.org/html/2001.03229)\n\n## How it is done\n\nA round proceeds as follows. The server sends the current global model to a fraction \\( r \\cdot n \\) of users chosen uniformly at random; each selected user performs \\( \\tau \\geq 1 \\) steps of stochastic gradient descent locally with respect to its meta-function \\( F_{i} \\), and the server aggregates.[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf) In FedMeta's workflow, sampled clients receive the algorithm parameters, train a task-specific model on a support set, evaluate on a disjoint query set, and upload the test losses; the server then performs the outer (algorithm) update, so only algorithm parameters and losses travel, never raw data.[2](https://arxiv.org/pdf/1802.07876) In an edge-oriented framework, each node applies one gradient step on its training split and evaluates on its test split, with \\( T_{0} \\) local update steps between aggregations.[7](https://ar5iv.labs.arxiv.org/html/2001.03229) Hyperparameters matter: with \\( \\alpha = 0.001 \\) and \\( \\tau = 10 \\), both Per-FedAvg variants beat FedAvg, but at \\( \\alpha = 0.01 \\) the first-order variant drops sharply while the Hessian-approximating variant improves.[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)\n\n## Origin\n\nThe method combines two precursors. McMahan and colleagues reported Federated Averaging (FedAvg) in 2016 on arXiv[10](https://doi.org/10.48550/arxiv.1602.05629), and Finn, Abbeel, and Levine reported MAML in 2017 on arXiv.[11](https://doi.org/10.48550/arxiv.1703.03400) Two 2018–2020 works brought them together. Chen and colleagues shared the FedMeta framework in 2018 on arXiv, integrating MAML into federated learning with a parameterized algorithm as the shared object.[12](https://doi.org/10.48550/arxiv.1802.07876) Fallah, Mokhtari, and Ozdaglar presented Per-FedAvg at NeurIPS 2020 with convergence theory for the MAML formulation of personalized FL[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf); they note that the same formulation had been proposed independently in another work and studied numerically, their contribution being the theoretical analysis.[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf) Separately, a 2019 paper on arXiv established the FedAvg–MAML connection and the FedAvg–Reptile equivalence under equal client data sizes[3](https://arxiv.org/pdf/1909.12488), so attribution of the canonical reference is split between the framework paper (FedMeta), the theory paper (Per-FedAvg), and the interpretation paper (Jiang et al.).\n\n## Variants\n\n**Per-FedAvg** comes in two implementations: Per-FedAvg (FO), which drops the Hessian term as in FO-MAML, and Per-FedAvg (HF), which approximates the Hessian-vector product by a difference of gradients, \\( \\nabla^{2}\\phi(w)u \\approx (\\nabla\\phi(u + \\delta v) - \\nabla\\phi(u - \\delta v))/\\delta \\).[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf) **FedMeta** integrates MAML, FOMAML, and Meta-SGD, the last converging faster than MAML in its experiments.[2](https://arxiv.org/pdf/1802.07876) • [13](https://doi.org/10.48550/arxiv.1707.09835) **pFedMe** replaces the MAML objective with Moreau envelopes, \\( F_{i}(w) = \\min_{\\theta_{i}} f_{i}(\\theta_{i}) + (\\lambda/2)\\|\\theta_{i} - w\\|^{2} \\), decoupling personalized from global optimization in a bi-level problem, with a server parameter \\( \\beta \\) that reduces to FedAvg averaging at \\( \\beta = 1 \\).[14](https://proceedings.neurips.cc/paper/2020/file/f4f1f13c8289ac1b1ee0ff176b56fc60-Paper.pdf) **ARUBA** meta-learns a per-coordinate learning rate for MAML, Reptile, and FedAvg, giving tuning-free personalization.[15](https://proceedings.neurips.cc/paper/2019/file/f4aa0dd960521e045ae2f20621fb4ee9-Paper.pdf) Hypernetwork methods such as PeFLL and FedL2P learn an update rule or hyperparameter mapping rather than an initialization.[16](https://proceedings.iclr.cc/paper_files/paper/2024/file/8957cfb29ae633d6e8c3615be038d4d6-Paper-Conference.pdf) • [4](https://arxiv.org/pdf/2310.02420) Air-meta-pFL (Wen, Xing, and Simeone, 2024) frames MAML-based meta-personalization as federated pre-training and fine-tuning over a shared wireless channel, proving convergence bounds under smooth non-convex losses and a generalization-error bound, and showing that channel noise during pre-training can act as a regularizer while larger transmit power speeds convergence but may degrade generalization.[17](https://arxiv.org/pdf/2406.11569v6.pdf)\n\n## Applications\n\nAgainst the costs of the inner loop, FedMeta reports 2.82–4.33× lower communication and 3.23%–14.84% higher accuracy than FedAvg.[2](https://arxiv.org/pdf/1802.07876) On wireless benchmarks after fifty rounds with 20–40 devices and one local step, NUFM reached 68.04% ([Fashion-MNIST](https://www.edgepedia.org/fashion-mnist)), 58.80% (CIFAR-10), 23.95% (CIFAR-100), and 34.04% (ImageNet), versus Per-FedAvg at 62.75%, 58.22%, 21.49%, and 30.98%, and FedAvg at 61.04%, 54.31%, 10.13%, and 12.14%.[9](https://arxiv.org/pdf/2108.06453v4.pdf) Per-FedAvg (HF) outperforms FedAvg in all tested cases, with the gap growing under increased heterogeneity; on MNIST the gain is marginal, on CIFAR-10 more significant.[1](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf) PeFLL achieves the best test accuracy in almost all benchmark cases, with the largest gains on previously unseen clients.[16](https://proceedings.iclr.cc/paper_files/paper/2024/file/8957cfb29ae633d6e8c3615be038d4d6-Paper-Conference.pdf)\n\n## Limitations and alternatives\n\n**Failure modes.** Meta-learning-based FL suffers from inner-loop fluctuation: for the same client across sampling loops, adaptation goals are unstable because of the non-convex loss and SGD randomness, hindering convergence.[18](https://arxiv.org/pdf/2306.16703) Bi-level optimization instability is a recognized challenge of MAML-style methods generally.[19](https://arxiv.org/pdf/2307.04722) [Performance](https://www.edgepedia.org/performance) deteriorates as dissimilarity among tasks increases, indicating a globally shared set of meta-parameters may not capture task heterogeneity.[19](https://arxiv.org/pdf/2307.04722) Greater client heterogeneity and larger inner learning rate \\( \\alpha \\) increase the convergence error of the global meta-model.[5](https://openreview.net/pdf?id=H5UUzau7r6) Under non-IID data, FedAvg itself suffers client drift, where local models move away from the optimal global model, causing unstable and slow convergence.[20](https://arxiv.org/html/2210.13111v2) A practical asymmetry favors meta-learning at adaptation time: the FedAvg-trained model overfits when fine-tuned with few samples, whereas the meta-model improves with additional fine-tuning steps.[7](https://ar5iv.labs.arxiv.org/html/2001.03229)\n\n**Alternatives.** FedProx adds proximal regularization penalizing updates that deviate from the global model, SCAFFOLD introduces [control variates](https://www.edgepedia.org/control-variates) to correct drift, and clustered FL groups similar clients and aggregates per cluster before global fusion.[21](https://www.mdpi.com/2073-431X/15/3/155) • [22](https://doi.org/10.48550/arxiv.1812.06127) Model-based personalization approaches, including meta-learning, assume a single global model and architecture, making them unsuitable under large client distribution differences.[23](https://ar5iv.labs.arxiv.org/html/2103.00710)\n\n## References\n\n1. [Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach (Per-FedAvg, NeurIPS 2020)](https://proceedings.neurips.cc/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf)\n2. [Federated Meta-Learning with Fast Convergence and Efficient Communication (FedMeta, arXiv:1802.07876)](https://arxiv.org/pdf/1802.07876)\n3. [Improving Federated Learning via Model Aggregation (Jiang et al., arXiv:1909.12488, 2019)](https://arxiv.org/pdf/1909.12488)\n4. [FedL2P: Federated Learning to Personalize (arXiv:2310.02420)](https://arxiv.org/pdf/2310.02420)\n5. [First-order Personalized Federated Meta-Learning via Over-the-Air Computations (OpenReview)](https://openreview.net/pdf?id=H5UUzau7r6)\n6. [Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients (Per-FedAvg-PG, arXiv:2609.22833)](https://arxiv.org/abs/2609.22833)\n7. [Real-Time Edge Intelligence in the Making: A Collaborative Learning Framework via Federated Meta-Learning (FedML, arXiv:2001.03229)](https://ar5iv.labs.arxiv.org/html/2001.03229)\n8. [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML, ICML 2017)](https://proceedings.mlr.press/v70/finn17a/finn17a.pdf)\n9. [Efficient Federated Meta-Learning over Wireless Networks (NUFM/URAL, arXiv:2108.06453)](https://arxiv.org/pdf/2108.06453v4.pdf)\n10. [McMahan, H. Brendan and colleagues (2016). Communication-Efficient Learning of Deep Networks from Decentralized Data. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1602.05629)\n11. [Finn, Chelsea, Abbeel, Pieter, Levine, Sergey (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1703.03400)\n12. [Chen, Fei and colleagues (2018). Federated Meta-Learning with Fast Convergence and Efficient Communication. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1802.07876)\n13. [Li, Zhenguo and colleagues (2017). Meta-SGD: Learning to Learn Quickly for Few-Shot Learning. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1707.09835)\n14. [Personalized Federated Learning with Moreau Envelopes (pFedMe, NeurIPS 2020)](https://proceedings.neurips.cc/paper/2020/file/f4f1f13c8289ac1b1ee0ff176b56fc60-Paper.pdf)\n15. [Adaptive Gradient-Based Meta-Learning Methods (ARUBA, NeurIPS 2019)](https://proceedings.neurips.cc/paper/2019/file/f4aa0dd960521e045ae2f20621fb4ee9-Paper.pdf)\n16. [PeFLL: Personalized Federated Learning by Learning to Learn (ICLR 2024)](https://proceedings.iclr.cc/paper_files/paper/2024/file/8957cfb29ae633d6e8c3615be038d4d6-Paper-Conference.pdf)\n17. [Pre-Training and Personalized Fine-Tuning via Over-the-Air Federated Meta-Learning: Convergence-Generalization Trade-Offs (Air-meta-pFL, arXiv:2406.11569)](https://arxiv.org/pdf/2406.11569v6.pdf)\n18. [FedEC: An Elastic-Constrained Meta-Learner for Federated Learning (arXiv:2306.16703, 2023)](https://arxiv.org/pdf/2306.16703)\n19. [A Comprehensive Technical Overview of Meta-learning (survey, arXiv:2307.04722)](https://arxiv.org/pdf/2307.04722)\n20. [Federated Learning and Meta Learning: Approaches, Applications, and Directions (tutorial/survey, arXiv:2210.13111)](https://arxiv.org/html/2210.13111v2)\n21. [Federated Learning: A Survey of Core Challenges, Current Methods, and Opportunities (MDPI Computers, 2025)](https://www.mdpi.com/2073-431X/15/3/155)\n22. [Li, Tian and colleagues (2018). Federated Optimization in Heterogeneous Networks. arXiv (Cornell University).](https://doi.org/10.48550/arxiv.1812.06127)\n23. [Towards Personalized Federated Learning (survey, arXiv:2103.00710)](https://ar5iv.labs.arxiv.org/html/2103.00710)\n24. [Bench hvp (iclr-blogposts.github.io)](https://iclr-blogposts.github.io/2024/blog/bench-hvp/)\n\n---\n*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Machine learning methods › Ensemble, boosting, and transfer methods › Transfer learning and domain adaptation*\n\n*Initially written Sep 29, 2026 · Reviewed: Sep 30, 2026 · Edited: Sep 30, 2026 · Last review: Sep 30, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [], "url": "https://www.edgepedia.org/federated-meta-learning", "markdown_url": "https://www.edgepedia.org/federated-meta-learning.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Federated meta-learning\", Edgepedia (EdgeChat), https://www.edgepedia.org/federated-meta-learning. Edgepedia Community License 1.0.", "credit_md": "\"[Federated meta-learning](https://www.edgepedia.org/federated-meta-learning)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/federated-meta-learning](https://www.edgepedia.org/federated-meta-learning). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Federated meta-learning\", Edgepedia (EdgeChat), https://www.edgepedia.org/federated-meta-learning. Edgepedia Community License 1.0.", "speakable": "Federated meta-learning is a machine learning approach combining federated learning with meta-learning, so a shared model learned without exchanging raw data adapts to each client with a few local gradient steps." }