# Fine-tuning (deep learning)

**Fine-tuning** is the process of continuing the training of a pretrained machine learning model on a task- or domain-specific dataset to optimize it for a specific task<sup>[1](https://developers.openai.com/cookbook/examples/fine_tuning_direct_preference_optimization_guide)</sup>. In LLM adaptation, two common forms of fine-tuning are: continual pretraining (also called domain-adaptive pretraining), which extends the original pretraining objective on new domain text, and instruction fine-tuning (supervised fine-tuning, or SFT), which trains the model to follow instructions and produce specific outputs<sup>[2](https://llmbook.icsgen-ai.org/part-4-training-adaptation/module-16-fine-tuning-fundamentals/section-16.1.html)</sup>.

| Key fact | Detail |
|---|---|
| What updates | Full fine-tuning updates every parameter; parameter-efficient methods (PEFT) update typically 1–2% of parameters<sup>[3](https://link.springer.com/article/10.1007/s10462-025-11236-4)</sup> |
| Data needed | Minimum 10 examples for API fine-tuning, with improvements typically from 50–100<sup>[4](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)</sup>; practitioner guides cite roughly 100–10,000 labeled pairs<sup>[5](https://superml.org/courses/llm-fine-tuning/why-fine-tune-llms)</sup> |
| Memory, 7B model | Full fine-tuning about 56–60 GB, LoRA about 16–23 GB, QLoRA about 5.5 GB<sup>[6](https://arxiv.org/html/2407.11046v4)</sup><sup> • </sup><sup>[2](https://llmbook.icsgen-ai.org/part-4-training-adaptation/module-16-fine-tuning-fundamentals/section-16.1.html)</sup> |
| Cost, small run | Self-hosted LoRA on four A5000 GPUs: 56.4–63.6 GPU-hours, about $14.66–$16.54 per run<sup>[7](https://finlora.readthedocs.io/en/latest/benchmark_results/resources.html)</sup> |
| Cost, API | $162.02 for Gemini 2.0 FL and $312.00 for GPT-4o-mini in one benchmark<sup>[7](https://finlora.readthedocs.io/en/latest/benchmark_results/resources.html)</sup> |
| Scale of LoRA | On GPT-3 175B, LoRA cut trainable parameters by 10,000x and GPU memory by 3x<sup>[8](https://huggingface.co/docs/course/main/en/chapter11/4)</sup> |
| Main risk | Catastrophic forgetting: gradient updates on fine-tuning data overwrite weights encoding general knowledge<sup>[2](https://llmbook.icsgen-ai.org/part-4-training-adaptation/module-16-fine-tuning-fundamentals/section-16.1.html)</sup> |
| Knowledge vs behavior | RAG significantly outperformed fine-tuning for injecting new knowledge in controlled comparisons<sup>[9](https://arxiv.org/pdf/2312.05934)</sup> |

## What fine-tuning is

Fine-tuning sits inside transfer learning: a model pretrained on a large general corpus already encodes general language or visual competence, and fine-tuning adapts that competence to a narrower purpose. The Keras transfer-learning guide describes the standard workflow: instantiate a base model with pretrained weights, freeze its layers by setting `trainable = False`, attach a new model on top, and train on the new dataset<sup>[10](https://keras.io/guides/transfer_learning/)</sup>.

A handbook classifies the field into four methods: supervised instruction tuning, which improves generalization across tasks using supervised data; continual learning, which adds new knowledge while guarding against catastrophic forgetting; parameter-efficient fine-tuning, which adapts models by adjusting only a small subset of parameters; and semi-supervised fine-tuning, which makes use of unlabeled data<sup>[11](https://link.springer.com/chapter/10.1007/978-3-032-06264-2_6)</sup>.

## How it works: what gets updated

The central design choice is which weights receive gradient updates, and it matters because it trades memory, forgetting risk, and overfitting risk against capacity.

**Full fine-tuning** updates the entire parameter network. It gives the model the most capacity to absorb new material but requires the most memory and, per Meta's documentation, as much VRAM as pre-training, with a risk that the model forgets knowledge or becomes worse on tasks not in the fine-tuning data<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>.

**Feature extraction** freezes the base model entirely and trains only a small head on top. The Keras guide describes a lighter variant in which the base model runs once and its outputs feed a smaller new model; this is faster and cheaper but prevents dynamic input modification such as data augmentation<sup>[10](https://keras.io/guides/transfer_learning/)</sup>.

**Partial unfreezing** is the middle path. After the head converges, unfreezing all or part of the base model and retraining end-to-end with a very low learning rate can yield incremental improvements, but it risks rapid overfitting on small datasets; the low learning rate is critical because you are training a much larger model on a typically very small dataset<sup>[10](https://keras.io/guides/transfer_learning/)</sup>.

Evaluation follows the same pattern regardless of method: Microsoft's guidance directs practitioners to evaluate on a validation set using accuracy, precision, recall, and F1 score, and to adjust hyperparameters, architecture, or data if performance is unsatisfactory<sup>[13](https://learn.microsoft.com/en-us/windows/ai/fine-tuning)</sup>. OpenAI recommends carving holdout data from the same collected dataset as a control group for evals comparing the fine-tuned model against the base model<sup>[4](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)</sup>.

## Methods and variants

**LoRA** (Low-Rank [Adaptation](https://www.edgepedia.org/adaptation)) freezes the pretrained weights and injects trainable rank decomposition matrices into the model's layers, typically reducing trainable parameters by about 90%<sup>[8](https://huggingface.co/docs/course/main/en/chapter11/4)</sup>. It decomposes the weight update ΔW into two small matrices B (d×r) and A (r×k), which can be stored and reused as pluggable modules for cross-task generalization<sup>[6](https://arxiv.org/html/2407.11046v4)</sup>. Concretely, for the first FFN layer of LLaMA2-7B with rank r = 4, LoRA tunes 60,416 parameters versus 45,088,768 for full fine-tuning, roughly one-thousandth<sup>[6](https://arxiv.org/html/2407.11046v4)</sup>. At inference the adapter weights can be merged with the base model, so there is no additional latency overhead<sup>[8](https://huggingface.co/docs/course/main/en/chapter11/4)</sup>.

**QLoRA** combines LoRA with a quantized base model: only adapter parameters are trainable, while base weights are loaded in lower precision<sup>[8](https://huggingface.co/docs/course/main/en/chapter11/4)</sup>. 4-bit quantization reduces memory used for base weights to 25% of the unquantized base model, and 8-bit to about 50%<sup>[7](https://finlora.readthedocs.io/en/latest/benchmark_results/resources.html)</sup>. The tradeoff is precision: QLoRA may result in lower model quality than LoRA because quantization reduces numerical precision<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>.

**Other PEFT methods** include adapter tuning, [prefix tuning](https://www.edgepedia.org/prefix-tuning), prompt tuning, [P-tuning](https://www.edgepedia.org/p-tuning), and [BitFit](https://www.edgepedia.org/bitfit)<sup>[3](https://link.springer.com/article/10.1007/s10462-025-11236-4)</sup>. PEFT methods generally freeze most original weights and train only adapter layers, reducing memory and compute while often achieving comparable task performance; adapters can be swapped at inference time so one base model serves multiple fine-tuned variants<sup>[14](https://www.databricks.com/blog/llm-fine-tuning)</sup>. Meta's practical ordering is to try LoRA first, QLoRA if resources are extremely limited, and full fine-tuning only when performance is unsatisfactory<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>.

**Instruction tuning versus domain fine-tuning.** [Instruction tuning](https://www.edgepedia.org/instruction-tuning) improves zero-shot and reasoning capabilities but does not necessarily teach the model new knowledge, so it is not, by itself, a reliable solution to the knowledge-injection problem<sup>[9](https://arxiv.org/pdf/2312.05934)</sup>. Preference-alignment methods fit in after supervised fine-tuning: OpenAI's cookbook presents DPO (direct preference optimization) as a preference-alignment fine-tuning technique, and instruction-tuning evaluation compares the overhead of SFT-only versus SFT+RLHF pipelines<sup>[1](https://developers.openai.com/cookbook/examples/fine_tuning_direct_preference_optimization_guide)</sup><sup> • </sup><sup>[15](https://dl.acm.org/doi/10.1145/3777411)</sup>.

## By the numbers

**Data.** OpenAI's supervised fine-tuning accepts a minimum of 10 examples, with improvements typically seen from 50–100; it recommends starting with 50 well-crafted demonstrations and evaluating before scaling<sup>[4](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)</sup>. Practitioner guides put the typical range at 100 to 10,000 labeled input-output pairs, and note that collecting and cleaning this data is often the most expensive part of the project<sup>[5](https://superml.org/courses/llm-fine-tuning/why-fine-tune-llms)</sup>. Databricks reports that a smaller dataset of high-quality examples consistently outperforms a larger noisy dataset across domains, and that data preparation is the most time-consuming phase<sup>[14](https://www.databricks.com/blog/llm-fine-tuning)</sup>.

**Memory.** For a 7-billion-parameter model, one reference gives approximately 56.0 GB for full fine-tuning, 16.0 GB for LoRA, and 5.5 GB for QLoRA, with checkpoints of 14.0 GB, 0.1 GB, and 0.1 GB respectively<sup>[2](https://llmbook.icsgen-ai.org/part-4-training-adaptation/module-16-fine-tuning-fundamentals/section-16.1.html)</sup>. A LoRA survey gives about 60 GB for full fine-tuning of LLaMA2-7B, exceeding an RTX4090, versus about 23 GB for LoRA<sup>[6](https://arxiv.org/html/2407.11046v4)</sup>. The figures agree on the ordering and the practical conclusion that full fine-tuning of a 7B model exceeds common consumer GPUs while LoRA fits on one. LoRA's savings come because optimization and gradient memory decrease by approximately 25 GB and 14 GB respectively, with only about 2 GB of added activation and weight memory from the incremental parameters<sup>[6](https://arxiv.org/html/2407.11046v4)</sup>, and LoRA runs 1.9x faster than full fine-tuning in that comparison<sup>[6](https://arxiv.org/html/2407.11046v4)</sup>.

**Hardware reach.** With a consumer-grade GPU, a Llama 3.1 8B model can be fine-tuned with LoRA and a Llama 3.3 70B model with QLoRA<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>. In the Hugging Face PEFT library, bigscience/T0_3B (3B parameters) drops from 47.14 GB GPU under full fine-tuning to 14.4 GB with LoRA, and a 12B model that runs out of memory on an 80 GB GPU trains with 56 GB GPU and 3 GB CPU<sup>[16](https://github.com/huggingface/peft?tab=readme-ov-file)</sup>. DeepSpeed CPU offloading pushes further: bigscience/bloomz-7b1 trains with 18.1 GB GPU and 35 GB CPU instead of 32 GB GPU with plain LoRA<sup>[16](https://github.com/huggingface/peft?tab=readme-ov-file)</sup>.

**Cost.** Self-hosted LoRA fine-tuning on four NVIDIA A5000 GPUs takes 14.1–15.9 hours wall-clock (56.4–63.6 GPU-hours), about $14.66–$16.54 per run at $0.26 per GPU-hour<sup>[7](https://finlora.readthedocs.io/en/latest/benchmark_results/resources.html)</sup>. Training a 7B model for a few hours on an A100 costs roughly $10–50 on cloud providers<sup>[5](https://superml.org/courses/llm-fine-tuning/why-fine-tune-llms)</sup>. Commercial API fine-tuning in the same benchmark cost $162.02 for [Gemini 2](https://www.edgepedia.org/gemini-2).0 FL and $312.00 for GPT-4o-mini<sup>[7](https://finlora.readthedocs.io/en/latest/benchmark_results/resources.html)</sup>. At the other end of the scale, PEFT research cites about 40 million GPU-hours on H100 GPUs to train LLaMA-3.1 405B from scratch, and notes that a full SFT process requiring 4 million GPU-hours can be reduced to 400K GPU-hours or less with PEFT<sup>[3](https://link.springer.com/article/10.1007/s10462-025-11236-4)</sup>. BloombergGPT's domain-specific training took 53 days on 512 A100 GPUs at an estimated $2.7M<sup>[7](https://finlora.readthedocs.io/en/latest/benchmark_results/resources.html)</sup>.

## How it compares with prompting and RAG

A controlled comparison of knowledge injection in LLMs found that in MMLU and current-events experiments, retrieval-augmented generation significantly outperformed fine-tuning for injecting new knowledge; using RAG with the base model as generator was consistently better than fine-tuning alone, though replacing the base generator with a fine-tuned one sometimes improved results, which the authors attribute to fine-tuning's inherent instability<sup>[9](https://arxiv.org/pdf/2312.05934)</sup>.

Practitioner decision frameworks order the options by cost: exhaust zero-shot prompting, then few-shot prompting or RAG, before fine-tuning, and reserve training from scratch for cases with more than 10k examples, roughly $50,000 of compute, and a fundamentally new domain<sup>[2](https://llmbook.icsgen-ai.org/part-4-training-adaptation/module-16-fine-tuning-fundamentals/section-16.1.html)</sup>. There is also a maintenance asymmetry: a fine-tuned model is a snapshot of its training data, so when requirements change it must be retrained, whereas with RAG the document store can simply be updated<sup>[5](https://superml.org/courses/llm-fine-tuning/why-fine-tune-llms)</sup>.

The two approaches also combine. In an EMNLP 2025 Findings study of four RAG pipelines on HotPotQA and PopQA, independent, joint, and two-phase fine-tuning of the embedding and generator models were about equally effective, with independent fine-tuning the least computationally expensive; it requires context labels in the training data, which the other strategies do not<sup>[17](https://aclanthology.org/2025.findings-emnlp.1247.pdf)</sup>.

On quality relative to full-parameter fine-tuning, the LoRA survey reports that the low-rank constraint limits memorization of downstream knowledge: the effective rank of full fine-tuning is 10–100x higher than LoRA's, and increasing LoRA rank narrows the gap, especially in code and math domains<sup>[6](https://arxiv.org/html/2407.11046v4)</sup>. Databricks' practitioner guide states that PEFT often achieves comparable task performance to full fine-tuning<sup>[14](https://www.databricks.com/blog/llm-fine-tuning)</sup>.

## Recent developments

The PEFT design space has kept moving. PEANuT (KDD 2026) argues that LoRA-style methods rely on weight-agnostic linear approximations that limit expressiveness, and addresses this with weight-aware neural tweakers<sup>[18](https://dlnext.acm.org/doi/10.1145/3770854.3780230)</sup>. Trans-PEFT (ACL 2025) tackles a practical pain point: when a base model is updated, existing adapters normally have to be re-tuned. Trans-PEFT transfers PEFT adapters across updated base model versions, outperforming direct transfer by 15.2% on LoRA, 9.0% on [Adapter](https://www.edgepedia.org/adapter-machine-learning), and 30.7% on DoRA on Qwen commonsense reasoning tasks; direct transfer of DoRA adapters yields only 55.2% average performance, while Trans-PEFT matches re-fine-tuning on the new version<sup>[19](https://aclanthology.org/2025.acl-long.719.pdf)</sup>. The stakes are concrete: fine-tuning PEFT methods on a commonsense reasoning dataset typically takes 14 to 24 GPU-hours on an A800 GPU, and re-tuning after a base model update doubles that to 28 to 48 hours<sup>[19](https://aclanthology.org/2025.acl-long.719.pdf)</sup>.

Quantized fine-tuning has matured into consumer practice, as shown by Meta's guidance that QLoRA fine-tunes a 70B model on a consumer-grade GPU<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>. Vendors have also added safety controls: after a fine-tuning job completes, OpenAI assesses the resulting model's behavior across 13 distinct safety categories<sup>[4](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)</sup>.

## Practice and tooling

A typical workflow runs: prepare and clean data (the most time-consuming phase<sup>[14](https://www.databricks.com/blog/llm-fine-tuning)</sup>), split off a validation set, choose a method and hyperparameters, train, and evaluate on validation metrics such as accuracy, precision, recall, and F1<sup>[13](https://learn.microsoft.com/en-us/windows/ai/fine-tuning)</sup>. For LoRA specifically, recommended starting rank values are 4–8, with monitoring of training loss and validation sets to prevent overfitting<sup>[8](https://huggingface.co/docs/course/main/en/chapter11/4)</sup>.

The tooling ecosystem centers on libraries such as Hugging Face PEFT, whose practical advantage extends beyond training: because only a small fraction of parameters is tuned, each checkpoint is only a few MBs in size instead of GBs, and adapters can reduce catastrophic forgetting and overfitting of the backbone model<sup>[16](https://github.com/huggingface/peft?tab=readme-ov-file)</sup>. In practice, API customers fine-tune hosted models with tens to thousands of examples<sup>[4](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)</sup>, while open-weight practitioners fine-tune 8B models with LoRA and 70B models with QLoRA on consumer-grade GPUs<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>.

## Limitations and risks

[Catastrophic forgetting](https://www.edgepedia.org/catastrophic-forgetting) refers to the loss, once a model has been adapted to a particular task through fine-tuning, of its performance on other tasks that it had previously been able to carry out. Symptoms include degraded MMLU and [HellaSwag](https://www.edgepedia.org/hellaswag) scores, loss of instruction-following, and degenerate repetition; causes include too many epochs, too high a learning rate, and too narrow a dataset<sup>[2](https://llmbook.icsgen-ai.org/part-4-training-adaptation/module-16-fine-tuning-fundamentals/section-16.1.html)</sup>. Full fine-tuning is the most exposed method, since it requires as much VRAM as pre-training and risks forgetting tasks not in the training data<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>. Mitigations follow from the mechanism: PEFT reduces catastrophic forgetting by limiting updates to adapter parameters<sup>[14](https://www.databricks.com/blog/llm-fine-tuning)</sup>, and the Llama Cookbook highlights PEFT as a recommended method for exactly this reason<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>.

**Overfitting** is the most common technical challenge in fine-tuning large language models. The best defenses are data augmentation, PEFT methods that limit the number of trainable parameters, and early stopping based on validation loss<sup>[14](https://www.databricks.com/blog/llm-fine-tuning)</sup>. The Keras guide adds the same warning for the unfreezing stage: retraining a much larger model on a small dataset risks rapid overfitting<sup>[10](https://keras.io/guides/transfer_learning/)</sup>.

**Method-specific limits.** QLoRA's quantization can lower model quality relative to LoRA<sup>[12](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)</sup>, and LoRA's low-rank constraint can limit how much downstream knowledge the model absorbs, with the gap widest in code and math<sup>[6](https://arxiv.org/html/2407.11046v4)</sup>. For knowledge injection specifically, fine-tuning underperformed RAG in controlled tests and showed run-to-run instability<sup>[9](https://arxiv.org/pdf/2312.05934)</sup>. As a control on the safety side, vendor evaluation of fine-tuned models across 13 safety categories before release is now part of the API workflow<sup>[4](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)</sup>.

## References

1. [Fine-Tuning Techniques – Choosing Between SFT, DPO, and RFT, OpenAI Cookbook](https://developers.openai.com/cookbook/examples/fine_tuning_direct_preference_optimization_guide)
2. [Section 16.1: When and Why to Fine-Tune, LLM Book](https://llmbook.icsgen-ai.org/part-4-training-adaptation/module-16-fine-tuning-fundamentals/section-16.1.html)
3. [Parameter-efficient fine-tuning (PEFT): a survey of methods and sustainability, Artificial Intelligence Review (Springer)](https://link.springer.com/article/10.1007/s10462-025-11236-4)
4. [Supervised fine-tuning, OpenAI API documentation](https://developers.openai.com/api/docs/guides/supervised-fine-tuning)
5. [Why Fine-Tune? When and When Not To, superml.org](https://superml.org/courses/llm-fine-tuning/why-fine-tune-llms)
6. [A Survey on LoRA of Large Language Models, arXiv](https://arxiv.org/html/2407.11046v4)
7. [Resource Usage and Cost Analysis, FinLoRA](https://finlora.readthedocs.io/en/latest/benchmark_results/resources.html)
8. [LoRA (Low-Rank Adaptation), Hugging Face Course](https://huggingface.co/docs/course/main/en/chapter11/4)
9. [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, arXiv](https://arxiv.org/pdf/2312.05934)
10. [Transfer learning & fine-tuning, Keras documentation](https://keras.io/guides/transfer_learning/)
11. [Fine-Tuning, handbook chapter (Springer)](https://link.springer.com/chapter/10.1007/978-3-032-06264-2_6)
12. [Fine-tuning, How-to guides, Meta Llama documentation](https://dev.meta.ai/llama/docs/how-to-guides/fine-tuning)
13. [AI model fine-tuning concepts, Microsoft Learn](https://learn.microsoft.com/en-us/windows/ai/fine-tuning)
14. [A Practical Guide to LLM Fine Tuning, Databricks](https://www.databricks.com/blog/llm-fine-tuning)
15. [Instruction Tuning for Large Language Models: A Survey, ACM Computing Surveys](https://dl.acm.org/doi/10.1145/3777411)
16. [huggingface/peft, PEFT library README](https://github.com/huggingface/peft?tab=readme-ov-file)
17. [A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation, EMNLP 2025 Findings](https://aclanthology.org/2025.findings-emnlp.1247.pdf)
18. [PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers, KDD](https://dlnext.acm.org/doi/10.1145/3770854.3780230)
19. [Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models, ACL 2025](https://aclanthology.org/2025.acl-long.719.pdf)

---
*Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Artificial intelligence and data › Machine learning and neural computation › Neural networks and deep learning › Deep learning software and hardware › Fine-tuning and transfer of deep models*

*Initially written Sep 17, 2026 · Reviewed: Sep 17, 2026; Sep 30, 2026; Oct 15, 2026 · Edited: Sep 30, 2026; Oct 11, 2026; Oct 15, 2026 · Last review: Oct 15, 2026*

*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*

License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license