Ollama

Ollama is an open-source, MIT-licensed tool, distributed as a single Go binary, that downloads, runs, and serves large language models on a local computer with one command and a local REST API.¹ Its stated purpose is to make open models usable in desktop apps, coding agents, and applications without data leaving the machine: with open models that run locally, the company says, "your data never has to leave your machine."² The project uses llama.cpp as an inference engine, created by Georgi Gerganov.³

History and development

Ollama v0.0.1 shipped on 2023-07-08 for macOS as "an early preview" that pulled a model from its library and ran it with one command, exposing a REST API for applications.⁶ It is written in Go and built on top of llama.cpp by Ollama Inc., founded by Jeffrey Morgan and Michael Chiang, both formerly of Docker and Y Combinator alumni.¹⁰ Founder and CEO Jeff Morgan has said that open models in 2023 were geared toward researchers rather than programmers and were hard to get running, which is the gap the tool filled.⁴

The company raised $88 million from Peter Fenton at Benchmark, Tomasz Tunguz at Theory Ventures, and Alex Kolicich at 8VC, with angels including Solomon Hykes, Spencer Kimball, Quinn Slack, and Y Combinator.² A $65 million Series B led by Theory Ventures followed in July 2026, bringing total funding to $88 million, with usage doubling since January and nearly a million new installs per week.¹¹ The project is run by the company with no foundation; its license file reads MIT License over Copyright (c) Ollama.⁶

How it works

Ollama wraps llama.cpp in a user-friendly interface inspired by Docker, adding pull/run workflows, automatic GPU detection, memory management, and API serving.¹² More precisely, it is a Go-based service that bundles a pinned llama.cpp build plus a model-pull layer that fetches GGUF files from a registry; GGUF-model generation uses llama.cpp, while supported models on Apple silicon can use MLX.¹³ llama.cpp is a C/C++ inference library created by Georgi Gerganov in 2023 to run Meta's LLaMA on commodity hardware, and the GGUF (GGML Universal File) format was introduced in August 2023 by the llama.cpp developers as a successor to the earlier GGML format.¹⁴ llama.cpp implements a large set of weight-only quantization formats in GGUF, including the original schemes (Q4_0, Q4_1, Q5_0, Q5_1, Q8_0) and a K-quant family (Q2_K, Q3_K*, Q4_K*, Q5_K*, Q6_K).¹⁵

Modelfile. A Modelfile packages a custom model configuration as a reusable unit, with instructions including FROM (base model, required), PARAMETER (runtime settings such as temperature and context length), TEMPLATE (prompt format), SYSTEM (system prompt), and ADAPTER (LoRA adapters), built with ollama create -f Modelfile.^{12 • 16} Models can be created from another model, a safetensors directory, or one or more GGUF files, and model names follow a model:tag format with an optional namespace such as example/model; the tag defaults to latest.¹⁷

API. At http://localhost:11434 the server offers a native REST API whose endpoints include /api/generate, /api/chat, /api/embeddings, /api/pull, and /api/create. It also provides an OpenAI-compatible interface under /v1, covering /v1/chat/completions, /v1/completions, /v1/models, and /v1/embeddings; because of this, programs built on the OpenAI SDK need only swap the base URL to target Ollama.¹⁶ The OpenAI compatibility is a subset of the OpenAI API, allowing OpenAI clients to connect.¹ The API is not strictly versioned but is expected to be stable and backwards compatible, with deprecations announced in release notes.¹

Models and hardware requirements

The library carries major open model families: the repository README advertises one-command setup for Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, and Gemma, among others.³ Models are pulled with ollama pull, or ollama run hf.co/{username}/{repository} runs any GGUF on the Hugging Face Hub; several models load and unload on demand, up to 3 times the number of GPUs at once by default.⁶ A statistical analysis counted 239 models in the library with over 1.04 billion total pulls, led by llama3.1 with 119.2 million pulls and deepseek-r1 with 92.4 million; another comparison puts the curated library at roughly 150 models plus imported GGUFs, so the exact count depends on how the library is measured.^{5 • 18} In the 239-model census, 93 models (38.9%) carried the tools capability tag, 42 (17.6%) had thinking or reasoning tags, 38 (15.9%) supported vision, and 18 (7.5%) were cloud-hosted.⁵

Hardware needs scale with parameter count at 4-bit-class quantization (Q4_K_M or Q5_K_M is the commonly recommended balance of quality, speed, and hardware cost, and actual memory use exceeds file size because of the KV cache, context, and buffers):¹⁹

• 7B models: 16 GB RAM for CPU inference or 8–12 GB VRAM for GPU inference; 8–20 tokens/s on CPU versus 35–90 tokens/s on GPU.¹⁹ A second guide reports 6–15 tokens/s for a 7B Q4 on a Zen 4 or Apple Silicon CPU, a somewhat lower range.²⁰
• 70B models: 64–96 GB RAM (CPU) or 48–80 GB VRAM (GPU) for efficient inference; roughly 40 GB of weights, 48 GB recommended memory, and 140 GB disk.^{19 • 20} A 70B Q4 model runs at 1–3 tokens/s on CPU.²⁰
• Small models: a 3B model needs 8 GB RAM (CPU) or 4 GB VRAM (GPU).¹⁹

On a single RTX 4090, Ollama yields 228 tokens/s for Llama 3 8B Q4_K_M (batch 1, context 512, output 128) using 6.2 GB VRAM, and Mistral 7B Q4_K_M delivers 312 tokens/s at 4.1 GB VRAM.²¹

By the numbers

Adoption grew quickly from the 2023 launch: by mid-2026 the repository had about 176,000–180,000 stars (repository metadata shows 180,067; the founder cited 176,000 at the Series B announcement) and nearly 17,000 forks.^{3 • 4} The company reports over 8.9 million monthly active developers and presence in 85% of the Fortune 500, with only 14 employees.⁴ Installs run at nearly a million per week, and the v0.40.3 release dated 2026-10-11 alone recorded 639,000 downloads.¹¹

Throughput under load is the number that most distinguishes Ollama from GPU-serving stacks. Under concurrent load, benchmarks show vLLM reaching roughly 793 tokens/s while Ollama tops out around 41 tokens/s, a gap of roughly 16–20× explained by vLLM's PagedAttention and continuous batching, while Ollama supports continuous batching but lacks PagedAttention.^{18 • 22 • 31} For single-user local use this rarely matters; for multi-user serving it is the reason practitioners switch.²²

How it compares with alternatives

Ollama sits in a family of local-model tools that includes interfaces using llama.cpp and GGUF, such as LM Studio, KoboldCPP, GPT4All, and Jan.¹³ On licensing, Ollama and llama.cpp are MIT-licensed open source, while LM Studio is proprietary but free including for work use since July 2025; default ports are 11434, 1234, and 8080 respectively.²³

• Raw llama.cpp is faster: in GPU-accelerated tests it generated tokens 8–15% faster than Ollama and 2–5% faster than LM Studio, though Ollama's daemon architecture gives it the fastest warm time-to-first-token for repeated queries.²⁴ Another measurement attributes 15–25% higher token generation speed to direct llama.cpp use, due to wrapper overhead.⁷
• vLLM is the choice for multi-user production serving on GPUs, at roughly 16–20× Ollama's throughput under concurrent load; Ollama is positioned as the lowest-friction default for one-developer prototyping.²²
• Format limits: The llama.cpp-based runner uses GGUF rather than GPTQ, AWQ, or exl2 quantized models, and Ollama's sequential-by-default queue can limit its suitability for more than two or three concurrent users.⁷
• As a backend: Ollama is commonly used to power interfaces such as AnythingLLM and Open WebUI, a self-hosted ChatGPT-like client with RAG, voice, and plugins; direct competitors include LM Studio (GUI-first), llama.cpp itself, vLLM (production-scale multi-GPU), Jan, and MLX.²⁵

Security and privacy

The privacy rationale is local inference: open models running on the user's own machine mean prompts and data need not leave it.² The server binds by default to 127.0.0.1 on TCP port 11434, but many deployments override this with OLLAMA_HOST=0.0.0.0:11434, and the /api/create and /api/push endpoints have no authentication in the upstream distribution; large public-internet exposure has been observed.^{26 • 8} The security firm Wiz counted over 1,000 exposed instances when it disclosed CVE-2024-37032 on 2024-06-24, a flaw in model downloads that let a malicious registry overwrite files on the server; Ollama shipped a fix in v0.1.34 within days of the 2024-05-05 report.⁶

CVE-2026-7482 is a heap out-of-bounds read in the GGUF model loader affecting versions before 0.17.1, triggered via /api/create with a GGUF file whose declared tensor offset and size exceed the file's actual length. Leaked memory may include environment variables, API keys, system prompts, and concurrent users' conversation data, exfiltratable via /api/push to an attacker-controlled registry.⁸ Earlier, Sonar researchers found a remote code execution vulnerability in versions before 0.7.0, exploitable by an attacker with API access who can load and run a malicious model; version 0.7.0 removed the vulnerable code.²⁷

The Windows updater chain is the most serious recent case. Researchers at Striga disclosed CVE-2026-42248 and CVE-2026-42249, affecting versions 0.12.10 through 0.17.5: the Windows update verification routine unconditionally returns success, performing no digital signature or trust validation before staging or executing update payloads, and a path traversal via attacker-controlled HTTP response headers passed to filepath.Join lets files be written outside the update staging directory, including the Windows Startup directory; chaining the two yields automatic persistent code execution.⁹ After five weeks with no engagement from maintainers, CERT Polska took over disclosure coordination and published a warning on April 29, 2026.²⁸ As of May 7, 2026 the fix had not shipped in a tagged release (the latest tagged release, v0.23.2, predates it), so every Windows build from 0.12.10 through v0.23.2 should be treated as vulnerable; the fix adds Authenticode verification via WinVerifyTrustEx requiring signer organization "Ollama Inc."²⁶ RedEye Security reported that these vulnerabilities exposed more than 300,000 servers to memory leaks and persistent code execution.²⁹ Defensively, the practical guidance follows from the failure modes: keep the default 127.0.0.1 binding unless remote access is genuinely needed, front any exposed instance with authentication, and update Windows installations promptly once a fixed release ships.

Recent developments

The release timeline shows the tool expanding from a macOS CLI into a multi-platform product:⁶

• 2023: v0.0.1 (July, macOS); v0.1.0 (September) added Linux with NVIDIA acceleration out of the box.
• 2024: Windows previewed 2024-02-15; v0.4.0 (2024-11-05) added Llama 3.2 Vision using Ollama's own Go runner.
• 2025: v0.6.0 (2025-03-11) introduced a new engine required for Gemma 3; v0.10.0 (2025-07-30) shipped a desktop app for macOS and Windows; v0.12.0 (2025-09-18) added cloud models in preview, followed by a web search API in v0.12.2.
• 2026: v0.19.0 (2026-03-27) added Apple's MLX as an engine on Apple silicon in preview, raising decode speed from 58 to 112 tokens/s on an M5 Max running Qwen 3.5-35B-A3B; v0.30.0 (2026-06-01) moved GGUF models onto upstream llama.cpp's llama-server and enabled Vulkan by default; since v0.40.0, MLX is the default for supported models on Apple Silicon.^{6 • 18 • 23}

Commercialization arrived alongside the cloud. Ollama's cloud hosts larger models via subscription tiers from free to $100 per month, billing by GPU time rather than token limits, and the same OpenAI-compatible API that addresses a local 7B model can swap to a 400B cloud model with a single string; cloud requests require an API key while local requests do not.^{4 • 25 • 1} The firm has also started to transform what was once a free open-source utility into a tiered commercial offering, with the Pro tier costing $20 monthly and a more expensive Max plan above it.³⁰ On the developer side, the ollama launch command configures external coding tools, including Claude Code, Codex, VS Code, OpenCode, and Droid, to use a locally running Ollama model as backend, and the API gained a think parameter controlling a model's thinking output.^{16 • 17}

References

1. Ollama documentation
2. Ollama: all aboard open models
3. ollama/ollama GitHub repository
4. Ollama raises $65M, grows to nearly 9M users, TechCrunch
5. Ollama Model Library, By the Numbers, DevToolHub
6. A short history of Ollama, Strata2Signal Research
7. Ollama 2026 Review: The Default Local LLM Runner, aifoss.dev
8. NVD – CVE-2026-7482
9. Vulnerabilities in Ollama software, CERT Polska
10. Tool Highlight: Ollama, dreaming.press
11. Ollama – Repository Radar
12. Section 60.2: The Edge Framework Landscape, Building Language AI
13. GGUF format and k-quants: local LLM inference, explained, ZeroEntropy
14. Small and Fast LLMs on Commodity Hardware: Post-Training Quantization in llama.cpp, University of Bonn, 2025
15. Which Quantization Should I Use? A Unified Evaluation of llama.cpp Quantization on Llama-3.1-8B-Instruct, arXiv
16. Ollama Review 2026: Local LLM CLI, API & Install Guide, Prompt Quorum
17. docs/api.md — Ollama API reference
18. Ollama vs LM Studio vs vLLM vs llama.cpp vs MLX 2026, Codersera
19. Ollama Server Requirements: RAM, VRAM, and CPU by Model Size, Valebyte
20. Ollama Hardware Requirements, 2026 Guide, MyAIHardware
21. Ollama Benchmark: RTX 4090 Hits 228 Tokens/Sec, MarkAICode
22. Ollama vs LM Studio vs vLLM vs llama.cpp: Which Should You Use in 2026?, Convly
23. Ollama vs LM Studio vs llama.cpp (2026): Which to Use, Decode the Future
24. Ollama vs LM Studio vs llama.cpp: 5 Speed Tests Ranked, AIBytes
25. Open-source AI developer tool Ollama raises $65M to grow its platform, SiliconANGLE
26. Three Ollama CVEs in One Week, Mondoo
27. Ollama Remote Code Execution, Sonar
28. Unpatched flaws turn Ollama's auto-updater into a persistent RCE vector, Help Net Security
29. Critical Ollama Vulnerabilities Expose 300,000+ Servers, RedEye Security
30. Ollama: Developer Tools commercialization case study, Ferryman
31. github.com

Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Modern AI: foundation models, generative AI, and the AI industry › AI companies, people, and products › AI products and assistants › AI agents and developer tools

Initially written Sep 17, 2026 · Reviewed: Sep 30, 2026; Oct 15, 2026 · Edited: Sep 30, 2026; Oct 11, 2026; Oct 15, 2026 · Last review: Oct 15, 2026

© 2026 EdgeChat AI, a subsidiary of Biostate AI. Free to use with credit under the Edgepedia Community License. Developers: read Edgepedia by API or MCP. Embed a reference card.