What is Ollama?
Ollama is a tool for running large language models and other open-weight models on a local computer. It downloads models from the Ollama model library, loads them onto the available CPU or GPU, and serves them through a command-line interface, a desktop chat app and a REST API on localhost:11434. The project is developed in the ollama/ollama repository, created in June 2023, is written mainly in Go and is released under the MIT license. The README lists llama.cpp as a supported inference backend.
Ollama is distributed as a desktop application for macOS and Windows, as an install script and tarballs for Linux on amd64 and ARM64, and as the official ollama/ollama image on Docker Hub. Official client libraries exist for Python (ollama-python) and JavaScript (ollama-js). The macOS documentation requires macOS 14 Sonoma or newer, with Apple M-series chips using CPU and GPU and Intel Macs running on the CPU only; Windows requires Windows 10 22H2 or newer. The hardware support page lists NVIDIA GPUs with compute capability 5.0 or higher and driver 550 or newer, AMD Radeon GPUs through ROCm v7, and additional AMD GPUs through Vulkan.
Besides local inference, Ollama operates a cloud service for larger models. Cloud models are used from the same app, CLI and API after signing in, or directly through https://ollama.com/api with an API key. According to the pricing page, a Pay as you go tier bills per token with no subscription and three concurrent requests, Pro costs $20 per month or $200 per year with $60 of monthly usage credits, Max costs $100 per month with $300 of credits and ten concurrent requests, Team costs $500 per month with unlimited users and $1,000 of shared credits, and Enterprise is priced on request. Running models on local hardware is not metered. The FAQ states that prompts are not seen by Ollama when models run locally, that cloud prompts and responses are processed but not stored, logged or used for training, and that cloud features can be disabled entirely.
The latest release is v0.40.16, published on November 9, 2026. It adds the Ripple Preview model, a 65B dense model for agentic coding, reduces the memory use of Qwen3.8 Flash Next on Apple silicon by about 27 GiB, makes ollama launch claude always route Claude Code to the selected model, fixes truncation handling in /v1/responses, adds CUDA 13.5 support for Windows on ARM and updates llama.cpp. Releases in the 0.40 series appear several times a week.
Similar local inference projects on iThub include llama.cpp, LocalAI and vLLM.
Ollama screenshots
Explore the interface. Open a screenshot to view it at full size.
The Ollama desktop chat app with its model selector.
ollama launch listing Claude Code, Codex, Droid and OpenCode with the model assigned to each.
A vision model answering a question about a local image in an ollama run session.
The ollama.com model library filtered by the Tools capability, from the tool support announcement.
Source: official Ollama project page.
Key capabilities
- Model library and pulls Models are downloaded by name and tag with
ollama pullor on first use withollama run, for exampleollama run gemma4. The library on ollama.com can be filtered by capability, such as embedding, vision, tools and code, and lists available sizes and tags for each model. - Command-line interface The
ollamaCLI runs interactive chats, lists installed models (ollama list), shows loaded models and whether they sit on CPU or GPU (ollama ps), and creates, copies, pushes and removes models. Runningollamawithout arguments opens a prompt to choose a model or connect an integration. The CLI reference lists every command. - Desktop app On macOS and Windows, Ollama installs an app with a chat window, a model selector and a settings screen for network exposure, model storage location, context length and an airplane mode that disables cloud models and web search. The apps download updates automatically and register as a login item, which can be disabled.
- Local REST API The server listens on port 11434 and provides endpoints to generate text, chat, create embeddings, list and pull models, show model details and report running models. Responses can be streamed. The API reference and an OpenAPI specification document each endpoint.
- OpenAI and Anthropic compatibility Ollama exposes a subset of the OpenAI API (including
/v1/chat/completions,/v1/responsesand/v1/models) and of the Anthropic API, so existing clients can point at a local or cloud Ollama endpoint by changing the base URL. - Model capabilities Depending on the model, the documentation covers streaming, thinking output, structured outputs with a JSON schema, vision input, embeddings for retrieval, tool calling, text-to-speech and decision-style yes/no scoring.
- Modelfile and imports A Modelfile defines a base model, parameters, a system prompt and a prompt template, and
ollama createbuilds a new model from it. Models can be imported from GGUF files or Safetensors weights and pushed to ollama.com for sharing. - Context length control The default context window is 4,096 tokens. It can be raised globally with
OLLAMA_CONTEXT_LENGTH, per session with/set parameter num_ctx, per request with thenum_ctxoption, or through the slider in the desktop app. - GPU acceleration NVIDIA GPUs run through CUDA, AMD GPUs through ROCm v7 or Vulkan, and Apple silicon through its GPU. When a model does not fit on a single GPU, Ollama spreads it across all available GPUs;
CUDA_VISIBLE_DEVICESlimits which NVIDIA cards are used. - Integrations with coding agents
ollama launchstarts tools such as Claude Code, Codex CLI, Copilot CLI, OpenCode, Droid and DeepSeek Harness configured to use an Ollama model. The integration guides also cover editors and apps such as VS Code, JetBrains, Xcode, Zed, Cline, Goose and n8n. - Cloud models Larger models run on Ollama's infrastructure and are selected like local ones, for example with a
:cloudtag. They can also be called without installing Ollama by sending requests with an API key to ollama.com. The pricing page lists per-million-token prices for each cloud model, with lower off-peak rates for some models. - Web search API A REST endpoint at
ollama.com/api/web_searchreturns search results that models can use as tool input, with deeper integration in the Python and JavaScript libraries. It requires an API key from a free Ollama account. - Local-only mode Setting
disable_ollama_cloudin~/.ollama/server.jsonorOLLAMA_NO_CLOUD=1turns off cloud models and web search; the logs then confirm that cloud access is disabled. - Installation on macOS and Windows MacOS uses
curl -fsSL https://ollama.com/install.sh | shor a DMG download, and Windows usesirm https://ollama.com/install.ps1 | iexor theOllamaSetup.exeinstaller. - Installation on Linux The same install script sets up Ollama on Linux. Manual installs use
.tar.zstarchives for amd64 and ARM64, with a separate ROCm package for AMD GPUs, as described in the Linux guide. - Docker image
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollamastarts the server in a container, with--gpus=allfor NVIDIA GPUs and therocmimage tag for AMD GPUs. NVIDIA Jetson systems select JetPack 5 or 6 through an environment variable.
Where Ollama fits
Chatting with an open model on a laptop
After installing the app or running the install script, ollama run with a model name downloads the model and opens an interactive session; the desktop app offers the same through a chat window with a model picker. Images can be passed to vision models by including a file path in the prompt. ollama ps shows how much of a loaded model sits on the GPU.
Serving a local API for applications
Applications call the REST API on port 11434, or use the Python and JavaScript libraries, to generate chat responses, structured JSON output and embeddings. Because of the OpenAI and Anthropic compatibility layers, existing SDKs can be pointed at Ollama by changing the base URL. To reach the server from other machines, the documentation describes binding to the network, reverse proxies, ngrok and Cloudflare Tunnel.
Running coding agents on local or cloud models
ollama launch lists supported coding tools and starts the selected one, such as Claude Code, Codex or OpenCode, with an Ollama model. The same tools can use local models on capable hardware or cloud models when a larger model is required.
Retrieval and embeddings
Embedding models from the library generate vectors through the embed endpoint for semantic search and retrieval-augmented generation. Local chat front ends such as Open WebUI connect to an Ollama server for chat and document workflows.
Packaging custom models
A Modelfile sets the base model, generation parameters, a system prompt and a template, and ollama create builds a named model from it. Weights in GGUF or Safetensors format can be imported, and finished models can be pushed to an ollama.com namespace.
Running Ollama in containers and on servers
The official Docker image runs the server with a named volume for models and optional GPU access. On Linux hosts, the install script or manual archives set up the server directly, and environment variables control the context length, model storage location, loaded model lifetime, request queue and concurrency.
Install
curl -fsSL https://ollama.com/install.sh | shReviews of Ollama
Be the first to share your experience with Ollama.
Write a review
Loading review form…
Alternatives to Ollama
LocalAILocalAI is an MIT-licensed inference server that runs language, speech, image, video and vision models on CPUs or GPUs and serves them through OpenAI, Anthropic, Ollama and ElevenLabs-compatible APIs, with backends pulled on demand.
vLLMvLLM is an Apache-2.0 library and OpenAI-compatible server for LLM inference, with PagedAttention, continuous batching and distributed execution.