ollama/ollama

Ollama is an open-source platform written in Go that makes running large language models locally as straightforward as a single terminal command. Where tools like llama.cpp expose the raw inference engine, Ollama wraps the entire lifecycle -- model discovery, download, weight management, GPU acceleration, and serving -- into a polished developer experience. Running a model is as simple as typing `ollama run deepseek-v4` or `ollama run qwen3-coder`, and the system handles everything from pulling the right quantization for your hardware to allocating GPU memory and launching an API server. With over 164,000 GitHub stars and 14,700+ forks, Ollama has become the default way developers interact with open-source language models on their own machines. The project builds on llama.cpp for its inference backend but adds critical infrastructure layers on top: a model registry with thousands of pre-packaged models, automatic hardware detection across NVIDIA CUDA, AMD ROCm, and Apple Metal backends, and a REST API server that runs on localhost:11434 by default. The API is compatible with both the OpenAI Chat Completions format and, as of v0.14.0, the Anthropic Messages API -- meaning tools like Claude Code, Codex, Droid, and OpenCode can connect directly to local Ollama instances without proxy layers. The model library is one of Ollama's strongest differentiators. It provides ready-to-run versions of DeepSeek, Qwen, Gemma, Kimi-K2.5, GLM-5, MiniMax, gpt-oss, Mistral, LLaMA, Phi, and dozens more families across a range of parameter sizes and quantization levels. As of early 2026, the library supports over 40,000 model integrations. Specialized models like GLM-OCR for document understanding and Qwen3-VL for vision tasks are available alongside general-purpose chat and coding models. The `ollama launch` command, introduced in v0.15, streamlines the setup of coding agents by automatically configuring environment variables and connecting your preferred development tool to a local or cloud-hosted model. Ollama runs cross-platform on macOS, Linux, and Windows, with official Docker images for containerized deployments. Installation is a one-liner on every platform: a shell script on Linux, a DMG on macOS, or a PowerShell command on Windows. On Apple Silicon, Metal acceleration is automatic with no driver installation required -- the unified memory architecture means your full system RAM is available as GPU memory. On NVIDIA systems, CUDA drivers 535+ are detected automatically. AMD GPU support is available through ROCm 6.0+ on Linux. Recent releases have added structured output support (constraining model responses to JSON schemas), a built-in web search API, NVFP4 and FP8 quantization for up to 35 percent faster token generation on supported hardware, and a redesigned desktop application with file drag-and-drop for document reasoning. The v0.17.6 release in March 2026 refined tool calling for Qwen 3.5 models and fixed GLM-OCR prompt rendering. The project also offers cloud-hosted inference for larger models like GLM-4.6 and Qwen3-coder-480B that exceed typical consumer hardware budgets. Ollama's ecosystem integration is vast. Over 100 third-party projects connect to it, spanning web UIs (Open WebUI, LibreChat), desktop applications (AnythingLLM, Dify, Jan), orchestration frameworks (LangChain, LlamaIndex, Spring AI, Semantic Kernel), and automation platforms (n8n). Native client libraries are available in Python and JavaScript, with community libraries covering Go, Rust, Java, and more.