How to run AI models on your own machine with Ollama

Moath Samara · Published October 17, 2026

Running an AI model on your own machine means what you type stays with you, and you do not pay a provider for every request. Ollama makes it a one-command job. This guide follows the official docs, from installation to calling the model from code.

What you need

• A computer running macOS, Windows or Linux.
• Disk space for the model. The example model in the official docs (Gemma 4 E2B) is about 7.2 GB.
• Enough memory. The docs recommend 8 GB of free VRAM, or unified memory on a Mac. With less, Ollama uses system RAM, but answers can be slower, and larger context windows need more memory.

Step 1: Install Ollama

Download Ollama from ollama.com/download. Open the app, or start from the terminal by typing the command below and following the prompts:

On Linux you can install with one command:

If the server is not running on Linux, start it with ollama serve. To check that everything works, run ollama -v.

Step 2: Run your first model

One command downloads the model and opens a chat on your machine:

Wait for the download, then type your question. To leave the chat, type /bye.

Step 3: Use the model from code

Ollama runs a local server on your machine. Pull the model first:

Then send a request to the local server:

curl http://localhost:11434/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:e2b",
    "messages": [
      {"role": "user", "content": "Say hello in one sentence."}
    ],
    "stream": false
  }'

The answer is in the field message.content. Ollama also offers OpenAI- and Anthropic-compatible endpoints for part of their features, such as http://localhost:11434/v1/chat/completions, so many tools can connect to your local model. See the compatibility page in the docs for what is supported.

Windows note: in PowerShell the command may need curl.exe and different quoting. You can also use Python or WSL.

Step 4 (optional): try a coding agent with a local model

From your project folder, launch a coding agent through Ollama:

Ollama offers to install Claude Code if you do not have it, then asks you to choose the model. Try a simple request such as Explain how this repository is organized. Ollama supports Codex CLI and OpenCode the same way. If the command is not found, update Ollama.

Local or cloud models?

Ollama shows two kinds of models. Local models run on your machine and need no API key. Cloud models need a sign-in and an API key, and they run on Ollama's servers, not on your machine. This guide covers local models.

Common problems

• Very slow, or out of memory: try a smaller model from the library and close heavy applications.
• The command cannot reach the server on Linux: run ollama serve in one window and your commands in another.
• Where are models stored? In ~/.ollama/models on macOS and Linux, and in %USERPROFILE%\.ollama\models on Windows (according to Real Python). Mind your disk space, since models are large.

FAQ

Sources

• Ollama Quickstart (official docs)
• Ollama on Linux (official docs)
• Download Ollama
• Real Python: How to Use Ollama (secondary source for model storage paths)