How to Run Local AI Models (LLMs) on Your Laptop for Free

Running an LLM locally means the model weights live on your own machine—giving you zero per-token API fees, offline access, and complete data privacy.

Hardware Requirements (Realistic Numbers)

Laptop RAM / VRAM	Recommended Model Size	Example Models
8 GB RAM	3B – 4B (quantized)	Llama 3.2 3B, Qwen 2.5 3B
16 GB RAM	7B – 9B (Q4_K_M)	Qwen 2.5 7B, Llama 3.1 8B, Gemma 2 9B
32+ GB RAM	14B – 32B	Qwen 2.5 Coder 14B / 32B

Apple Silicon Macs (M1/M2/M3/M4) perform especially well because unified memory allows the GPU to access system RAM directly.

Step 1: Install Ollama

Ollama is the fastest way to download and run quantized open-weight models from the terminal or via a local REST API.

1. Download Ollama from ollama.com for macOS, Windows, or Linux.
2. Verify installation in your terminal:

ollama --version

Step 2: Pull and Run Your First Model

Pick a model sized for your RAM. On a 16 GB laptop, a 7B–8B model is the sweet spot:

ollama run qwen2.5:7b

Ollama downloads the quantized weights once and opens an interactive chat prompt immediately.

Step 3: Call Your Local Model from Code

Ollama automatically exposes an OpenAI-compatible local endpoint at http://localhost:11434:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5:7b",
  "prompt": "Explain HTTP caching in 3 bullet points.",
  "stream": false
}'

Tips for Faster Local Inference

• Close heavy browser tabs so the model stays in RAM/VRAM instead of swapping to disk.
• Use coding-specialized models (like qwen2.5-coder) for autocomplete and refactoring tasks.
• Prefer Q4_K_M quantization—it cuts memory usage roughly in half with minimal quality loss.

Next, read How AI Agents Work: Tools, Memory, and Planning Explained to see how local models can power autonomous workflows.