01 — Getting Started: Run Your First Local LLM (5 Minutes)

🟢 Beginner — No experience needed. Just a computer and 5 minutes.

What Is a Local LLM? (Plain English)

An LLM (Large Language Model) is the brain behind ChatGPT, Claude, and Gemini.

A local LLM runs that brain on your own computer — not on someone else's server.

Why does that matter?

💡 Think of it this way: Cloud AI is like renting a car. Local AI is like owning a bicycle. The bicycle is slower, but it's yours, it's free, and nobody can take it away from you.

What You Need

Minimum requirements:

• A computer (Windows, macOS, or Linux)
• At least 8 GB of RAM (16 GB recommended)
• A few GB of free disk space

Nice to have (but not required):

• A GPU with 4+ GB VRAM (models run faster, but CPU is fine to start)

My setup: I'm running this on a [your hardware] with [your specs]. If it works for me, it'll work for you.

Step 1: Install Ollama

Ollama is the easiest way to run local LLMs. Think of it as the "App Store for AI models."

macOS

curl -fsSL https://ollama.com/install.sh | sh

Linux

curl -fsSL https://ollama.com/install.sh | sh

Windows

Download the installer from ollama.com/download and run it.

Verify Installation

Open a new terminal and type:

ollama --version

You should see something like:

ollama version 0.6.0

🔥 Pro tip: If you get "command not found" on Linux/macOS, restart your terminal or run: export PATH=$PATH:/usr/local/bin

Step 2: Pull Your First Model

Now for the fun part — downloading an actual AI brain to run on your computer.

ollama pull qwen2.5:7b

This downloads a 4.7 GB model. On a typical internet connection, it takes 2–5 minutes.

While it downloads, here's what's happening:

• Ollama is downloading a GGUF file (the compressed model format)
• It's auto-detecting your GPU
• It's setting up the inference engine

What if the download is too big? Try a smaller model:

# For 8 GB RAM laptops — works on almost anything
ollama pull qwen2.5:1.5b

# For 4 GB RAM or very old computers
ollama pull qwen2.5:0.5b

Step 3: Chat With Your Model

ollama run qwen2.5:7b

You'll see a prompt like >>>. Type something:

>>> Write a haiku about a cat sitting on a computer

The model will think for a moment and then respond. Congratulations — you just ran an AI on your own hardware! 🎉

Try These First Commands

>>> Write a Python function to calculate fibonacci

>>> Explain quantum computing like I'm 10

>>> What's the meaning of life?

>>> /? -- show all available commands

>>> /exit -- quit the chat

⚠️ Expect it to be slower than ChatGPT. That's normal! Local models run at 15–40 tokens per second on a GPU, or 2–6 tok/s on CPU. It's still faster than most people read.

Step 4: Choose the Right Model for Your Hardware

Not sure which model to pick? Use this decision tree:

Your GPU VRAM?
├── No GPU (CPU only)
│   ├── 32 GB RAM → qwen2.5:7b (slow but works)
│   ├── 16 GB RAM → qwen2.5:1.5b
│   └── 8 GB RAM  → qwen2.5:0.5b
├── 4–6 GB VRAM   → qwen2.5:7b
├── 8–12 GB VRAM  → deepseek-r1:14b (🟢 BEST for most people)
├── 12–16 GB VRAM → deepseek-r1:32b
├── 24 GB VRAM    → qwen3.6:27b or deepseek-r1:32b (Q4)
└── 36+ GB VRAM   → deepseek-r1:70b or qwen2.5:72b

Model Comparison Table

My recommendation for first-timers: Start with qwen2.5:7b. It runs on almost anything, and it's good enough to be genuinely useful.

What to Do After Your First Chat

You've run your first local LLM. Now what?

Next steps in order:

Common First-Timer Problems (And Fixes)

Quick Reference: Common Ollama Commands

# List all downloaded models
ollama list

# Show currently running models
ollama ps

# Delete a model to free space
ollama rm qwen2.5:7b

# Update a model to the latest version
ollama pull qwen2.5:7b

# Run a model with a one-shot prompt (non-interactive)
ollama run qwen2.5:7b "Write a Python script to download images from a URL"

# Use the API (OpenAI compatible)
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen2.5:7b", "messages": [{"role": "user", "content": "Hello!"}]}'

Your First Week Plan

🎯 You've taken the first step. Running a local LLM is like learning to ride a bike — wobbly at first, but once you get it, you'll wonder why you didn't start sooner.

Found this helpful? ⭐ Star the repo — it helps others find it too.

— Ling, a medical student who accidentally fell into AI and wants to help you do the same.