VS Code and Continue with Ollama: A Local AI Coding Setup

The goal: Copilot-style help in VS Code, entirely local

Continue is an open-source extension that adds chat, inline edits, an agent mode and tab autocomplete to VS Code, and it treats Ollama as a native provider. Running Continue with Ollama gives you most of what a hosted assistant does without sending code anywhere. It is the closest thing VS Code users get to the setup I described for using Cursor with local LLM models, and it costs nothing.

The trick is not to use one model for everything. Chat, autocomplete and codebase search have very different needs, and Continue lets you assign a model to each role.

Step 1: pull three models

# chat, edit and agent work
ollama pull qwen2.5-coder:7b

# fast fill-in-the-middle autocomplete (a base model, not instruct)
ollama pull qwen2.5-coder:1.5b-base

# embeddings for @codebase search
ollama pull nomic-embed-text

If you have 32GB or more, swap the chat model for something larger such as qwen3-coder:30b. Keep autocomplete small: it fires on every pause in typing, so latency matters more than cleverness.

Step 2: install Continue

1. Install the Continue extension from the VS Code Marketplace.
2. Open the Continue sidebar and skip any sign-in prompt; local models do not need an account.
3. Open the local configuration from the config selector in the sidebar. Continue’s global local config is a config.yaml in the .continue folder in your home directory.

Step 3: write config.yaml

Continue now uses config.yaml; the older config.json is deprecated. This config assigns each model a role:

name: Local Config
version: 1.0.0
schema: v1
models:
  - name: Qwen2.5 Coder 7B
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply
    capabilities:
      - tool_use
    defaultCompletionOptions:
      contextLength: 16384
  - name: Qwen2.5 Coder 1.5B (autocomplete)
    provider: ollama
    model: qwen2.5-coder:1.5b-base
    roles:
      - autocomplete
  - name: Nomic Embed
    provider: ollama
    model: nomic-embed-text
    roles:
      - embed

• model must match exactly what ollama list shows, including the tag.
• capabilities: [tool_use] is needed for agent mode if Continue does not detect tool support on its own.
• contextLength keeps memory use predictable. Raise it if you have headroom.
• The embed role powers codebase retrieval; without it, @codebase style context will not work properly.

Step 4: use the roles

• Chat: ask questions with selected code or files as context.
• Edit: highlight code, describe the change, review the diff.
• Agent: lets the model call tools to read and modify files. This is where small models struggle most, so be specific in your prompts.
• Autocomplete: inline suggestions as you type, served by the small base model.

Troubleshooting

• 404 or “model not found”: the tag is not pulled. ollama pull qwen2.5-coder fetches :latest, which may not be the tag your config names.
• Agent says tools are unsupported: add tool_use to capabilities, or switch to a model with confirmed tool support. Reasoning-focused models often do not cooperate here.
• “Model requires more system memory”: lower contextLength or use a smaller model.
• Autocomplete is junk or repeats itself: make sure the autocomplete model is a base/FIM-capable model, not a chat instruct model.
• Cannot connect: check curl http://localhost:11434. Use ollama pull, not ollama run; Continue does not need an interactive session open.

# check what context length a loaded model is actually using
ollama ps

# raise the server-wide default (or use the slider in the Ollama app settings)
OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Verdict

Continue with Ollama is the most flexible local option for VS Code because each job gets the right model. Autocomplete on a 1.5B base model feels instant on almost any machine; chat and edit are fine on 7B-14B; agent mode is where you will want the biggest tool-capable model you can fit. If you prefer a terminal agent alongside it, see Claude Code with local LLM models.

This post is part of a series on running AI coding tools against local models. The others cover Zed with Ollama, VS Code and Continue with Ollama, JetBrains AI Assistant with local models, Codex CLI with Ollama, Xcode with Ollama, and a capstone on choosing a local coding model for your hardware.