Unlock the Hidden Local AI LLM On Your Mac Already Built In

If you’ve been chasing local LLMs by downloading multi-gigabyte model files through Ollama or LM Studio, here’s a surprise: if you’re on an Apple Silicon Mac running macOS Tahoe, you already have a language model sitting on your machine. Apple ships it as part of Apple Intelligence, and it powers Siri, Writing Tools, and a handful of system features. It just doesn’t have a front door.

That’s what apfel fixes. It’s a small, open-source Swift binary that wraps Apple’s FoundationModels framework and exposes the on-device model as a CLI tool, an OpenAI-compatible server, and an interactive chat – all from a single brew install. No model to download, no API key to generate, nothing to configure.

What Model Is Actually Running?

Apple’s on-device model (AFM, or Apple Foundation Model) is a roughly 3 billion parameter LLM, quantised down to a mix of 2-bit and 4-bit precision, running entirely on the Neural Engine. It has a 4,096 token context window and supports nine languages including English, German, Spanish, French, Japanese, and Chinese.

That’s small by the standards of a full Ollama setup – a 4K context window won’t hold a large codebase – but it’s genuinely capable at the things a small model is good at: text transforms, classification, translation, short summaries, and shell scripting glue. And because it’s already on disk as part of the OS, there’s zero setup cost to try it.

Requirements

• macOS 26 (Tahoe) or newer – the FoundationModels framework doesn’t exist on Sequoia or earlier
• Apple Silicon (M1 or later)
• Apple Intelligence enabled in System Settings, which is what actually downloads the model onto your machine

Installing apfel

With Homebrew, it’s one line:

brew install apfel
apfel "Hello, Mac"

Apple says the round trip from install to output takes about 12 seconds. If you’d rather build from source, clone the GitHub repo and run make install – you’ll need the Command Line Tools and the macOS 26.4 SDK.

Three Ways to Use It

As a UNIX tool

apfel is pipe-friendly by design, with stdin/stdout, JSON output, file attachments, and proper exit codes:

apfel "What is the capital of Austria?"
apfel -o json "Translate to German: Apple" | jq .content
apfel --code "flush the dns cache on macos"

That composability is the real draw. You can chain apfel with everyday tools like ps, lsof, du, git, and pbpaste in ways a cloud API simply can’t match, because a cloud model never sees your running processes or your clipboard:

# which app is hogging the CPU
ps aux | sort -k 3 -nr | head -5 | apfel "which app is using the most CPU"

# fix clipboard grammar and put it straight back
pbpaste | apfel "fix grammar" | pbcopy

# summarise your last half hour of commits
git log --since="30 min ago" --oneline | apfel "summarize my last 30 minutes of work in one sentence"

As an OpenAI-compatible server

This is the part that plugs straight into the wider local-LLM ecosystem. Run:

apfel --serve
# Server running on http://127.0.0.1:11434

and any OpenAI SDK can talk to it by changing one line:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="unused"
)

resp = client.chat.completions.create(
    model="apple-foundationmodel",
    messages=[{"role": "user", "content": "What is 1+1?"}],
)
print(resp.choices[0].message.content)

It supports streaming, tool calling, CORS, and response_format: json_object. Note the port – 11434 is the same default Ollama uses, so if you’re already running Ollama, start apfel’s server on a different port or stop one of them first.

As an interactive chat

apfel --chat -s "You are a coding assistant"

Multi-turn conversations with automatic context trimming – useful given how tight that 4,096 token window is.

MCP Support

apfel speaks the Model Context Protocol natively, so you can give Apple’s on-device model tools:

apfel --mcp ./mcp/calculator/server.py "What is 15 times 27?"
# tool: multiply({"a": 15, "b": 27}) = 405
# 15 times 27 is 405.

This works across all three modes, auto-discovers tools on startup, and supports remote MCP servers over HTTPS with bearer token auth. Since AFM’s own math and factual recall are weak, pairing it with a calculator or search MCP server is genuinely useful rather than a novelty.

Privacy: The Actual Selling Point

Everything runs on the Neural Engine. No network calls happen during normal use – the only time apfel touches the internet is when you explicitly run apfel --update to check Homebrew for a newer version. Apple’s own documentation states it doesn’t collect prompts or responses from on-device inference, and doesn’t use user interactions to train its foundation models. apfel itself adds no analytics, telemetry, or crash reporting on top of that.

apfel vs Ollama and LM Studio

These aren’t really competing for the same job. If you want frontier-adjacent local performance for coding or long-context work, you still want Ollama or LM Studio running a proper 14B-70B parameter model with a real GPU behind it – see our guides on using local models with Claude Code and Cursor for that. What apfel gives you is the zero-friction option: no multi-gigabyte download, no model selection, no config file, for the moments when a fast, private, small model is all a task needs.

Factor	apfel	Ollama / LM Studio
Setup	One brew install, model already present	Install runtime + download model (GBs)
Model size	~3B, fixed	Choose from 1B to 100B+
Context window	4,096 tokens	Up to 128K+ depending on model
Best for	Quick shell tasks, text transforms, scripting glue	Coding assistants, long documents, agentic workflows
Hardware needed	Any Apple Silicon Mac	More RAM/VRAM for larger models

Where It Struggles

Worth being upfront about the limits: the 4K context window rules out feeding it large files or long conversations, and Apple’s own documentation flags math, factual recall, and complex code generation as weak points. The model is deliberately tuned to refuse rather than hallucinate, which is the right design choice for a system-level assistant but means it won’t bluff its way through something outside its comfort zone the way a larger model might.

Getting Started

1. Enable Apple Intelligence in System Settings if you haven’t already (this is what downloads the model)
2. Run brew install apfel
3. Try apfel "Hello, Mac" to confirm it works
4. Grab a few of the demo scripts from the project’s demo folder – cmd, oneliner, and explain are good starting points
5. If you want it wired into an existing local-LLM workflow, run apfel --serve and point any OpenAI-compatible client at localhost:11434

It’s a genuinely useful tool for the free AI that’s been sitting on your Mac the whole time. For anything heavier – long-context coding, agentic workflows, or the biggest local models you can fit on your hardware – our Claude Code local LLM guide and Ollama guide are the next stop.