CARL

Can't Afford Remote LLMs

AI Slop Coded LLM Runner, So You Can Code AI Slop Locally

A local coding model runner on your Apple Silicon Mac, for OpenCode and Pi.
One command starts the server and a live dashboard.

---

Install

You need: an Apple Silicon Mac, Homebrew, python3, and 4.6–23 GB of free disk space for each model.

brew install llama.cpp aria2 ansifilter zstd
./carl.sh download default            # the best model for this Mac (resumable, checksum-verified)
./carl.sh install                     # OpenCode + Pi into ~/.local (no sudo), connected to the server

• If you skip the brew install, ./carl.sh finds the missing tools and asks to install them.
• If you skip the download, ./carl.sh asks to download the best model for this Mac.
• ./carl.sh install asks one time which clients (OpenCode, Pi or both) and which options you want (the coder, the browser, web search, LSP). --yes takes the defaults.
• Your own OpenCode and Pi settings stay. The setup adds CARL next to them. It keeps a backup of each file that it changes (FILE.before-carl, the original, and one FILE.bak, the version before CARL's last change).

Another computer or a VM (macOS or Linux): start the server for that network (for example ./carl.sh --vm), then make the client package and run its setup there:

./carl.sh package                     # on the server Mac: dist/carl-client-VERSION-HOST.zip
unzip carl-client-*.zip && cd carl-client && ./setup      # on the other computer

• The zip holds the server's API key: keep it secret, and delete it after the copy.
• The server serves this Mac only (127.0.0.1) by default. Then package makes no zip and tells you how to change the network. More: Clients in the VM, Clients on another computer, The client package.

Use

./carl.sh                              # start the server and the dashboard

Then, in a second terminal, go to your project folder and start opencode or pi there. They work on the folder that you start them in.

cd ~/path/to/your/project
opencode

• The first time, CARL starts the model that Auto fit chooses for your Mac: the best stock model that holds two slots of 96K tokens (or the next setup of its order, down to 48K). ./carl.sh fit shows the choice and why.
• The next time, it starts llama.cpp with the settings that you saved.
• If a server runs already, the dashboard attaches to it.
• ./carl.sh -h shows all commands. ./carl.sh help COMMAND (or ./carl.sh COMMAND --help) shows the help for one command.

Models

Model	For
qwen3.6-35b-a3b	The default: fast MoE, everyday agent coding (32 GB+)
qwen3.8-27b	The dense 27B: hard code, slower (32 GB+)
IQ3 and Q3 builds	24 GB Macs
gemma-4-e4b, gemma-4-12b, saluki-27b	16 GB Macs: the E4B is fast, the 12B is better at code, Saluki (a 2-bit 27B) keeps a coder subagent
orcarouter-27b…, heretic-35b-a3b…	Abliterated (uncensored) builds
gemma-4-26b-a4b, gemma-4-31b	Google's larger Gemma 4 models (32 GB+)
qwen3.8-9b	A small distilled Qwen, for more subagents at the same time

• ./carl.sh models lists the catalogue and each .gguf in ~/models/gguf.
• ./carl.sh download hf:OWNER/REPO/FILE.gguf gets any GGUF from Hugging Face.
• ./carl.sh tune NAME measures the best settings for a model on your Mac.
• More: Choosing a model, Router mode (OpenCode and Pi change the model).

The dashboard

The live state of the server and the Mac: what the server does now, the slots, the speed, the memory, this Mac (memory pressure, swap, GPU, power, heat), the requests and the log.

• Tabs: Live, Connect (install the clients, make the client package, send their config), Requests, Log, Settings.
• Settings (tab 5): seven panels: Server, Models, Agents, Auto fit, Auto-tune, Router, Caching. Push [ or ] to change the panel.
• Levels: each section of a screen has its own level: collapsed, simple or full. Push Tab to select a section and L to change its level, or click its title. Push D to set every section to simple or full. The dashboard keeps your choice.
• Every tab scrolls as one page. The footer shows the keys of the screen that you see. ? shows all keys of that screen.
• More: The dashboard.

In OpenCode and Pi

• Model: /models in OpenCode, /model in Pi.
• Thinking: /variants (or ctrl+t) in OpenCode, the thinking level in Pi (Thinking).
• The coder: large tasks go to a coder subagent in the background. Type @coder TASK in OpenCode to ask for it.
• Tools: web search, LSP, a browser subagent, background subagents (Tools). Web search sends the queries to Exa: answer off in the setup (or ./carl.sh install --web-search off) to turn it off.

Plugins

The setup (./carl.sh install, or ./setup on another computer) adds CARL's plugins to OpenCode and Pi:

• Disk cache: fast starts; sessions come back after a restart (OpenCode, Pi).
• Coder subagent, in the background: large tasks go to a specialist coder while the main session stays free (OpenCode, Pi).
• Subagents panel and session switcher in OpenCode's sidebar and prompt box.
• Model check: a warning when the model that you select is not the one that the server runs (OpenCode, Pi).
• /carl: a control panel: turn each CARL piece on or off; the config sync (OpenCode, Pi).
• Speeds: the server's read and write speed in OpenCode's sidebar and Pi's status line, while the dashboard runs.

What each one does and how to turn it off: USERGUIDE.md, "CARL's plugins and extensions". How they work: reference/plugins.md.

Read more

• USERGUIDE.md: setup, daily use, models, thinking levels, the dashboard, troubleshooting.
• reference/: how it works, the caches, measurements, the files in this folder, and the design decisions.
• CHANGELOG.md: what changed in each release.