PerchAI
Using Perch

Local models

Run Perch on a model on your own computer through Ollama, LM Studio or a llama.cpp server. Turns on it are not metered, and the address never leaves your machine.

Perch can run on a model served from your own computer instead of a hosted one. The model call is made on your machine, so the prompt never reaches Perch's servers and the turn is not metered against your allowance.

The short version

In Perch Desktop, Settings → Models, under Local models. In the CLI, perch byolm. Open to every plan. Not available in the browser, for the reason below.

Why it is desktop and CLI only

A local model lives at an address on your computer, usually localhost. Perch's servers cannot reach it, and neither can a browser tab. Perch Desktop and the CLI run the model call in their own process on your machine, which is the only place that address can be reached from. Keeping it there also means the address and any key stay on the machine: they are never saved to your account.

Setting one up in Perch Desktop

Open Settings → Models and look under Local models. Perch looks for Ollama and LM Studio on the computer and lists the models each one has.

  • A running app with models lists them, with the context window each one holds and whether it can call tools. Pick one and choose Use.
  • An app that is installed but not running shows as not running, so you start it and check again.
  • Neither one found gives you links to both. Download a model inside the app first. LM Studio also needs its local server turned on.
  • Anything else goes in by hand under A model at another address: a llama.cpp server, or a model app on another computer on your network. You give the address, the model name, and a key if it needs one.

The connection test

Saving a model runs one small request against it, and it is worth taking seriously. A local model that cannot call tools looks like it works and then stalls on the first file read, so Perch checks for that up front rather than letting you find out mid-task.

The test reports what went wrong in plain terms:

ProblemWhat it means
The window is too smallPerch's prompt does not leave the model room to work. Use a larger context window.
It answered, but did not call a toolPerch reads files and runs commands through tool calls, so it cannot work with this model. On a llama.cpp server, start it with --jinja: without that the server does not pass tool calls through.
It did not finish in timeThe app may still be loading the model. A first load can take a minute or more.
Not answeringThe app serving the model is not running.

A passing test also records the speed: output tokens a second, and how long the first answer took. If only part of the model fits on the graphics card, or it is running on the processor instead, Perch says so, because that is the usual reason a local model feels slow. A smaller model is the fix.

Two settings worth knowing

  • Context window. How much of the chat and your files the model reads at once. Bigger needs more memory. The default is 32K. Some servers set their own window, and then you change it where you start the server.
  • Every tool. Off, the model gets the short list local models handle well: files, search and commands. On, it gets every tool and the full prompt, which leaves no room to work at 32K, so the default starts one size up at 64K.

In the CLI

perch byolm detect                                    # find local model apps that are running
perch byolm use llama3.1                              # pick one of their models, no URL needed
perch byolm set http://localhost:11434/v1 llama3.1   # or register one by address
perch byolm test                                      # probe that it responds and calls tools
perch byolm status                                    # show the current local model
perch byolm off                                       # pause it; perch byolm on resumes
perch byolm clear                                     # remove it and go back to Roost

--key on set is optional, and most local runtimes do not need one: Perch omits the authorization header entirely rather than sending a placeholder. --all-tools on use or set is the same switch as Every tool above.

What it does and does not change

  • Not metered. Turns on your local model do not count against your plan's allowances. Your included allowance stays untouched while you use it.
  • No silent fallback. A turn on a local model never quietly switches to a hosted model. If the model cannot do the job, the turn says so.
  • Chats are still saved to your account, like any other chat, so they are there on your other devices.
  • The rest of Perch works as usual. The workspace, memory, retrieval and the tools behave the same way.