Run a local LLM in your Foundry game.

The runner, the network, and the bill

A runner is an app that serves a model on your machine. You install one, load a model into it, and Familiar sends your messages there instead of to a hosted provider. Your prompts, the world context that travels with them, and every reply all stay on your own hardware.

A few requests still reach the network when a world loads, and not one of them carries your game. Familiar refreshes the model list so the dropdowns stay current, and it checks your licence with Polar. That call sends the key and the product id, nothing else. The licence answer is cached for a week, so an offline evening plays straight through.

No provider key, and no per-token bill. That is not the same as free. You pay in the graphics card you already own and the power it draws while it thinks, and if you own the wrong hardware you pay in waiting.

Local setups vary far more than hosted ones, so this is where the rough edges surface first. If yours bites in a way this guide did not warn you about, the Discord is where I hear about it.

What it asks of your setup

Two things decide whether local play feels good: the model you load and the hardware under it. Check both before you download eight gigabytes of weights.

The model has to be able to call tools. Familiar works by calling them: rolling the dice, applying a condition, opening the right journal page. Hand the job to a model that cannot, and it will narrate an attack that never happened, which is worse than a refusal, because it looks like it worked. Tool skill collapses below about 9B and stays unreliable up to 14B, so the floor is a tool-trained model of 14B or more, at Q4_K_M quantisation or better.

That floor buys you tool calls, not fights. Switching combat on roughly triples the tool definitions the model has to hold, and turns a single turn into a chain of calls. The comfortable sizes for combat start well above the floor.

Hardware sets the speed. The request Familiar sends carries the core tool definitions plus your world context, roughly 8,000 to 10,000 tokens before you have typed a word. A 24 GB card works through that in seconds. On a CPU-only machine we measured 5 to 10 minutes per turn, and a whole exchange is capped at fifteen minutes, so a multi-step turn does not fit on CPU at all.

• A tool-trained model, 14B or larger, at Q4_K_M or better. Your runner can often tell you before you load it, and a non-thinking variant is the right pick when speed matters.
• For combat, 27B to 32B dense, or 30B to 35B active on a mixture-of-experts model, which wants about 24 GB of video memory. An 8 GB card holds about an 8B model, under the floor: single tool calls land, a fight is marginal at best.
• The runner set to a context window of at least 16K, or 32K if you run combat through the chat.
• A way of reaching Foundry that allows the call. The desktop app or a localhost tab is always fine; hosted games and Safari have limits, covered under Where your browser allows this below.

Familiar waits up to five minutes for the first token from a local server, where a hosted provider gets two. That is headroom for slow hardware, not a target.

Pick a runner

Familiar ships a preset for four runners, and each one leaves you a single setting to change on your side. Keep the runner and the model apart while you set this up: the runner is the plumbing, the model you load into it is the brain.

Point Familiar at your server

Everything for this path lives on one settings tab. In Foundry, open Module Settings, then Familiar, then Configure AI, and pick the Local AI tab at the end of the list.

Four fields, one switch, one test. You do this once.

1. 1

   Pick your runner

   The Preset dropdown lists Ollama, LM Studio, llama.cpp, and LiteLLM, plus an Other entry for anything else that speaks the OpenAI chat API.

2. 2

   Check the Base URL

   The preset fills it in up to /v1, and Familiar appends the rest itself. If your runner reported a different port when it started, correct it here.

3. 3

   Leave the API key empty

   A local runner normally wants no key at all. A gateway does: paste the key it expects, such as your LiteLLM master key.

4. 4

   Type the model id

   Type the exact id your server serves. Once the server answers, the field suggests what it found, but it is required either way.

5. 5

   Flip Use for chat on

   This is the switch that moves the built-in chat over to your server, and it is the only route there: the provider list on the Chat tab does not offer your server, and neither does asking your AI to change provider. The hosted provider on the Chat tab stays saved, so going back later is one flip.

6. 6

   Hit Test connection

   It sends a small real message to your server, so a connected result means the whole path works and not merely that the port is open.

Then open the chat and ask it to roll a d20. If the roll lands in Foundry's chat log, your model is calling tools and you are done.

Ollama: raise the context window first

Ollama on the same machine is close to zero configuration. Its own defaults already accept a Foundry page served from localhost, so there is nothing to allow and no key to set.

Skipping one setting is the likeliest reason a local setup disappoints: Ollama ignores the context size a client asks for, loads at 4K, and then truncates in silence. No error appears. The model forgets the tool definitions and half the conversation, and starts describing actions instead of taking them.

Set it in the environment that starts Ollama, restart Ollama, and check the result with ollama ps. The CONTEXT column reports what the loaded model got, which is the only number worth trusting here.

OLLAMA_CONTEXT_LENGTH=32768

ollama ps

LM Studio: turn CORS on before anything else

LM Studio serves the model from its Developer view, and one of its switches is off by default: Enable CORS. Without it the browser blocks the request before it leaves the page, and Familiar reports that it could not connect to the AI provider. Nothing ever reached LM Studio.

LM Studio warns you when you turn CORS on, and it is right to: with it on, any website your browser visits can call your server, so stop the server when you are not playing.

The context length is the second step, and LM Studio wants it set as you load the model, not after. Drag Context Length to at least 16,384 before you press Load, or 32,768 if you plan to run combat through the chat. The default is 4,096 or 8,192. Both fill up before your first message is read.

• Look for the tool-use badge, a small hammer, beside the loaded model. No hammer means it cannot call Familiar's tools, however well it writes.

Where your browser allows this

The chat calls your server straight from the Foundry page. That is what keeps this path simple, with no bridge and nothing in the middle. It also hands the decision to your browser: what is allowed depends on how you reach Foundry, and the grid below is its ruling.

Most setups clear this without a thought: the desktop app and a localhost tab are both fine. The rules only bite once Foundry and the model sit on different machines, or once https is in the path.

Familiar supports the field, not your backend. The Local AI tab and the connection it makes are ours to fix; the server, the model, and the machine under them are yours. If a model misbehaves through your runner but behaves elsewhere, start there.

When it does not work

Four failures cover almost everything people hit on a local setup, and each one points somewhere specific. Work down the table before you change the model.

Try it on one scene

Load a model, point the Local AI tab at it, and run one short scene: a conversation, a skill check, a single fight. That is enough to tell you whether your machine is up for a whole session.

If it drags, you have spent a download and learned something concrete about your hardware. If it flies, the whole table is yours.

• Install Familiar in Foundry
• How to add AI to Foundry VTTThe install and first-connection path, start to finish.
• Running local? Tell me how it went in the Discord