Run a local AI model in your Foundry game.

What local actually means here

A runner is an app that serves a model on your machine. You install one, load a model into it, and Familiar sends your messages there instead of to a hosted provider. Your prompts, the world context that travels with them, and every reply stay on your own hardware.

A few requests still reach the network when a world loads, and not one of them carries your game. Familiar refreshes the model list so the dropdowns stay current, and it checks your licence with Polar, which sends the key and the product id and nothing else. The licence answer is cached for a week, so an offline evening plays straight through.

No provider key, and no per-token bill. That is not the same as free. You pay in the graphics card you already own and the power it draws while it thinks, and if you own the wrong hardware you pay in waiting.

This path is new. If it runs well on your machine, or bites in a way this guide did not warn you about, say so in the Discord. Local setups vary far more than hosted ones, and that is where the rough edges surface first.

What it asks of your machine

Two things decide whether local play feels good: the model you load and the hardware under it. Neither is something Familiar can fix from the module side, so it is worth being blunt about both before you download eight gigabytes of weights.

The model has to be able to call tools. Familiar works by calling them: rolling the dice, applying a condition, opening the right journal page. A model that cannot will narrate an attack that never happened, which is worse than a refusal, because it looks like it worked. Tool skill falls off sharply in smaller models, so the floor is a tool-trained model of 14B parameters or more, at Q4_K_M quantisation or better.

That floor buys you tool calls, not fights. Switching combat on roughly triples the tool definitions the model has to hold, and turns a single turn into a chain of calls, so the comfortable sizes for combat start well above it.

Hardware sets the speed. The request Familiar sends carries the core tool definitions plus your world context, roughly 8,000 to 10,000 tokens before you have typed a word. A 24 GB card works through that in seconds. On a CPU-only machine we measured 5 to 10 minutes per turn, and a multi-step turn does not fit inside that at all.

• A tool-trained model, 14B parameters or larger, at Q4_K_M or better. Under 9B, tool calling falls off a cliff.
• For combat, 27B to 32B dense, or 30B to 35B active on a mixture-of-experts model, which wants about 24 GB of video memory. An 8 GB card sits at the floor, where running a fight is marginal at best.
• The runner set to a context window of at least 16K, or 32K if you run combat through the chat.

Familiar waits up to five minutes for the first token from a local server, where a hosted provider gets two. That is headroom for slow hardware, not a target. If your turns land anywhere near it, the answer is a faster machine or a hosted model, not a setting.

Point Familiar at your server

Everything for this path lives on one settings tab. In Foundry, open Module Settings, then Familiar, then Configure AI, and pick the Local AI tab at the end of the list.

Four fields, one switch, one test. You do this once.

1. 1

   Pick your runner

   The Preset dropdown lists Ollama, LM Studio, llama.cpp, and LiteLLM, plus an Other entry for anything else that speaks the OpenAI chat API.

2. 2

   Check the Base URL

   The preset fills it in up to /v1, and Familiar appends the rest itself. If your runner reported a different port when it started, correct it here.

3. 3

   Leave the API key empty

   A local runner normally wants no key at all. A gateway does: paste the key it expects, such as your LiteLLM master key.

4. 4

   Type the model id

   Type the exact id your server serves. Once the server answers, the field suggests what it found, but it is required either way.

5. 5

   Flip Use for chat on

   This is the switch that moves the built-in chat over to your server. The hosted provider on the Chat tab stays saved, so going back later is one flip.

6. 6

   Hit Test connection

   It sends a small real message to your server, so a connected result means the whole path works and not merely that the port is open.

Then open the chat and ask it to roll a d20. If the roll lands in Foundry's chat log, your model is calling tools and you are done.

Pick a runner

Familiar ships a preset for four runners, and each one leaves you a single setting to change on your side. Keep the two apart while you set this up: the runner is the plumbing, the model you load into it is the brain.

Ollama: raise the context window first

Ollama on the same machine is close to zero configuration. Its own defaults already accept a Foundry page served from localhost, so there is nothing to allow and no key to set.

One thing does need setting, and skipping it is the likeliest reason a local setup disappoints. Ollama ignores the context size a client asks for, loads at 4K, and then truncates in silence. No error appears. The model simply forgets the tool definitions and half the conversation, and starts describing actions instead of taking them.

Set it in the environment that starts Ollama, restart Ollama, and check the result with ollama ps. The CONTEXT column reports what the loaded model actually got, which is the only number worth trusting here.

OLLAMA_CONTEXT_LENGTH=32768

ollama ps

LM Studio: turn CORS on before anything else

LM Studio serves the model from its Developer view, and one switch there is off by default: Enable CORS. Without it the browser blocks the request before it leaves the page, and Familiar reports that it could not connect to the AI provider. Nothing ever reached LM Studio.

Heed LM Studio's own warning when you flip that switch. With CORS on, any website your browser visits can call your server, so stop the server when you are not playing.

The context length is the second step, and you set it as you load the model rather than afterwards. Drag Context Length to at least 16,384 before you press Load, or 32,768 if you plan to run combat through the chat. Its own default lands at 4,096 or 8,192, and both fill up before your first message is read.

• Look for the tool-use badge, a small hammer, beside the loaded model. No hammer means it cannot call Familiar's tools, however well it writes.

Where your browser allows this

The chat calls your server straight from the Foundry page. That is what keeps this path simple, with no bridge and nothing in the middle, and it also hands the decision to your browser. What is allowed depends on how you reach Foundry, not on anything in Familiar.

In the Foundry desktop app the page and the model server both sit on your machine, so none of the rules below come into play and no permission prompt appears.

Familiar supports the field, not your backend. The Local AI tab and the connection it makes are ours to fix; the server, the model, and the machine under them are yours. If a model misbehaves through your runner but behaves elsewhere, start there.

When it does not work

Four failures cover almost everything people hit on a local setup, and each one points somewhere specific. Work down the table before you change the model.

Try it on one scene

Load a model, point the Local AI tab at it, and run one short scene: a conversation, a skill check, a single fight. That is enough to tell you whether your machine is up for a whole session.

If it drags, you have spent a download and learned something concrete about your hardware. If it flies, you have a table that runs entirely on your own.

• Install Familiar in Foundry
• How to add AI to Foundry VTTThe install and first-connection path, start to finish.
• Running local? Tell me how it went in the Discord