Sparehand

Docs › Local AI models on your workers

Local AI models on your workers

A worker with a graphics card (or a Mac with Apple silicon) can run AI models for every app on your hub. Sparehand installs the model server (Ollama) there, keeps it running, and gives you one address for all of them.

The easy way: open the dashboard, find the Local AI card on the Machines tab, and press Set up Local AI… (or Add a model…). Pick a machine and a model. Each model says in plain words what it's good for and how well it runs on that machine: Runs well, Slower (too big for its graphics card, so part of it runs on the processor), or Too big. One model per kind is marked Suggested, and Fast marks models that answer quickly. Downloads show their progress and can be stopped. Chat… then lets you talk to your models right in the dashboard, with nothing leaving your network.

From the command line:

sparehand ai install gaming-pc          # install Ollama there (or update it)
sparehand ai pull gaming-pc qwen3:8b    # download a model there
sparehand ai                            # which machines have which models, what's in memory
sparehand ai models                     # the model library: what each model is for, and how
                                        # well it runs on each of your workers

Then point any app that works with Ollama or with an OpenAI-compatible API at:

http://localhost:11434          # Ollama API
http://localhost:11434/v1       # OpenAI-compatible API (any API key)
curl localhost:11434/api/generate -d '{"model": "qwen3:8b", "prompt": "Hello", "stream": false}'

What happens

Which models

A graphics card's memory decides what fits: about 8 GB runs models up to roughly 8 billion parameters (quantised, as Ollama ships them), 12–16 GB up to 14 billion, 24 GB up to about 30 billion. A Mac uses its unified memory. Ollama's library lists the sizes.

Sparehand's library (lib/ai-models.json) does this sum for you. A model needs about its download size plus 15% and 1 GB for the conversation. It runs well when that fits in 92% of the graphics card's memory (on a Mac: 65% of its memory, leaving the rest for macOS and your apps), and runs, slower when it only fits with help from normal memory. Any other model from ollama.com/library can be installed by name.

Cloud models (free tiers)

Beta. The cloud providers work and are part of the licence, but they are newer than the core and may change between versions, as the providers' free tiers do. Nothing goes to any of them until you add its key; the privacy policy names each one.

When your machines aren't enough (a model too big for them, or they're all busy or asleep), the same address can also reach the free tiers of cloud AI services: Groq, Cerebras, Google Gemini, OpenRouter's free models, NVIDIA, SambaNova, Cloudflare Workers AI, GitHub Models, Cohere and Hugging Face. Mistral and xAI (Grok) are there too, marked Paid: they have no free tier, so what they answer is billed to your account with them, and auto never uses them unless you put them in "aiAuto". Nothing goes to any of them until you add a key.

Some providers aren't reachable from every country or network: Groq, for one, answers "Access denied" to some, from its website and its API alike. Cloud providers… then shows the provider as not working; use another.

Setting up: Local AI card → Add cloud models… (or Cloud providers…). Each provider has a Get a key ↗ link; make a key there, paste it, Save. The key stays in this computer's Keychain (on Linux in ~/.config/sparehand/ai-cloud-keys.json, readable only by you, or from SPAREHAND_AI_KEY_GROQ and the like in the dashboard's environment). The dashboard never shows it again. Cloudflare asks for two things, in two boxes: your account ID (the 32-character code in the Cloudflare dashboard's address, dash.cloudflare.com/…, or under Workers AI → Use REST API) and an API token (My Profile → API Tokens → Create Token → the Workers AI template). A token that may list your accounts can be given alone, and Sparehand finds the account. GitHub's is a fine-grained token with the Models (read) permission. From the terminal:

sparehand ai cloud                   # each provider: its key, models, today's use, whether it's resting
sparehand ai cloud key groq          # add or replace a key (typed without showing, or piped in)
sparehand ai cloud models groq       # the models to ask for
sparehand ai cloud remove groq       # forget the key (also: test PROVIDER, catalog)

Using them: a cloud model's name is provider/model, as the model list shows it:

curl localhost:11434/v1/chat/completions -d '{"model": "groq/openai/gpt-oss-120b", "messages": [{"role": "user", "content": "Hello"}]}'

bash ANTHROPIC_BASE_URL=http://localhost:11434 ANTHROPIC_API_KEY=local ANTHROPIC_MODEL=auto claude

A Claude model's name (what Claude Code asks for unless told otherwise) means auto, and a Haiku auto:fast; any other name is used as it is (groq/…, qwen3:14b). Answers are kept to 8,192 tokens each, which free tiers allow. Claude Code sends long prompts with many tools, so it works best with a large model on your own machine or a provider with a big allowance (Cerebras, Gemini); small free limits run out fast. It doesn't know these models' context windows and says so once; see its model-config docs to set one. A member's API key works as ANTHROPIC_API_KEY for the models on the machines.

Settings

In ~/.config/sparehand/app.json: "aiPort": 11434 to use another port, "ai": false to turn the hub off (the per-machine ports 21434… still work), "aiCloud": false to switch the cloud models off altogether. For the cloud models also "aiEndpoints", "aiAuto", "aiFusion", "aiCloudCatalogUrl", "aiCache", "aiCompress", "aiCloudUsdDay", "aiCloudLimits", "aiCloudMembers", "aiCloudMembersPaid" and "aiMemberRpm", described above.

Removing

sparehand ai rm gaming-pc qwen3:8b     # one model
sparehand ai uninstall gaming-pc       # Ollama itself (downloaded models stay where they are)

The Helper

The dashboard's Help button opens the Helper: ask in your own words ("how do I add my gaming PC?", "why do my tests run on this Mac?") and it answers with the steps, using what Sparehand knows about your machines right now, and offers buttons for the next step. It runs on your local models (it picks a small, quick one; "helperModel": "NAME" in app.json chooses another, and "helper": false switches it off). With no model yet it offers to set one up (about 4 GB in all, on the best machine you have) and meanwhile answers from the guide. The model is optional: switch off Use a local AI model in the Helper (or set "helperModel": false in app.json) and it answers from the guide alone, showing the matching sections. It only helps with Sparehand; for anything else it points to the chat. It only suggests: every action is a button you press, and its one check (sparehand doctor NAME) only reads.