flowchart LR
classDef planned stroke-dasharray: 5 5
piT["pi, opencode or Hermes on the Mac<br/>in a terminal or Orca"]
ui["web UI<br/>on the Mac or the phone"]
piA["pi as agent<br/>in tmux on the Spark"]
cc["Claude Code<br/>anywhere"]
app["an app with its own key<br/>Phase 3"]:::planned
fam["family and friends<br/>over a shared Tailscale node · Phase 6"]:::planned
guest["a guest at an event<br/>venue LAN, a key each · Phase 6"]:::planned
tunnel["SSH tunnel<br/>over Tailscale, or the LAN alias"]
guestui["guest web UI<br/>events only · Phase 6"]:::planned
serve["tailscale serve, 443<br/>Tailscale only"]
owui["Open WebUI"]
searx["SearXNG"]
swap["llama-swap, 127.0.0.1:9100<br/>a key on every request"]
litellm["LiteLLM on a LAN port<br/>Phase 3"]:::planned
anthropic(("Anthropic"))
piT --> tunnel --> swap
ui --> serve --> owui --> swap
owui --> searx
piA --> swap
cc --> anthropic
app -.-> litellm -.-> swap
fam -.-> serve
fam -.-> litellm
guest -.-> guestui -.-> litellm
guest -.-> litellm
Architecture
Where everything sits — the Mac, the phone, the Spark and the Synology — and the paths a request takes. Built parts are solid; planned parts are dashed, with their phase.
The overview first, then one picture of where everything sits, then the three paths a request can take. Every scenario follows one of those paths, so the table at the end names the path for each, and each scenario’s page says what happens along it. Solid parts are built and running; dashed parts are planned, labelled with the phase that brings them. The plan holds the reasons.
The overview
The tools, and how a request reaches a model. The home page shows the same diagram, from one file, website/_overview.qmd.
%% The overview: one file, included by index.qmd and architecture.qmd, never copied.
%% It changes with the Architecture page's diagrams (CLAUDE.md, Docs must be true).
flowchart TB
classDef planned stroke-dasharray: 5 5
subgraph mine["my Mac and phone"]
harness["pi, opencode, Hermes<br/>in a terminal or Orca"]
browser["web UI<br/>in a browser or as an app"]
end
later["my apps · Phase 3<br/>family and guests · Phase 6"]:::planned
claude["Claude Code and<br/>Claude Desktop<br/>wherever they run"]
anthropic(("Anthropic"))
subgraph spark["brightroar · the DGX Spark"]
owui["Open WebUI<br/>with SearXNG for search"]
agent["pi as agent<br/>in tmux"]
litellm["LiteLLM<br/>a key per app · Phase 3"]:::planned
swap["llama-swap<br/>starts the model<br/>a request names"]
gate["spark-gate<br/>takes both over · Phase 2"]:::planned
guard["fit check, memory brake<br/>a model loads only if it fits"]
engines["llama.cpp, whisper.cpp<br/>models for chat, coding,<br/>embeddings, speech"]
vllm["vLLM or SGLang<br/>Phase 5"]:::planned
end
nas["Synology<br/>alerts · Phase 2<br/>backups · Phase 4"]:::planned
harness -- "SSH tunnel, over Tailscale<br/>or the LAN" --> swap
browser -- "HTTPS over Tailscale" --> owui
later -.-> litellm
claude -- "never through the Spark" --> anthropic
owui --> swap
agent --> swap
litellm -.-> swap
swap --> guard
gate -.-> guard
guard --> engines
guard -.-> vllm
gate -. "alerts" .-> nas
Where everything sits
flowchart TB
classDef planned stroke-dasharray: 5 5
subgraph mac["heartsbane · the Mac"]
direction LR
orca["Orca"]
claude_mac["Claude Code,<br/>Claude Desktop"]
pi_mac["pi"]
oc_mac["opencode, Hermes"]
webapp["web UI,<br/>Safari web app"]
term["terminal"]
end
subgraph phone["Android phone"]
pwa["web UI,<br/>home-screen app"]
end
anthropic(("Anthropic"))
apps["my pipelines and<br/>homelab apps · Phase 3"]:::planned
family["family and friends<br/>shared Tailscale node · Phase 6"]:::planned
guests["event guests<br/>venue LAN only · Phase 6"]:::planned
subgraph spark["brightroar · the Spark"]
tss["tailscale serve,<br/>HTTPS 443"]
guestui["guest web UI<br/>events only · Phase 6"]:::planned
owui["Open WebUI · 127.0.0.1:3000<br/>root's Compose<br/>capabilities dropped"]
searx["SearXNG · 127.0.0.1:8888<br/>root's Compose<br/>capabilities dropped"]
agent_tmux["pi and Claude Code<br/>as agent, in tmux"]
litellm["LiteLLM, a key per app<br/>Phase 3"]:::planned
swap["llama-swap · 127.0.0.1:9100<br/>as spark, keys"]
gate["spark-gate<br/>Phase 2"]:::planned
brake["memory brake<br/>as spark"]
engines["engines · 127.0.0.1:5800 and up<br/>Gemma, embeddings, whisper, coder"]
vllm["vLLM or SGLang, for NVFP4<br/>Phase 5"]:::planned
end
subgraph nas["Synology"]
ntfy["ntfy and the watchdog<br/>Phase 2"]:::planned
shares["shares and backups<br/>Phase 4"]:::planned
end
orca -- starts --> claude_mac
orca -- starts --> pi_mac
orca -- starts --> oc_mac
claude_mac --> anthropic
webapp -- "HTTPS over Tailscale" --> tss
pwa -- "HTTPS over Tailscale" --> tss
pi_mac -- "SSH tunnel, make tunnel" --> swap
oc_mac -- "same tunnel" --> swap
term -- "ssh brightroar-agent" --> agent_tmux
tss --> owui
owui --> searx
owui --> swap
agent_tmux --> swap
agent_tmux -- "its Claude Code" --> anthropic
apps -.-> litellm
family -.-> tss
family -.-> litellm
guests -.-> guestui
guests -.-> litellm
guestui -.-> litellm
litellm -.-> swap
swap -- "spark launch" --> engines
swap -.-> vllm
brake -- "unloads through llama-swap" --> engines
gate -.-> engines
gate -.-> ntfy
agent_tmux -.-> shares
- Everything on the Spark listens on 127.0.0.1, except SSH and
tailscale serve’s port 443. The tailnet’s rules let my own devices reach the Spark on those two ports only, and ufw allows SSH alone from the LAN. - Three accounts on the Spark. Mine, the admin;
spark, which runs llama-swap, the engines and the brake; andagent, for coding agents in tmux, with no sudo, no docker and keys of its own. Root runs the web services from its own copies of their units. - Claude never uses the Spark’s models. Claude Code and Claude Desktop talk straight to Anthropic, wherever they run; Orca only starts them. The Spark is a separate mode.
Reaching the Spark
pi on the Mac reaches llama-swap only while make tunnel runs, since nothing on the Spark listens beyond 127.0.0.1 but SSH. The web UI needs Tailscale: on another tailnet, over WireGuard, the LAN alias still carries SSH and so pi’s tunnel, but the web UI waits. agent’s pi needs no tunnel. opencode and Hermes on the Mac go through the same tunnel, with the same key (since 2026-10-14, make clients). From Phase 3, apps reach the models through LiteLLM, each with a key of its own. From Phase 6, family and friends reach the web UI and LiteLLM over a shared Tailscale node that shows them only the Spark, and guests at an event reach LiteLLM and a guest web UI of their own, on the venue’s network only.
Loading a model
flowchart TD
classDef planned stroke-dasharray: 5 5
req["a request names a model"] --> loaded{"already loaded?"}
loaded -- yes --> answer["the engine answers"]
loaded -- no --> launch["llama-swap starts it<br/>through spark launch"]
launch --> held{"is the brake holding<br/>new loads?"}
held -- yes --> refuse["refused, with its reason<br/>make status shows it"]
held -- no --> fits{"does its footprint fit in<br/>free memory, less the reserve?"}
fits -- no --> refuse
fits -- yes --> load["the engine loads"] --> answer
low["free memory falls<br/>toward the freeze band"] --> brake["the brake warns, then holds new loads<br/>and unloads models, on-demand ones first"]
brake --> held
gate["Phase 2, the gate: a live fit check, one load at a time,<br/>idle unload, session pins, make-room, notifications"]:::planned
gate -.-> launch
A model loads only if it fits: nothing is evicted to make room, and nothing is swapped for a smaller model. Free memory is the smaller of MemAvailable and what the GPU can allocate, and the reserve stays free, because overcommitting a GB10 can freeze it without an out-of-memory kill.
Apps and speech (Phase 3, planned)
flowchart LR
classDef planned stroke-dasharray: 5 5
app["an app or pipeline<br/>with its own key"]:::planned --> litellm["LiteLLM: limits and a wait per key,<br/>refusals inline, the real model named"]:::planned
litellm -.-> pg[("Postgres<br/>keys and usage")]:::planned
litellm -.-> swap["llama-swap"]
swap --> chat["chat and embeddings"]
swap --> stt["whisper, interactive"]
swap -.-> batch["whisper, batch"]:::planned
swap -.-> diar["speaker labels"]:::planned
Today the web UI and pi reach llama-swap directly with llama-swap’s own keys. From Phase 3, my own pipelines and homelab apps come in through LiteLLM instead, each with a key of its own; it never logs what they send.
When something breaks, and upkeep
These follow runbooks rather than a request path: a freeze while I’m away (S13), the NAS down (S15), a rebuild after a factory reset (S18), and upgrade day (S23; Updates).