Files
dotfiles/mcp-config
noconnorandClaude Opus 5 135112eee5 mcp-config/hermes: make the gateway's context limits actually reach the clients
Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.

Already in flight here (unchanged in intent):
  - sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
    install.sh records the checkout path so it can be found. That recording
    is load-bearing: without it sync.sh cannot find the repo, silently falls
    back to Gitea, and every local edit to opencode.gateway.json is ignored.
    Observed on this box — ~/.config/mcp-sync/source did not exist, the
    default ~/.dotfiles is not where the checkout lives, and the Gitea base
    still served models:{}.
  - the hermes custom-provider wiring.

Fixed on top:

opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.

hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.

NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.

Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
2026-09-11 16:38:03 -04:00
..

mcp-config — shared client config for laptops

Single source of truth for what the agents see: the covert-gpt model provider (LiteLLM at llm.covert-gpt.com) and the MCP servers, which run client-side. Users install once; their config then refreshes automatically on each new shell.

Two clients consume it: opencode (deployed to ~/.config/opencode/opencode.gateway.json) and Hermes (its mcp_servers: block in ~/.hermes/config.yaml is rendered from the same definitions — see "One definition, two clients" below).

MCP runs on the CLIENT, never on the gateway

The LiteLLM gateway hosts no MCP servers — deliberately. Every API worth exposing (Gitea, Proxmox, UniFi, NetBird, k8s) is user-scoped, so hosting it centrally means one shared token per server: every user acts as the same identity and attribution is gone.

LiteLLM can do per-request BYOK (env values written ${X-Header} are filled from the caller's headers — verified working for proxmox on v1.87.0), but only for servers that tolerate a credential-less spawn. LiteLLM builds its tool→server mapping from a headerless background listing, and gitea-mcp hard-exits without a token, so it never registers and its tool calls 404. The gateway could therefore host only some servers — and a split brain (some MCPs server-side, some client-side) is worse than either. So all MCP servers are defined here, each reading the user's own credentials from their own environment. The config is identical for everyone; only the secrets differ.

One definition, two clients

The servers are defined once, in opencode.gateway.json's mcp block. Hermes speaks a different schema, so sync.sh renders that one definition into mcp_servers: in ~/.hermes/config.yaml rather than keeping a second copy — two copies would drift, one pinned to gitea-mcp@0.0.10 and the other to whatever got typed. Add a server here once and both clients get it, within the same TTL. The translation is mechanical:

opencode Hermes
command: ["npx","-y","pkg@1.2.3"] (argv list) command: npx + args: ["-y","pkg@1.2.3"] — Hermes' command must be a string; a list stringifies and fails at exec
environment: {K: "{env:VAR}"} env: {K: "${VAR}"}
type: remote, url, headers url, headers
enabled: false entry omitted

Both placeholder forms are by-NAME references resolved at runtime, so no secret is written to either file. Hermes hands an MCP child only a filtered baseline environment, so every secret a server needs must be listed in env: — which is exactly what the rendered block does.

Servers you add to Hermes by hand are left alone: sync.sh records the names it rendered in ~/.config/mcp-sync/hermes-managed.json, so a server dropped from this repo is dropped from Hermes too, while anything not in that manifest is never touched. Rendering is skipped silently when Hermes isn't installed, and needs PyYAML (it warns if that's missing). Nothing needs adding to Hermes' platform_toolsets — MCP servers are auto-enabled by name; listing some would turn it into an allowlist, and no_mcp disables them all.

Credentials are referenced as {env:VAR} and are never written into the config. Each user exports them in ~/.zshrc.local:

MCP server env vars
gitea GITEA_KEY
proxmox PROXMOX_TOKEN_ID, PROXMOX_TOKEN_SECRET

Requires npx — the MCP servers are npm packages that opencode spawns as npx -y <pkg>@<version>. bootstrap.sh installs Node for this (node/). Without it every server dies with "command not found", which opencode surfaces only as a dead server — easy to misread as a broken config.

This repo lives in Gitea at public/dotfiles. (It is mirrored from the ai-k8s-iac repo under mcp-config/; edit it there or here and keep them in sync — the laptops pull from Gitea's main.)

How it works

  • opencode.gateway.json — the shared base config:

    • the covert-gpt model provider (LiteLLM's OpenAI-compatible endpoint at https://llm.covert-gpt.com/v1, models map left empty — it is filled by discovery at sync time), and
    • the MCP servers, each type: local, spawned by opencode as npx -y <pkg>@<pinned-version> on the user's own machine.

    Identical for every user: every credential (the LiteLLM key, the Gitea PAT, the Proxmox token) is injected via {env:VAR} and never written into the file. Versions are pinned — npx runs whatever the registry serves, and that process gets your tokens.

  • sync.sh — sourced from each user's shell rc. On a new shell (throttled ~1h) it reads the base config from the local checkout (see "Where the config comes from" below), queries the gateway's /v1/models with the user's key to auto-discover models (opencode can't list models from a custom provider itself, so the map is materialized client-side; if discovery fails the previous map is carried over), deploys the repo's skills (see below), deploys the config to ~/.config/opencode/opencode.gateway.json, and exports LITELLM_KEY (from ~/.config/mcp-sync/key) plus OPENCODE_CONFIG (pointing at the deployed file — opencode does NOT auto-load extra files in its config dir, so without this export the file is ignored). Silent + non-fatal: a Gitea hiccup never breaks the shell, last-good config stays in place.

  • ../skills/ — the shared skill folders, deployed to ~/.config/mcp-sync/skills and wired in through skills.paths in the config above. Skills are plain files, so — unlike models and MCP servers — there is nothing on the gateway to discover and the repo is the source of truth; the tree is copied from the checkout (or, with no checkout, fetched whole from the repo tarball). They deploy to mcp-sync's own directory rather than ~/.config/opencode/skills so the atomic replace can never delete a skill the user wrote by hand. See skills/README.md.

  • install.sh — one-time bootstrap. Installs sync.sh (from the checkout it was run out of, else the Gitea API), records that checkout in ~/.config/mcp-sync/source, stores the user's key (0600), and adds the rc hook.

opencode merges the OPENCODE_CONFIG file on top of the user's global ~/.config/opencode/opencode.json (and under any project opencode.json), so the synced config coexists with each user's own settings and any local-only MCP servers (e.g. a client-side kubernetes pointed at their own kubeconfig). Sync never touches those.

Where the config comes from

The local checkout is the source. bootstrap.sh already clones this repo over authenticated git, so the per-shell hook reads opencode.gateway.json and skills/ straight off disk. No HTTP, no anonymous read, works offline.

This is deliberate. The hook used to re-download both from Gitea's public /raw/ URL on every sync, which meant the repo had to be world-readable for a shell hook to work — publishing every internal hostname and the whole MCP server inventory as the price of a config refresh. Worse, Gitea answers an unauthorized /raw/ request with a 200 and an HTML login page, so curl -f reported success and wrote HTML over the config; the failure was silent and the config simply froze at its last good copy.

Resolution order, for both the config and the skills:

  1. MCP_SYNC_REPO / MCP_SYNC_RAW_URL / MCP_SYNC_SKILLS_SRC if set — the dev-checkout workflow in zsh/zshrc.local.example.
  2. The checkout recorded in ~/.config/mcp-sync/source by install.sh, falling back to $DOTFILES_DIR (~/.dotfiles).
  3. The Gitea API with $GITEA_KEY — for a box with no checkout at all. API endpoints, not the web /raw/ route, because the API returns an honest 401/403 that curl -f can actually detect.

To pick up a pushed change, the hook runs git pull --ff-only on the checkout — but only the bootstrap-managed one at $DOTFILES_DIR, and only when its tree is clean and on the default branch. A checkout you work in is read, never pulled out from under you.

Model discovery is unaffected: it still queries the gateway's /v1/models live on every sync, so a new model appears without touching the repo.

User setup (one time)

Each user needs a LiteLLM virtual key (for models) plus their own credentials for each MCP server. Run bootstrap.sh (it installs Node, which provides the npx the MCP servers need), then:

# From a checkout (what bootstrap.sh does — no network needed for the config):
./mcp-config/install.sh

# Or standalone on a box with no checkout:
curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config/install.sh | sh
# (prompts for the sk-... key; or pass it: ... | LITELLM_KEY=sk-... sh)
# If this repo is private, export GITEA_KEY first so the fallback can read it.

Then export your own MCP credentials in ~/.zshrc.local:

export GITEA_KEY=...                  # your Gitea PAT (repo + issue scopes)
export PROXMOX_TOKEN_ID=...           # e.g. alice@pve!mcp
export PROXMOX_TOKEN_SECRET=...       # the token UUID

Open a new terminal, run opencode, and the tools are there — acting as you.

Adding an MCP server (admin)

Add it to mcp.<name> in opencode.gateway.json as type: local, with a pinned package version and {env:VAR} refs for every secret, then push to main. Users pick it up within the sync TTL (or immediately on a new shell).

Two hard requirements:

  • It must be an npm package (opencode spawns it with npx).
  • Its credentials must be per-user, via {env:VAR}. Never commit a token, and never introduce a shared one — that is the whole reason MCP is not on the gateway.

Models are different: they live on the gateway (charts/inference/values.yaml → model_list) and are auto-discovered by the /v1/models query in sync.sh. No client change needed.

Adding a skill (admin)

Add a folder under skills/ and push to main — see skills/README.md. Verify with opencode debug skill.

"I added a model on the gateway and opencode doesn't see it"

Almost always a stale local config, not a gateway or key problem. The sync runs only at shell start and is throttled to MCP_SYNC_TTL (1h), so a new model can take up to an hour and a new terminal to appear.

Pull it immediately:

mcp-sync          # force a refresh now, ignoring the TTL; prints the model list

If the model still isn't listed, it isn't reaching your key — check the gateway, not the client:

curl -fsS -H "Authorization: Bearer $LITELLM_KEY" \
  https://llm.covert-gpt.com/v1/models | jq -r '.data[].id'

Whatever that prints is exactly what sync.sh will deploy. A model present here but missing in opencode is a sync problem; a model missing here is a LiteLLM model_list or key-scoping problem.

Notes / caveats

  • Local-server precedence: opencode merges with later-config-wins on key conflicts. If a user names a local server the same as a gateway one, theirs wins. Use distinct names.
  • Key rotation: re-run install.sh (or edit ~/.config/mcp-sync/key).
  • Other clients: Hermes is wired (see "One definition, two clients"). Claude Code / open-webui would each still need a parallel render step in sync.sh; not done yet.
  • Throttle: mcp-sync forces one refresh; MCP_SYNC_TTL=0 in the environment makes every new shell pull.
  • python3 is required for model discovery. Without it sync.sh deploys the base config verbatim, so the provider comes up with an EMPTY model list. It warns when this happens; bootstrap.sh installs python3.