noconnorandClaude Opus 5
d42d0b29ff
OpenAI's /v1/models schema carries no context window, so a client that only
lists models cannot know how large a request it may send, and oversizing one
is a hard ContextWindowExceededError 400 — vLLM counts prompt + max_tokens
against its window. opencode already read the window from /v1/model/info.
Hermes and pi never consult that endpoint: they take it from static config,
which was hand-copied and drifted. Hermes named a model the gateway had
stopped serving while its replacement (the one model with a SMALLER window)
had no entry; pi's reserve was half its maxTokens. Both fail only once a
session grows, which is why it looked like the model or the GPU.
This renders one /v1/model/info response into all three clients. It is not a
plugin per harness and needs no upstream change — it extends the renderer that
already translates the MCP servers into Hermes' schema, with the same
manifest discipline.
SAME SOURCE, THREE DIFFERENT NUMBERS, deliberately — each client means
something different by "context":
opencode limit.context = window - 1024 reserves limit.output and targets
context - output, so needs headroom
Hermes context_length = window omits max_tokens entirely; vLLM
bounds output by what is left
pi maxTokens <= reserveTokens-2048 compacts at window - reserveTokens
(compaction.js shouldCompact) then
requests maxTokens
pi's maxTokens is derived FROM its reserve rather than the reserve being raised
to fit: reserveTokens is one global setting shared by every provider pi knows,
so raising it for this gateway would shrink everyone's usable context.
Safety properties, each tested in a sandbox against copies of the real
configs before touching live ones (11 cases):
- Ownership is per FIELD. A value is retracted only if this set it and the
gateway stopped declaring it. Hand-added models and fields are never
touched; a model removed upstream loses only what was rendered.
- Vision is retracted, not just added. The chart states that supports_vision
is what makes clients send image parts, so if a backend falls back to
language-only, the `image` input this added is taken back — but a user's own
`image` on an entry is left alone.
- Raw /v1/model/info is cached on each successful fetch, because refresh is
throttled and a throttled shell never fetches. A bad fetch (403 body, HTML
error page, empty list, rows without model_name) never replaces a good cache.
- Never invents a window: claude-* declares none and is skipped. A gateway
that suddenly declares no windows is treated as broken, not empty.
- One output cap (MCP_SYNC_OUTPUT_RESERVE_CAP) is read by opencode and pi, so
the clients cannot drift apart on it. Parses under dash as well as bash,
since install.sh sources this under /bin/sh.
First live run found a bug nobody knew about: pi's Claude entries carried
maxTokens 64000 against a 34816 reserve, so claude-haiku-4-5 sent
(200000 - 34816) + 64000 = 229184 tokens at a 200000 window — still broken
after the earlier manual reserve fix, which only covered 32768. Now 32768, and
every served model satisfies the invariant. It also gave Hermes limits for the
Claude routes it had none for.
hermes/gateway.yaml: the static map is marked as a first-boot seed that the
next sync overwrites, so nobody hand-edits it believing it is authoritative.
The source of truth is model_info.max_input_tokens in ai-k8s-iac's
charts/inference/values.yaml. README documents all of the above.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
285 lines
14 KiB
Markdown
285 lines
14 KiB
Markdown
# mcp-config — shared client config for laptops
|
||
|
||
Single source of truth for what the agents see: the **covert-gpt model provider**
|
||
(LiteLLM at `llm.covert-gpt.com`) and the **MCP servers**, which run
|
||
**client-side**. Users install once; their config then refreshes automatically
|
||
on each new shell.
|
||
|
||
Two clients consume it: **opencode** (deployed to
|
||
`~/.config/opencode/opencode.gateway.json`) and **Hermes** (its `mcp_servers:`
|
||
block in `~/.hermes/config.yaml` is rendered from the same definitions — see
|
||
"One definition, two clients" below).
|
||
|
||
## MCP runs on the CLIENT, never on the gateway
|
||
|
||
The LiteLLM gateway hosts **no MCP servers** — deliberately. Every API worth
|
||
exposing (Gitea, Proxmox, UniFi, NetBird, k8s) is **user-scoped**, so hosting it
|
||
centrally means one shared token per server: every user acts as the same
|
||
identity and attribution is gone.
|
||
|
||
LiteLLM *can* do per-request BYOK (env values written `${X-Header}` are filled
|
||
from the caller's headers — verified working for proxmox on v1.87.0), but only
|
||
for servers that tolerate a credential-less spawn. LiteLLM builds its
|
||
tool→server mapping from a **headerless** background listing, and `gitea-mcp`
|
||
hard-exits without a token, so it never registers and its tool calls 404. The
|
||
gateway could therefore host only *some* servers — and a split brain (some MCPs
|
||
server-side, some client-side) is worse than either. So **all** MCP servers are
|
||
defined here, each reading the user's **own** credentials from their own
|
||
environment. The config is identical for everyone; only the secrets differ.
|
||
|
||
## One definition, two clients
|
||
|
||
The servers are defined **once**, in `opencode.gateway.json`'s `mcp` block.
|
||
Hermes speaks a different schema, so `sync.sh` *renders* that one definition into
|
||
`mcp_servers:` in `~/.hermes/config.yaml` rather than keeping a second copy — two
|
||
copies would drift, one pinned to `gitea-mcp@0.0.10` and the other to whatever
|
||
got typed. Add a server here once and **both** clients get it, within the same
|
||
TTL. The translation is mechanical:
|
||
|
||
| opencode | Hermes |
|
||
|---|---|
|
||
| `command: ["npx","-y","pkg@1.2.3"]` (argv list) | `command: npx` + `args: ["-y","pkg@1.2.3"]` — Hermes' `command` must be a **string**; a list stringifies and fails at exec |
|
||
| `environment: {K: "{env:VAR}"}` | `env: {K: "${VAR}"}` |
|
||
| `type: remote`, `url`, `headers` | `url`, `headers` |
|
||
| `enabled: false` | entry omitted |
|
||
|
||
Both placeholder forms are by-NAME references resolved at runtime, so no secret
|
||
is written to either file. Hermes hands an MCP child only a filtered baseline
|
||
environment, so every secret a server needs **must** be listed in `env:` — which
|
||
is exactly what the rendered block does.
|
||
|
||
Servers you add to Hermes by hand are left alone: `sync.sh` records the names it
|
||
rendered in `~/.config/mcp-sync/hermes-managed.json`, so a server dropped from
|
||
this repo is dropped from Hermes too, while anything not in that manifest is
|
||
never touched. Rendering is skipped silently when Hermes isn't installed, and
|
||
needs PyYAML (it warns if that's missing). Nothing needs adding to Hermes'
|
||
`platform_toolsets` — MCP servers are auto-enabled by name; listing *some*
|
||
would turn it into an allowlist, and `no_mcp` disables them all.
|
||
|
||
### Context limits — discovered, not hand-copied
|
||
|
||
OpenAI's `/v1/models` carries **no context window**, so a client that only
|
||
lists models cannot know how big a request it may send, and oversizing one is a
|
||
hard `ContextWindowExceededError` 400 (vLLM counts `prompt + max_tokens` against
|
||
its window). The window lives in LiteLLM's `/v1/model/info`, and `sync.sh`
|
||
renders it into every client from that one response:
|
||
|
||
| client | reads | what `sync.sh` writes | why that number |
|
||
|---|---|---|---|
|
||
| opencode | `opencode.gateway.json` | `limit.context = window − 1024`, `limit.output = min(out, cap)` | reserves `output` and targets `context − output`, so needs headroom |
|
||
| Hermes | `~/.hermes/config.yaml` | `context_length = window` | omits `max_tokens`; vLLM bounds output by what is left, so no margin |
|
||
| pi | `~/.pi/agent/models.json` | `contextWindow = window`, `maxTokens = min(out, cap, reserveTokens − 2048)` | compacts at `window − reserveTokens` then asks for `maxTokens`, so `maxTokens` must fit inside the reserve |
|
||
|
||
Same source, three different numbers — each client means something different by
|
||
"context". Do not "simplify" them into one.
|
||
|
||
**The single source of truth is `model_info.max_input_tokens` in ai-k8s-iac's
|
||
`charts/inference/values.yaml`.** A wrong window is fixed there and propagates on
|
||
the next sync. Editing a rendered value in a client config is overwritten.
|
||
|
||
Requirements and behaviour:
|
||
|
||
- **The key needs `/v1/model/info`.** A virtual key scoped to `llm_api_routes`
|
||
gets a 403 and every client silently falls back to its own defaults. Grant it
|
||
in the LiteLLM UI (Virtual Keys → key → Allowed Routes). The match is on the
|
||
exact path: `/v1/model/info` does not also grant `/model/info`.
|
||
- **Cached** at `~/.config/mcp-sync/model-info.json` on each successful fetch, so
|
||
a throttled shell still renders. A bad fetch (403, error page, empty list)
|
||
never replaces a good cache.
|
||
- **Ownership is tracked, per field.** `hermes-models-managed.json` and
|
||
`pi-managed.json` record what was set, so a value is retracted only if `sync.sh`
|
||
set it and the gateway stopped declaring it. That includes vision: if a backend
|
||
falls back to language-only and `supports_vision` flips off, the `image` input
|
||
`sync.sh` added is taken back rather than left to send image parts the backend
|
||
rejects. Models and fields you add by hand are never touched.
|
||
- **Never invents a window.** A wildcard route like `claude-*` declares none, so
|
||
it is skipped. A gateway that suddenly declares no windows at all is treated as
|
||
broken, not empty — nothing is retracted.
|
||
- **`pi`'s `maxTokens` is derived from its reserve**, not the other way round:
|
||
`reserveTokens` is one global setting shared by every provider pi knows, so
|
||
raising it to suit this gateway would shrink everyone's context. Raise
|
||
`compaction.reserveTokens` in pi's settings to buy back output. The shared
|
||
output cap is `MCP_SYNC_OUTPUT_RESERVE_CAP` (default 32768), read by opencode
|
||
and pi alike.
|
||
- Opt out per client with `MCP_SYNC_HERMES_CONFIG=` or `MCP_SYNC_PI_MODELS=`.
|
||
|
||
Credentials are referenced as `{env:VAR}` and are **never** written into the
|
||
config. Each user exports them in `~/.zshrc.local`:
|
||
|
||
| MCP server | env vars |
|
||
|---|---|
|
||
| `gitea` | `GITEA_KEY` |
|
||
| `proxmox` | `PROXMOX_TOKEN_ID`, `PROXMOX_TOKEN_SECRET` |
|
||
|
||
**Requires `npx`** — the MCP servers are npm packages that opencode spawns as
|
||
`npx -y <pkg>@<version>`. `bootstrap.sh` installs Node for this (`node/`).
|
||
Without it every server dies with "command not found", which opencode surfaces
|
||
only as a dead server — easy to misread as a broken config.
|
||
|
||
This repo lives in Gitea at `public/dotfiles`. (It is mirrored from the
|
||
`ai-k8s-iac` repo under `mcp-config/`; edit it there or here and keep them in
|
||
sync — the laptops pull from Gitea's `main`.)
|
||
|
||
## How it works
|
||
|
||
- `opencode.gateway.json` — the shared base config:
|
||
- the `covert-gpt` model provider (LiteLLM's OpenAI-compatible endpoint at
|
||
`https://llm.covert-gpt.com/v1`, models map left empty — it is filled by
|
||
discovery at sync time), and
|
||
- the **MCP servers**, each `type: local`, spawned by opencode as
|
||
`npx -y <pkg>@<pinned-version>` on the user's own machine.
|
||
|
||
**Identical for every user**: every credential (the LiteLLM key, the Gitea
|
||
PAT, the Proxmox token) is injected via `{env:VAR}` and never written into
|
||
the file. Versions are pinned — `npx` runs whatever the registry serves, and
|
||
that process gets your tokens.
|
||
- `sync.sh` — sourced from each user's shell rc. On a new shell (throttled
|
||
~1h) it reads the base config **from the local checkout** (see "Where the
|
||
config comes from" below), queries the gateway's
|
||
`/v1/models` with the user's key to **auto-discover models** (opencode
|
||
can't list models from a custom provider itself, so the map is
|
||
materialized client-side; if discovery fails the previous map is carried
|
||
over), deploys the repo's **skills** (see below), deploys the config to
|
||
`~/.config/opencode/opencode.gateway.json`, and exports `LITELLM_KEY`
|
||
(from `~/.config/mcp-sync/key`) plus `OPENCODE_CONFIG` (pointing at the
|
||
deployed file — opencode does NOT auto-load extra files in its config dir,
|
||
so without this export the file is ignored). Silent + non-fatal: a Gitea
|
||
hiccup never breaks the shell, last-good config stays in place.
|
||
- `../skills/` — the shared skill folders, deployed to
|
||
`~/.config/mcp-sync/skills` and wired in through `skills.paths` in the
|
||
config above. Skills are plain files, so — unlike models and MCP servers —
|
||
there is nothing on the gateway to discover and **the repo is the source of
|
||
truth**; the tree is copied from the checkout (or, with no checkout, fetched
|
||
whole from the repo tarball). They deploy to
|
||
mcp-sync's own directory rather than `~/.config/opencode/skills` so the
|
||
atomic replace can never delete a skill the user wrote by hand. See
|
||
`skills/README.md`.
|
||
- `install.sh` — one-time bootstrap. Installs `sync.sh` (from the checkout it
|
||
was run out of, else the Gitea API), records that checkout in
|
||
`~/.config/mcp-sync/source`, stores the user's key (0600), and adds the rc
|
||
hook.
|
||
|
||
opencode merges the `OPENCODE_CONFIG` file on top of the user's global
|
||
`~/.config/opencode/opencode.json` (and under any project `opencode.json`),
|
||
so the synced config coexists with each user's own settings and any
|
||
**local-only** MCP servers (e.g. a client-side `kubernetes` pointed at their
|
||
own kubeconfig). Sync never touches those.
|
||
|
||
## Where the config comes from
|
||
|
||
**The local checkout is the source.** `bootstrap.sh` already clones this repo
|
||
over authenticated git, so the per-shell hook reads `opencode.gateway.json` and
|
||
`skills/` straight off disk. No HTTP, no anonymous read, works offline.
|
||
|
||
This is deliberate. The hook used to re-download both from Gitea's public
|
||
`/raw/` URL on every sync, which meant **the repo had to be world-readable for
|
||
a shell hook to work** — publishing every internal hostname and the whole MCP
|
||
server inventory as the price of a config refresh. Worse, Gitea answers an
|
||
unauthorized `/raw/` request with a **200 and an HTML login page**, so `curl -f`
|
||
reported success and wrote HTML over the config; the failure was silent and the
|
||
config simply froze at its last good copy.
|
||
|
||
Resolution order, for both the config and the skills:
|
||
|
||
1. `MCP_SYNC_REPO` / `MCP_SYNC_RAW_URL` / `MCP_SYNC_SKILLS_SRC` if set — the
|
||
dev-checkout workflow in `zsh/zshrc.local.example`.
|
||
2. The checkout recorded in `~/.config/mcp-sync/source` by `install.sh`,
|
||
falling back to `$DOTFILES_DIR` (`~/.dotfiles`).
|
||
3. The **Gitea API** with `$GITEA_KEY` — for a box with no checkout at all.
|
||
API endpoints, not the web `/raw/` route, because the API returns an honest
|
||
401/403 that `curl -f` can actually detect.
|
||
|
||
To pick up a pushed change, the hook runs `git pull --ff-only` on the checkout
|
||
— but **only** the bootstrap-managed one at `$DOTFILES_DIR`, and only when its
|
||
tree is clean and on the default branch. A checkout you work in is read, never
|
||
pulled out from under you.
|
||
|
||
Model discovery is unaffected: it still queries the gateway's `/v1/models`
|
||
live on every sync, so a new model appears without touching the repo.
|
||
|
||
## User setup (one time)
|
||
|
||
Each user needs a LiteLLM **virtual key** (for models) plus their **own**
|
||
credentials for each MCP server. Run `bootstrap.sh` (it installs Node, which
|
||
provides the `npx` the MCP servers need), then:
|
||
|
||
```sh
|
||
# From a checkout (what bootstrap.sh does — no network needed for the config):
|
||
./mcp-config/install.sh
|
||
|
||
# Or standalone on a box with no checkout:
|
||
curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config/install.sh | sh
|
||
# (prompts for the sk-... key; or pass it: ... | LITELLM_KEY=sk-... sh)
|
||
# If this repo is private, export GITEA_KEY first so the fallback can read it.
|
||
```
|
||
|
||
Then export your own MCP credentials in `~/.zshrc.local`:
|
||
|
||
```sh
|
||
export GITEA_KEY=... # your Gitea PAT (repo + issue scopes)
|
||
export PROXMOX_TOKEN_ID=... # e.g. alice@pve!mcp
|
||
export PROXMOX_TOKEN_SECRET=... # the token UUID
|
||
```
|
||
|
||
Open a new terminal, run `opencode`, and the tools are there — acting as **you**.
|
||
|
||
## Adding an MCP server (admin)
|
||
|
||
Add it to `mcp.<name>` in `opencode.gateway.json` as `type: local`, with a
|
||
**pinned** package version and `{env:VAR}` refs for every secret, then push to
|
||
`main`. Users pick it up within the sync TTL (or immediately on a new shell).
|
||
|
||
Two hard requirements:
|
||
- It must be an **npm package** (opencode spawns it with `npx`).
|
||
- Its credentials must be **per-user**, via `{env:VAR}`. Never commit a token,
|
||
and never introduce a shared one — that is the whole reason MCP is not on the
|
||
gateway.
|
||
|
||
Models are different: they live on the gateway
|
||
(`charts/inference/values.yaml` → `model_list`) and are auto-discovered by the
|
||
`/v1/models` query in `sync.sh`. No client change needed.
|
||
|
||
## Adding a skill (admin)
|
||
|
||
Add a folder under `skills/` and push to `main` — see `skills/README.md`.
|
||
Verify with `opencode debug skill`.
|
||
|
||
## "I added a model on the gateway and opencode doesn't see it"
|
||
|
||
Almost always a stale local config, not a gateway or key problem. The sync runs
|
||
**only at shell start** and is throttled to `MCP_SYNC_TTL` (1h), so a new model
|
||
can take up to an hour *and* a new terminal to appear.
|
||
|
||
Pull it immediately:
|
||
|
||
```sh
|
||
mcp-sync # force a refresh now, ignoring the TTL; prints the model list
|
||
```
|
||
|
||
If the model still isn't listed, it isn't reaching your key — check the gateway,
|
||
not the client:
|
||
|
||
```sh
|
||
curl -fsS -H "Authorization: Bearer $LITELLM_KEY" \
|
||
https://llm.covert-gpt.com/v1/models | jq -r '.data[].id'
|
||
```
|
||
|
||
Whatever that prints is exactly what `sync.sh` will deploy. A model present here
|
||
but missing in opencode is a sync problem; a model missing here is a LiteLLM
|
||
`model_list` or key-scoping problem.
|
||
|
||
## Notes / caveats
|
||
|
||
- **Local-server precedence**: opencode merges with later-config-wins on key
|
||
conflicts. If a user names a local server the same as a gateway one, theirs
|
||
wins. Use distinct names.
|
||
- **Key rotation**: re-run `install.sh` (or edit `~/.config/mcp-sync/key`).
|
||
- **Other clients**: Hermes is wired (see "One definition, two clients").
|
||
Claude Code / open-webui would each still need a parallel render step in
|
||
`sync.sh`; not done yet.
|
||
- **Throttle**: `mcp-sync` forces one refresh; `MCP_SYNC_TTL=0` in the
|
||
environment makes every new shell pull.
|
||
- **python3 is required** for model discovery. Without it `sync.sh` deploys the
|
||
base config verbatim, so the provider comes up with an EMPTY model list. It
|
||
warns when this happens; `bootstrap.sh` installs python3.
|