noconnorandClaude Opus 5
135112eee5
Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.
Already in flight here (unchanged in intent):
- sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
install.sh records the checkout path so it can be found. That recording
is load-bearing: without it sync.sh cannot find the repo, silently falls
back to Gitea, and every local edit to opencode.gateway.json is ignored.
Observed on this box — ~/.config/mcp-sync/source did not exist, the
default ~/.dotfiles is not where the checkout lives, and the Gitea base
still served models:{}.
- the hermes custom-provider wiring.
Fixed on top:
opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.
hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.
NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.
Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
238 lines
12 KiB
Markdown
238 lines
12 KiB
Markdown
# mcp-config — shared client config for laptops
|
|
|
|
Single source of truth for what the agents see: the **covert-gpt model provider**
|
|
(LiteLLM at `llm.covert-gpt.com`) and the **MCP servers**, which run
|
|
**client-side**. Users install once; their config then refreshes automatically
|
|
on each new shell.
|
|
|
|
Two clients consume it: **opencode** (deployed to
|
|
`~/.config/opencode/opencode.gateway.json`) and **Hermes** (its `mcp_servers:`
|
|
block in `~/.hermes/config.yaml` is rendered from the same definitions — see
|
|
"One definition, two clients" below).
|
|
|
|
## MCP runs on the CLIENT, never on the gateway
|
|
|
|
The LiteLLM gateway hosts **no MCP servers** — deliberately. Every API worth
|
|
exposing (Gitea, Proxmox, UniFi, NetBird, k8s) is **user-scoped**, so hosting it
|
|
centrally means one shared token per server: every user acts as the same
|
|
identity and attribution is gone.
|
|
|
|
LiteLLM *can* do per-request BYOK (env values written `${X-Header}` are filled
|
|
from the caller's headers — verified working for proxmox on v1.87.0), but only
|
|
for servers that tolerate a credential-less spawn. LiteLLM builds its
|
|
tool→server mapping from a **headerless** background listing, and `gitea-mcp`
|
|
hard-exits without a token, so it never registers and its tool calls 404. The
|
|
gateway could therefore host only *some* servers — and a split brain (some MCPs
|
|
server-side, some client-side) is worse than either. So **all** MCP servers are
|
|
defined here, each reading the user's **own** credentials from their own
|
|
environment. The config is identical for everyone; only the secrets differ.
|
|
|
|
## One definition, two clients
|
|
|
|
The servers are defined **once**, in `opencode.gateway.json`'s `mcp` block.
|
|
Hermes speaks a different schema, so `sync.sh` *renders* that one definition into
|
|
`mcp_servers:` in `~/.hermes/config.yaml` rather than keeping a second copy — two
|
|
copies would drift, one pinned to `gitea-mcp@0.0.10` and the other to whatever
|
|
got typed. Add a server here once and **both** clients get it, within the same
|
|
TTL. The translation is mechanical:
|
|
|
|
| opencode | Hermes |
|
|
|---|---|
|
|
| `command: ["npx","-y","pkg@1.2.3"]` (argv list) | `command: npx` + `args: ["-y","pkg@1.2.3"]` — Hermes' `command` must be a **string**; a list stringifies and fails at exec |
|
|
| `environment: {K: "{env:VAR}"}` | `env: {K: "${VAR}"}` |
|
|
| `type: remote`, `url`, `headers` | `url`, `headers` |
|
|
| `enabled: false` | entry omitted |
|
|
|
|
Both placeholder forms are by-NAME references resolved at runtime, so no secret
|
|
is written to either file. Hermes hands an MCP child only a filtered baseline
|
|
environment, so every secret a server needs **must** be listed in `env:` — which
|
|
is exactly what the rendered block does.
|
|
|
|
Servers you add to Hermes by hand are left alone: `sync.sh` records the names it
|
|
rendered in `~/.config/mcp-sync/hermes-managed.json`, so a server dropped from
|
|
this repo is dropped from Hermes too, while anything not in that manifest is
|
|
never touched. Rendering is skipped silently when Hermes isn't installed, and
|
|
needs PyYAML (it warns if that's missing). Nothing needs adding to Hermes'
|
|
`platform_toolsets` — MCP servers are auto-enabled by name; listing *some*
|
|
would turn it into an allowlist, and `no_mcp` disables them all.
|
|
|
|
Credentials are referenced as `{env:VAR}` and are **never** written into the
|
|
config. Each user exports them in `~/.zshrc.local`:
|
|
|
|
| MCP server | env vars |
|
|
|---|---|
|
|
| `gitea` | `GITEA_KEY` |
|
|
| `proxmox` | `PROXMOX_TOKEN_ID`, `PROXMOX_TOKEN_SECRET` |
|
|
|
|
**Requires `npx`** — the MCP servers are npm packages that opencode spawns as
|
|
`npx -y <pkg>@<version>`. `bootstrap.sh` installs Node for this (`node/`).
|
|
Without it every server dies with "command not found", which opencode surfaces
|
|
only as a dead server — easy to misread as a broken config.
|
|
|
|
This repo lives in Gitea at `public/dotfiles`. (It is mirrored from the
|
|
`ai-k8s-iac` repo under `mcp-config/`; edit it there or here and keep them in
|
|
sync — the laptops pull from Gitea's `main`.)
|
|
|
|
## How it works
|
|
|
|
- `opencode.gateway.json` — the shared base config:
|
|
- the `covert-gpt` model provider (LiteLLM's OpenAI-compatible endpoint at
|
|
`https://llm.covert-gpt.com/v1`, models map left empty — it is filled by
|
|
discovery at sync time), and
|
|
- the **MCP servers**, each `type: local`, spawned by opencode as
|
|
`npx -y <pkg>@<pinned-version>` on the user's own machine.
|
|
|
|
**Identical for every user**: every credential (the LiteLLM key, the Gitea
|
|
PAT, the Proxmox token) is injected via `{env:VAR}` and never written into
|
|
the file. Versions are pinned — `npx` runs whatever the registry serves, and
|
|
that process gets your tokens.
|
|
- `sync.sh` — sourced from each user's shell rc. On a new shell (throttled
|
|
~1h) it reads the base config **from the local checkout** (see "Where the
|
|
config comes from" below), queries the gateway's
|
|
`/v1/models` with the user's key to **auto-discover models** (opencode
|
|
can't list models from a custom provider itself, so the map is
|
|
materialized client-side; if discovery fails the previous map is carried
|
|
over), deploys the repo's **skills** (see below), deploys the config to
|
|
`~/.config/opencode/opencode.gateway.json`, and exports `LITELLM_KEY`
|
|
(from `~/.config/mcp-sync/key`) plus `OPENCODE_CONFIG` (pointing at the
|
|
deployed file — opencode does NOT auto-load extra files in its config dir,
|
|
so without this export the file is ignored). Silent + non-fatal: a Gitea
|
|
hiccup never breaks the shell, last-good config stays in place.
|
|
- `../skills/` — the shared skill folders, deployed to
|
|
`~/.config/mcp-sync/skills` and wired in through `skills.paths` in the
|
|
config above. Skills are plain files, so — unlike models and MCP servers —
|
|
there is nothing on the gateway to discover and **the repo is the source of
|
|
truth**; the tree is copied from the checkout (or, with no checkout, fetched
|
|
whole from the repo tarball). They deploy to
|
|
mcp-sync's own directory rather than `~/.config/opencode/skills` so the
|
|
atomic replace can never delete a skill the user wrote by hand. See
|
|
`skills/README.md`.
|
|
- `install.sh` — one-time bootstrap. Installs `sync.sh` (from the checkout it
|
|
was run out of, else the Gitea API), records that checkout in
|
|
`~/.config/mcp-sync/source`, stores the user's key (0600), and adds the rc
|
|
hook.
|
|
|
|
opencode merges the `OPENCODE_CONFIG` file on top of the user's global
|
|
`~/.config/opencode/opencode.json` (and under any project `opencode.json`),
|
|
so the synced config coexists with each user's own settings and any
|
|
**local-only** MCP servers (e.g. a client-side `kubernetes` pointed at their
|
|
own kubeconfig). Sync never touches those.
|
|
|
|
## Where the config comes from
|
|
|
|
**The local checkout is the source.** `bootstrap.sh` already clones this repo
|
|
over authenticated git, so the per-shell hook reads `opencode.gateway.json` and
|
|
`skills/` straight off disk. No HTTP, no anonymous read, works offline.
|
|
|
|
This is deliberate. The hook used to re-download both from Gitea's public
|
|
`/raw/` URL on every sync, which meant **the repo had to be world-readable for
|
|
a shell hook to work** — publishing every internal hostname and the whole MCP
|
|
server inventory as the price of a config refresh. Worse, Gitea answers an
|
|
unauthorized `/raw/` request with a **200 and an HTML login page**, so `curl -f`
|
|
reported success and wrote HTML over the config; the failure was silent and the
|
|
config simply froze at its last good copy.
|
|
|
|
Resolution order, for both the config and the skills:
|
|
|
|
1. `MCP_SYNC_REPO` / `MCP_SYNC_RAW_URL` / `MCP_SYNC_SKILLS_SRC` if set — the
|
|
dev-checkout workflow in `zsh/zshrc.local.example`.
|
|
2. The checkout recorded in `~/.config/mcp-sync/source` by `install.sh`,
|
|
falling back to `$DOTFILES_DIR` (`~/.dotfiles`).
|
|
3. The **Gitea API** with `$GITEA_KEY` — for a box with no checkout at all.
|
|
API endpoints, not the web `/raw/` route, because the API returns an honest
|
|
401/403 that `curl -f` can actually detect.
|
|
|
|
To pick up a pushed change, the hook runs `git pull --ff-only` on the checkout
|
|
— but **only** the bootstrap-managed one at `$DOTFILES_DIR`, and only when its
|
|
tree is clean and on the default branch. A checkout you work in is read, never
|
|
pulled out from under you.
|
|
|
|
Model discovery is unaffected: it still queries the gateway's `/v1/models`
|
|
live on every sync, so a new model appears without touching the repo.
|
|
|
|
## User setup (one time)
|
|
|
|
Each user needs a LiteLLM **virtual key** (for models) plus their **own**
|
|
credentials for each MCP server. Run `bootstrap.sh` (it installs Node, which
|
|
provides the `npx` the MCP servers need), then:
|
|
|
|
```sh
|
|
# From a checkout (what bootstrap.sh does — no network needed for the config):
|
|
./mcp-config/install.sh
|
|
|
|
# Or standalone on a box with no checkout:
|
|
curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config/install.sh | sh
|
|
# (prompts for the sk-... key; or pass it: ... | LITELLM_KEY=sk-... sh)
|
|
# If this repo is private, export GITEA_KEY first so the fallback can read it.
|
|
```
|
|
|
|
Then export your own MCP credentials in `~/.zshrc.local`:
|
|
|
|
```sh
|
|
export GITEA_KEY=... # your Gitea PAT (repo + issue scopes)
|
|
export PROXMOX_TOKEN_ID=... # e.g. alice@pve!mcp
|
|
export PROXMOX_TOKEN_SECRET=... # the token UUID
|
|
```
|
|
|
|
Open a new terminal, run `opencode`, and the tools are there — acting as **you**.
|
|
|
|
## Adding an MCP server (admin)
|
|
|
|
Add it to `mcp.<name>` in `opencode.gateway.json` as `type: local`, with a
|
|
**pinned** package version and `{env:VAR}` refs for every secret, then push to
|
|
`main`. Users pick it up within the sync TTL (or immediately on a new shell).
|
|
|
|
Two hard requirements:
|
|
- It must be an **npm package** (opencode spawns it with `npx`).
|
|
- Its credentials must be **per-user**, via `{env:VAR}`. Never commit a token,
|
|
and never introduce a shared one — that is the whole reason MCP is not on the
|
|
gateway.
|
|
|
|
Models are different: they live on the gateway
|
|
(`charts/inference/values.yaml` → `model_list`) and are auto-discovered by the
|
|
`/v1/models` query in `sync.sh`. No client change needed.
|
|
|
|
## Adding a skill (admin)
|
|
|
|
Add a folder under `skills/` and push to `main` — see `skills/README.md`.
|
|
Verify with `opencode debug skill`.
|
|
|
|
## "I added a model on the gateway and opencode doesn't see it"
|
|
|
|
Almost always a stale local config, not a gateway or key problem. The sync runs
|
|
**only at shell start** and is throttled to `MCP_SYNC_TTL` (1h), so a new model
|
|
can take up to an hour *and* a new terminal to appear.
|
|
|
|
Pull it immediately:
|
|
|
|
```sh
|
|
mcp-sync # force a refresh now, ignoring the TTL; prints the model list
|
|
```
|
|
|
|
If the model still isn't listed, it isn't reaching your key — check the gateway,
|
|
not the client:
|
|
|
|
```sh
|
|
curl -fsS -H "Authorization: Bearer $LITELLM_KEY" \
|
|
https://llm.covert-gpt.com/v1/models | jq -r '.data[].id'
|
|
```
|
|
|
|
Whatever that prints is exactly what `sync.sh` will deploy. A model present here
|
|
but missing in opencode is a sync problem; a model missing here is a LiteLLM
|
|
`model_list` or key-scoping problem.
|
|
|
|
## Notes / caveats
|
|
|
|
- **Local-server precedence**: opencode merges with later-config-wins on key
|
|
conflicts. If a user names a local server the same as a gateway one, theirs
|
|
wins. Use distinct names.
|
|
- **Key rotation**: re-run `install.sh` (or edit `~/.config/mcp-sync/key`).
|
|
- **Other clients**: Hermes is wired (see "One definition, two clients").
|
|
Claude Code / open-webui would each still need a parallel render step in
|
|
`sync.sh`; not done yet.
|
|
- **Throttle**: `mcp-sync` forces one refresh; `MCP_SYNC_TTL=0` in the
|
|
environment makes every new shell pull.
|
|
- **python3 is required** for model discovery. Without it `sync.sh` deploys the
|
|
base config verbatim, so the provider comes up with an EMPTY model list. It
|
|
warns when this happens; `bootstrap.sh` installs python3.
|