mcp-config/hermes: make the gateway's context limits actually reach the clients

Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.

Already in flight here (unchanged in intent):
  - sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
    install.sh records the checkout path so it can be found. That recording
    is load-bearing: without it sync.sh cannot find the repo, silently falls
    back to Gitea, and every local edit to opencode.gateway.json is ignored.
    Observed on this box — ~/.config/mcp-sync/source did not exist, the
    default ~/.dotfiles is not where the checkout lives, and the Gitea base
    still served models:{}.
  - the hermes custom-provider wiring.

Fixed on top:

opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.

hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.

NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.

Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
This commit is contained in:
2026-09-11 16:38:03 -04:00
co-authored by Claude Opus 5
parent 098b1e377f
commit 135112eee5
10 changed files with 496 additions and 68 deletions
+48 -7
View File
@@ -62,7 +62,7 @@ config. Each user exports them in `~/.zshrc.local`:
| MCP server | env vars |
|---|---|
| `gitea` | `GITEA_KEY` |
| `proxmox` | `PROXMOX_MCP_TOKEN_ID`, `PROXMOX_MCP_TOKEN_SECRET` |
| `proxmox` | `PROXMOX_TOKEN_ID`, `PROXMOX_TOKEN_SECRET` |
**Requires `npx`** — the MCP servers are npm packages that opencode spawns as
`npx -y <pkg>@<version>`. `bootstrap.sh` installs Node for this (`node/`).
@@ -87,7 +87,8 @@ sync — the laptops pull from Gitea's `main`.)
the file. Versions are pinned — `npx` runs whatever the registry serves, and
that process gets your tokens.
- `sync.sh` — sourced from each user's shell rc. On a new shell (throttled
~1h) it pulls the base config from Gitea, queries the gateway's
~1h) it reads the base config **from the local checkout** (see "Where the
config comes from" below), queries the gateway's
`/v1/models` with the user's key to **auto-discover models** (opencode
can't list models from a custom provider itself, so the map is
materialized client-side; if discovery fails the previous map is carried
@@ -101,12 +102,15 @@ sync — the laptops pull from Gitea's `main`.)
`~/.config/mcp-sync/skills` and wired in through `skills.paths` in the
config above. Skills are plain files, so — unlike models and MCP servers —
there is nothing on the gateway to discover and **the repo is the source of
truth**; the tree is fetched whole from the repo tarball. They deploy to
truth**; the tree is copied from the checkout (or, with no checkout, fetched
whole from the repo tarball). They deploy to
mcp-sync's own directory rather than `~/.config/opencode/skills` so the
atomic replace can never delete a skill the user wrote by hand. See
`skills/README.md`.
- `install.sh` — one-time bootstrap (curl-pipe). Fetches `sync.sh`, stores the
user's key (0600), and adds the rc hook.
- `install.sh` — one-time bootstrap. Installs `sync.sh` (from the checkout it
was run out of, else the Gitea API), records that checkout in
`~/.config/mcp-sync/source`, stores the user's key (0600), and adds the rc
hook.
opencode merges the `OPENCODE_CONFIG` file on top of the user's global
`~/.config/opencode/opencode.json` (and under any project `opencode.json`),
@@ -114,6 +118,38 @@ so the synced config coexists with each user's own settings and any
**local-only** MCP servers (e.g. a client-side `kubernetes` pointed at their
own kubeconfig). Sync never touches those.
## Where the config comes from
**The local checkout is the source.** `bootstrap.sh` already clones this repo
over authenticated git, so the per-shell hook reads `opencode.gateway.json` and
`skills/` straight off disk. No HTTP, no anonymous read, works offline.
This is deliberate. The hook used to re-download both from Gitea's public
`/raw/` URL on every sync, which meant **the repo had to be world-readable for
a shell hook to work** — publishing every internal hostname and the whole MCP
server inventory as the price of a config refresh. Worse, Gitea answers an
unauthorized `/raw/` request with a **200 and an HTML login page**, so `curl -f`
reported success and wrote HTML over the config; the failure was silent and the
config simply froze at its last good copy.
Resolution order, for both the config and the skills:
1. `MCP_SYNC_REPO` / `MCP_SYNC_RAW_URL` / `MCP_SYNC_SKILLS_SRC` if set — the
dev-checkout workflow in `zsh/zshrc.local.example`.
2. The checkout recorded in `~/.config/mcp-sync/source` by `install.sh`,
falling back to `$DOTFILES_DIR` (`~/.dotfiles`).
3. The **Gitea API** with `$GITEA_KEY` — for a box with no checkout at all.
API endpoints, not the web `/raw/` route, because the API returns an honest
401/403 that `curl -f` can actually detect.
To pick up a pushed change, the hook runs `git pull --ff-only` on the checkout
— but **only** the bootstrap-managed one at `$DOTFILES_DIR`, and only when its
tree is clean and on the default branch. A checkout you work in is read, never
pulled out from under you.
Model discovery is unaffected: it still queries the gateway's `/v1/models`
live on every sync, so a new model appears without touching the repo.
## User setup (one time)
Each user needs a LiteLLM **virtual key** (for models) plus their **own**
@@ -121,16 +157,21 @@ credentials for each MCP server. Run `bootstrap.sh` (it installs Node, which
provides the `npx` the MCP servers need), then:
```sh
# From a checkout (what bootstrap.sh does — no network needed for the config):
./mcp-config/install.sh
# Or standalone on a box with no checkout:
curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config/install.sh | sh
# (prompts for the sk-... key; or pass it: ... | LITELLM_KEY=sk-... sh)
# If this repo is private, export GITEA_KEY first so the fallback can read it.
```
Then export your own MCP credentials in `~/.zshrc.local`:
```sh
export GITEA_KEY=... # your Gitea PAT (repo + issue scopes)
export PROXMOX_MCP_TOKEN_ID=... # e.g. alice@pve!mcp
export PROXMOX_MCP_TOKEN_SECRET=... # the token UUID
export PROXMOX_TOKEN_ID=... # e.g. alice@pve!mcp
export PROXMOX_TOKEN_SECRET=... # the token UUID
```
Open a new terminal, run `opencode`, and the tools are there — acting as **you**.