mcp-config/hermes: make the gateway's context limits actually reach the clients

Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.

Already in flight here (unchanged in intent):
  - sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
    install.sh records the checkout path so it can be found. That recording
    is load-bearing: without it sync.sh cannot find the repo, silently falls
    back to Gitea, and every local edit to opencode.gateway.json is ignored.
    Observed on this box — ~/.config/mcp-sync/source did not exist, the
    default ~/.dotfiles is not where the checkout lives, and the Gitea base
    still served models:{}.
  - the hermes custom-provider wiring.

Fixed on top:

opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.

hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.

NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.

Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
This commit is contained in:
2026-09-11 16:38:03 -04:00
co-authored by Claude Opus 5
parent 098b1e377f
commit 135112eee5
10 changed files with 496 additions and 68 deletions
+9 -4
View File
@@ -72,7 +72,9 @@ Already have a clone? Just run it directly:
daemon; no-ops on systemd or non-rootless hosts. Re-run at each bootstrap
(the cgroup layout resets on redeploy); its PATH shim lives in `$HOME`.
- `mcp-config/` — shared MCP gateway config sync for laptops (LiteLLM /
opencode). Independent of `bootstrap.sh`; see its own README.
opencode). Run by `bootstrap.sh`, and standalone too. The config and skills
are read from the **local checkout**, not re-downloaded from Gitea, so this
repo does not have to be public for a shell to work; see its own README.
## Per-machine customizations and secrets
@@ -86,15 +88,18 @@ cp zsh/zshrc.local.example ~/.zshrc.local && chmod 600 ~/.zshrc.local
Variables modules look for:
- `GITEA_KEY` — personal access token for `git.nic-oconnor.com` (API/CLI
use; HTTPS git auth itself goes through `git/`'s credential helper).
- `GITEA_KEY` — personal access token for `git.nic-oconnor.com`. Used for
API/CLI work, and by `bootstrap.sh`/`mcp-config/` to read this repo when it
is **private** and there is no checkout yet. With a checkout on disk nothing
here needs it; HTTPS git auth otherwise goes through `git/`'s credential
helper.
- `LITELLM_KEY` — covert-gpt gateway key used by opencode. Normally you
don't set this by hand: run `mcp-config/install.sh` once and its shell
hook exports it from `~/.config/mcp-sync/key`. Set it in `~/.zshrc.local`
only on a box without mcp-sync.
- `OP_SERVICE_ACCOUNT_TOKEN` — 1Password service-account token for
non-interactive `op read` / `op run`.
- `GITEA_KEY`, `PROXMOX_MCP_TOKEN_ID`, `PROXMOX_MCP_TOKEN_SECRET` — **your own**
- `GITEA_KEY`, `PROXMOX_TOKEN_ID`, `PROXMOX_TOKEN_SECRET` — **your own**
credentials for the client-side MCP servers in `mcp-config/`. opencode reads
them from your environment (`{env:VAR}`) and passes them to the MCP server it
spawns, so the tools act as *you*. Keep them in `~/.zshrc.local`; they are