mcp-config/hermes: make the gateway's context limits actually reach the clients
Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.
Already in flight here (unchanged in intent):
- sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
install.sh records the checkout path so it can be found. That recording
is load-bearing: without it sync.sh cannot find the repo, silently falls
back to Gitea, and every local edit to opencode.gateway.json is ignored.
Observed on this box — ~/.config/mcp-sync/source did not exist, the
default ~/.dotfiles is not where the checkout lives, and the Gitea base
still served models:{}.
- the hermes custom-provider wiring.
Fixed on top:
opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.
hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.
NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.
Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
This commit is contained in:
+48
-7
@@ -62,7 +62,7 @@ config. Each user exports them in `~/.zshrc.local`:
|
||||
| MCP server | env vars |
|
||||
|---|---|
|
||||
| `gitea` | `GITEA_KEY` |
|
||||
| `proxmox` | `PROXMOX_MCP_TOKEN_ID`, `PROXMOX_MCP_TOKEN_SECRET` |
|
||||
| `proxmox` | `PROXMOX_TOKEN_ID`, `PROXMOX_TOKEN_SECRET` |
|
||||
|
||||
**Requires `npx`** — the MCP servers are npm packages that opencode spawns as
|
||||
`npx -y <pkg>@<version>`. `bootstrap.sh` installs Node for this (`node/`).
|
||||
@@ -87,7 +87,8 @@ sync — the laptops pull from Gitea's `main`.)
|
||||
the file. Versions are pinned — `npx` runs whatever the registry serves, and
|
||||
that process gets your tokens.
|
||||
- `sync.sh` — sourced from each user's shell rc. On a new shell (throttled
|
||||
~1h) it pulls the base config from Gitea, queries the gateway's
|
||||
~1h) it reads the base config **from the local checkout** (see "Where the
|
||||
config comes from" below), queries the gateway's
|
||||
`/v1/models` with the user's key to **auto-discover models** (opencode
|
||||
can't list models from a custom provider itself, so the map is
|
||||
materialized client-side; if discovery fails the previous map is carried
|
||||
@@ -101,12 +102,15 @@ sync — the laptops pull from Gitea's `main`.)
|
||||
`~/.config/mcp-sync/skills` and wired in through `skills.paths` in the
|
||||
config above. Skills are plain files, so — unlike models and MCP servers —
|
||||
there is nothing on the gateway to discover and **the repo is the source of
|
||||
truth**; the tree is fetched whole from the repo tarball. They deploy to
|
||||
truth**; the tree is copied from the checkout (or, with no checkout, fetched
|
||||
whole from the repo tarball). They deploy to
|
||||
mcp-sync's own directory rather than `~/.config/opencode/skills` so the
|
||||
atomic replace can never delete a skill the user wrote by hand. See
|
||||
`skills/README.md`.
|
||||
- `install.sh` — one-time bootstrap (curl-pipe). Fetches `sync.sh`, stores the
|
||||
user's key (0600), and adds the rc hook.
|
||||
- `install.sh` — one-time bootstrap. Installs `sync.sh` (from the checkout it
|
||||
was run out of, else the Gitea API), records that checkout in
|
||||
`~/.config/mcp-sync/source`, stores the user's key (0600), and adds the rc
|
||||
hook.
|
||||
|
||||
opencode merges the `OPENCODE_CONFIG` file on top of the user's global
|
||||
`~/.config/opencode/opencode.json` (and under any project `opencode.json`),
|
||||
@@ -114,6 +118,38 @@ so the synced config coexists with each user's own settings and any
|
||||
**local-only** MCP servers (e.g. a client-side `kubernetes` pointed at their
|
||||
own kubeconfig). Sync never touches those.
|
||||
|
||||
## Where the config comes from
|
||||
|
||||
**The local checkout is the source.** `bootstrap.sh` already clones this repo
|
||||
over authenticated git, so the per-shell hook reads `opencode.gateway.json` and
|
||||
`skills/` straight off disk. No HTTP, no anonymous read, works offline.
|
||||
|
||||
This is deliberate. The hook used to re-download both from Gitea's public
|
||||
`/raw/` URL on every sync, which meant **the repo had to be world-readable for
|
||||
a shell hook to work** — publishing every internal hostname and the whole MCP
|
||||
server inventory as the price of a config refresh. Worse, Gitea answers an
|
||||
unauthorized `/raw/` request with a **200 and an HTML login page**, so `curl -f`
|
||||
reported success and wrote HTML over the config; the failure was silent and the
|
||||
config simply froze at its last good copy.
|
||||
|
||||
Resolution order, for both the config and the skills:
|
||||
|
||||
1. `MCP_SYNC_REPO` / `MCP_SYNC_RAW_URL` / `MCP_SYNC_SKILLS_SRC` if set — the
|
||||
dev-checkout workflow in `zsh/zshrc.local.example`.
|
||||
2. The checkout recorded in `~/.config/mcp-sync/source` by `install.sh`,
|
||||
falling back to `$DOTFILES_DIR` (`~/.dotfiles`).
|
||||
3. The **Gitea API** with `$GITEA_KEY` — for a box with no checkout at all.
|
||||
API endpoints, not the web `/raw/` route, because the API returns an honest
|
||||
401/403 that `curl -f` can actually detect.
|
||||
|
||||
To pick up a pushed change, the hook runs `git pull --ff-only` on the checkout
|
||||
— but **only** the bootstrap-managed one at `$DOTFILES_DIR`, and only when its
|
||||
tree is clean and on the default branch. A checkout you work in is read, never
|
||||
pulled out from under you.
|
||||
|
||||
Model discovery is unaffected: it still queries the gateway's `/v1/models`
|
||||
live on every sync, so a new model appears without touching the repo.
|
||||
|
||||
## User setup (one time)
|
||||
|
||||
Each user needs a LiteLLM **virtual key** (for models) plus their **own**
|
||||
@@ -121,16 +157,21 @@ credentials for each MCP server. Run `bootstrap.sh` (it installs Node, which
|
||||
provides the `npx` the MCP servers need), then:
|
||||
|
||||
```sh
|
||||
# From a checkout (what bootstrap.sh does — no network needed for the config):
|
||||
./mcp-config/install.sh
|
||||
|
||||
# Or standalone on a box with no checkout:
|
||||
curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config/install.sh | sh
|
||||
# (prompts for the sk-... key; or pass it: ... | LITELLM_KEY=sk-... sh)
|
||||
# If this repo is private, export GITEA_KEY first so the fallback can read it.
|
||||
```
|
||||
|
||||
Then export your own MCP credentials in `~/.zshrc.local`:
|
||||
|
||||
```sh
|
||||
export GITEA_KEY=... # your Gitea PAT (repo + issue scopes)
|
||||
export PROXMOX_MCP_TOKEN_ID=... # e.g. alice@pve!mcp
|
||||
export PROXMOX_MCP_TOKEN_SECRET=... # the token UUID
|
||||
export PROXMOX_TOKEN_ID=... # e.g. alice@pve!mcp
|
||||
export PROXMOX_TOKEN_SECRET=... # the token UUID
|
||||
```
|
||||
|
||||
Open a new terminal, run `opencode`, and the tools are there — acting as **you**.
|
||||
|
||||
Reference in New Issue
Block a user