Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.
Already in flight here (unchanged in intent):
- sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
install.sh records the checkout path so it can be found. That recording
is load-bearing: without it sync.sh cannot find the repo, silently falls
back to Gitea, and every local edit to opencode.gateway.json is ignored.
Observed on this box — ~/.config/mcp-sync/source did not exist, the
default ~/.dotfiles is not where the checkout lives, and the Gitea base
still served models:{}.
- the hermes custom-provider wiring.
Fixed on top:
opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.
hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.
NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.
Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
Hermes had no MCP servers at all -- no mcp_servers: key -- so "use the gitea
MCP" had nothing to call. The model said so ("there's no Gitea-specific skill"),
which was true, and then looped reaching for the browser as the only way in.
The obvious fix -- paste a gitea block into ~/.hermes/config.yaml -- is the
wrong one: it makes a SECOND definition of a server this repo already defines
once, free to drift (one pinned to gitea-mcp@0.0.10, the other to whatever got
typed). README already calls this out: "all MCP servers are defined here", and
listed other clients as "not done yet". So render, don't duplicate: sync.sh
already translates LiteLLM's /model/info into opencode's schema; this is the
same move into Hermes' schema, from opencode.gateway.json's mcp block. Add a
server once, both clients get it within the same TTL.
Not a shared server PROCESS, which is what "use the same one" would imply: MCP
stdio servers are subprocesses spawned per client over stdin/stdout and dying
with it. Each client running its own `npx -y <pkg>` is the design -- same
package, same pinned version, same per-user credential, same identity. Sharing
one process would need a persistent daemon plus a stdio->HTTP bridge, for
nothing. And it stays client-side: the gateway hosts no MCP servers on purpose
(values.yaml:892; user-scoped APIs, no shared tokens), and gitea-mcp hard-exits
without a token so LiteLLM's headerless listing never registers it anyway.
Translation notes: Hermes' `command` must be a STRING (a list stringifies to
"['npx', '-y']" and fails at exec), so opencode's argv list splits into
command + args. `{env:VAR}` becomes `${VAR}` -- both by-name, so no secret is
written to either file. Hermes gives an MCP child only a filtered baseline env,
so every secret must be listed in env:, which the rendered block does. Nothing
is added to platform_toolsets: MCP servers auto-enable by name, and listing some
would turn it into an allowlist.
Rendered names are recorded in ~/.config/mcp-sync/hermes-managed.json so a
server dropped upstream is dropped from Hermes, while servers the user added by
hand -- never in the manifest -- are left alone. Only writes when the result
changes, skips silently when Hermes isn't installed, warns when PyYAML is
missing (an absent MCP block is invisible rather than obviously broken).
Verified via Hermes' own discovery: 72 tools register -- 30 gitea (incl.
issue__list) and 42 proxmox -- with ${GITEA_KEY} / ${PROXMOX_MCP_*} resolving
and no secret in config.yaml.
Also fixes a stale README claim that MCP tools come from the gateway's /mcp
endpoint; they never do, and values.yaml:892 says why.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two client-side fixes against the covert-gpt LiteLLM gateway, both traced
from opencode throwing intermittent ContextWindowExceededError 400s.
opencode (mcp-config/sync.sh):
spec_for() mapped LiteLLM's /model/info onto opencode's schema 1:1, but the
two mean different things by "output". LiteLLM's max_output_tokens is the
whole window (vLLM caps TOTAL context, not output separately, which is
correct for LiteLLM); opencode's limit.output is a RESERVE it subtracts from
context to pick a compaction point. Copying one to the other told opencode to
reserve the entire 262144 window, so it targeted an input of context-output
and the server counted one token more than opencode did -- landing on
context+1 and a hard 400, every time (measured: 196609 in + 65536 out =
262145 vs a 262144 window; the retries shrinking max_tokens by 65 never
helped because the input grew to compensate).
Capping the reserve alone would NOT have fixed it -- opencode would still sit
exactly on the boundary, just at a different input size. So: cap the reserve
(OUTPUT_RESERVE_CAP) and hold a margin back from context
(CONTEXT_SAFETY_MARGIN). Generated at sync time, so it fixes every gateway
model at once, including third-party ones.
hermes (hermes/gateway.yaml, hermes/install.sh):
Hermes reached the gateway by pretending it WAS OpenRouter -- the LiteLLM key
in OPENROUTER_API_KEY, the gateway in OPENROUTER_BASE_URL. Requests routed
fine, but fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and
ignores OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model
catalog -- models this gateway cannot serve -- while the gateway's own models,
including cg-1's, never appeared at all.
Replaced with a named custom provider. install.sh now merges gateway.yaml into
~/.hermes/config.yaml, so a fresh box gets a Hermes that can actually talk to
the gateway instead of one pointed at nothing. The merge is idempotent and
narrow: only the covert-gpt entry is touched, model.default/provider are
seeded only when unset, and a box still on the masquerade is migrated off it
(fingerprint: provider openrouter + base_url pointing at this gateway; a
genuine OpenRouter user never has that, and is left alone).
No secrets: gateway.yaml names the key via key_env, and mcp-config/sync.sh is
what exports it. Hermes needs no context safety margin -- it omits max_tokens
entirely, so vLLM bounds output by whatever context is left.
Also folds in the in-flight hermes bootstrap wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/v1/models returns ONLY model ids, so every model was deployed as {} and
opencode ran all of them on defaults -- no context window (so it cannot manage
truncation), no cost, and no idea whether the model can call tools. opencode
requires this metadata in the models map: it has no /v1/models discovery of its
own, and models.dev only covers known providers, not a private LiteLLM gateway.
sync.sh now also queries /model/info and maps LiteLLM's metadata onto opencode's
per-model spec:
max_input_tokens -> limit.context
max_output_tokens -> limit.output
input/output_cost_per_token -> cost.input/output (per-token -> $ per 1M)
supports_function_calling -> tool_call
supports_reasoning -> reasoning
supports_vision -> attachment
Best-effort and backwards-compatible: /model/info is outside the llm_api_routes
scope most virtual keys have, so an un-updated key 403s and falls back to
names-only -- exactly today's behaviour, no regression.
Note the gateway only reports metadata for models in LiteLLM's registry. The
self-hosted vLLM entries (qwen3-*) carry no model_info in the gateway's
model_list, so they still deploy bare. Fixing that is a gateway-side change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The sync runs only at shell start and is throttled to MCP_SYNC_TTL (1h), so a
model added on the gateway can take up to an hour AND a new terminal before
opencode sees it. There was no way to say "fetch it now", so the recurring
symptom was "I added a model and it isn't showing up" -- with a healthy
gateway and a correctly-scoped key.
`mcp-sync` re-sources the hook with the throttle disabled and prints the
resulting model list, so it also answers "what does the client actually have?"
in one step.
README gains a section splitting the two failure modes: a model the gateway
returns but opencode lacks is a stale client; a model the gateway does not
return is a LiteLLM model_list / key-scoping problem.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
~/.zshrc is MANAGED: zsh/install.sh overwrites it wholesale (cp) on every
bootstrap run. install.sh appended the mcp-sync hook to that same file, so
the next bootstrap silently deleted it -- sync.sh then never ran, so
OPENCODE_CONFIG was never exported and opencode came up with no models and
no skills. It looked like a broken gateway or a bad model; it was a wiped
shell hook.
~/.zshrc.local is the unmanaged file the managed zshrc sources last, so the
hook now survives a re-bootstrap. Verified: hook lands in ~/.zshrc.local
(0600), managed ~/.zshrc is left untouched, and OPENCODE_CONFIG still
exports after simulating the bootstrap overwrite.
Also warn instead of failing silently when python3 is missing: sync.sh then
deploys the base config verbatim with an EMPTY models map, which presents as
"the gateway has no models" rather than "a dependency is absent".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every API worth exposing (Gitea, Proxmox) is user-scoped, so hosting MCP
centrally means one shared token per server: everyone acts as the same
identity and attribution is lost. LiteLLM's per-request BYOK cannot cover
all of them either -- it builds its tool->server map from a headerless
background listing, and gitea-mcp hard-exits without a token, so it never
registers. A split brain (some MCPs server-side, some client-side) is worse
than either, so all of them move client-side.
Servers are declared local, spawned via npx with pinned versions, and read
each user's OWN credentials via {env:VAR}. No token is ever written to the
config; the file stays identical for every user.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Models and MCP servers are discovered from the LiteLLM gateway at sync time
(/v1/models, and the aggregate /mcp/ endpoint). Skills have no gateway side to
query: LiteLLM's /v1/skills is a passthrough to Anthropic's Skills API, which
lists skills on an Anthropic account rather than serving skill files, and
/public/skill_hub is a marketplace catalog. So the repo is the source of truth
and the tree is fetched whole, from the repo tarball.
Deploy to ~/.config/mcp-sync/skills rather than ~/.config/opencode/skills so
the atomic replace can never delete a skill the user wrote by hand; opencode
picks the tree up via skills.paths, which expands {env:HOME} — verified — so
the shared config stays identical for every user.
Same throttle and non-fatal discipline as the config sync: a fetch that yields
no skills/ leaves the last-good tree in place instead of emptying it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two changes to the synced opencode config:
- Model provider. opencode can't list models from a custom OpenAI-compatible
provider itself, so sync.sh queries the gateway's /v1/models with the
user's key and materializes the map client-side. New gateway models land
on every box at the next sync with no config edit. If discovery fails the
previous map is carried over, so a gateway hiccup never empties the list.
- Aggregate MCP endpoint. One entry at /mcp/ (trailing slash required — the
bare path 307-redirects and MCP clients don't follow it) exposes every
server the key can access, replacing the per-server gitea entry.
sync.sh also exports OPENCODE_CONFIG: opencode does not auto-load extra
files from its config dir, so without it the synced file was ignored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
noconnorandClaude Opus 5