2 Commits

Include renames

Author	SHA1	Message	Date
noconnorandClaude Opus 5	135112eee5	mcp-config/hermes: make the gateway's context limits actually reach the clients ... Folds in the in-flight local-checkout work and fixes the context accounting on top of it. Both halves are the same bug seen from two ends: clients were sending requests larger than the backend's window and getting hard ContextWindowExceededError 400s, intermittently, in a way that looked like the model or the GPU misbehaving. Already in flight here (unchanged in intent): - sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and install.sh records the checkout path so it can be found. That recording is load-bearing: without it sync.sh cannot find the repo, silently falls back to Gitea, and every local edit to opencode.gateway.json is ignored. Observed on this box — ~/.config/mcp-sync/source did not exist, the default ~/.dotfiles is not where the checkout lives, and the Gitea base still served models:{}. - the hermes custom-provider wiring. Fixed on top: opencode had NO context limit on ANY gateway model, not a wrong one. Two faults compounded. /model/info 403s for a day-to-day virtual key (only management-scoped keys may read it), which sync.sh treats as best-effort and falls back to names-only — as designed and as commented. But the merge then did `prov["models"] = models`, replacing the map wholesale, so limits declared in the base config were discarded too and there was no floor left. Base specs are now a floor that live /model/info overrides field by field: discovery still wins where present, so this adds no drift, and a 403 no longer strips the limits. Verified: ornith 195584/32768, the qwens 261120/32768. hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway stopped serving when Ornith replaced it — while ornith-1.0-35b, the model actually on cg-1 AND the only one with a window smaller than its neighbours (196608), had no entry at all and inherited Hermes' default. The one model needing a lower ceiling was the one not declaring one, which is exactly the shape that fails only once a session grows. NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing until install.sh re-runs — this machine's live config was patched by hand. Not addressed here, deliberately: the gateway-side context_window_fallback (ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing the oversized request client-side is what actually works today. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf	2026-09-11 16:38:03 -04:00
noconnorandClaude Opus 4.8	e42807c054	hermes: wire into dotfiles; fix opencode's generated context limits ... Two client-side fixes against the covert-gpt LiteLLM gateway, both traced from opencode throwing intermittent ContextWindowExceededError 400s. opencode (mcp-config/sync.sh): spec_for() mapped LiteLLM's /model/info onto opencode's schema 1:1, but the two mean different things by "output". LiteLLM's max_output_tokens is the whole window (vLLM caps TOTAL context, not output separately, which is correct for LiteLLM); opencode's limit.output is a RESERVE it subtracts from context to pick a compaction point. Copying one to the other told opencode to reserve the entire 262144 window, so it targeted an input of context-output and the server counted one token more than opencode did -- landing on context+1 and a hard 400, every time (measured: 196609 in + 65536 out = 262145 vs a 262144 window; the retries shrinking max_tokens by 65 never helped because the input grew to compensate). Capping the reserve alone would NOT have fixed it -- opencode would still sit exactly on the boundary, just at a different input size. So: cap the reserve (OUTPUT_RESERVE_CAP) and hold a margin back from context (CONTEXT_SAFETY_MARGIN). Generated at sync time, so it fixes every gateway model at once, including third-party ones. hermes (hermes/gateway.yaml, hermes/install.sh): Hermes reached the gateway by pretending it WAS OpenRouter -- the LiteLLM key in OPENROUTER_API_KEY, the gateway in OPENROUTER_BASE_URL. Requests routed fine, but fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and ignores OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model catalog -- models this gateway cannot serve -- while the gateway's own models, including cg-1's, never appeared at all. Replaced with a named custom provider. install.sh now merges gateway.yaml into ~/.hermes/config.yaml, so a fresh box gets a Hermes that can actually talk to the gateway instead of one pointed at nothing. The merge is idempotent and narrow: only the covert-gpt entry is touched, model.default/provider are seeded only when unset, and a box still on the masquerade is migrated off it (fingerprint: provider openrouter + base_url pointing at this gateway; a genuine OpenRouter user never has that, and is left alone). No secrets: gateway.yaml names the key via key_env, and mcp-config/sync.sh is what exports it. Hermes needs no context safety margin -- it omits max_tokens entirely, so vLLM bounds output by whatever context is left. Also folds in the in-flight hermes bootstrap wiring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 10:38:12 -04:00