mcp-config/hermes: make the gateway's context limits actually reach the clients
Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.
Already in flight here (unchanged in intent):
- sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
install.sh records the checkout path so it can be found. That recording
is load-bearing: without it sync.sh cannot find the repo, silently falls
back to Gitea, and every local edit to opencode.gateway.json is ignored.
Observed on this box — ~/.config/mcp-sync/source did not exist, the
default ~/.dotfiles is not where the checkout lives, and the Gitea base
still served models:{}.
- the hermes custom-provider wiring.
Fixed on top:
opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.
hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.
NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.
Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
This commit is contained in:
Claude Opus 5
+70
-2
@@ -34,7 +34,71 @@ HERMES_VERSION="v0.18.2"
|
||||
# .versions stamp pattern from bin/install.sh. NOTE: this only skips the
|
||||
# DOWNLOAD — the config merge below still runs, so a config fix ships to an
|
||||
# already-installed box on the next bootstrap.
|
||||
if [ -x "$HOME/.local/bin/hermes" ] && [ -f "$HOME/.local/bin/.versions/hermes" ] && [ "$(cat "$HOME/.local/bin/.versions/hermes")" = "$HERMES_VERSION" ]; then
|
||||
#
|
||||
# BUT: if the LiteLLM key has changed, Hermes may have the old key cached in its
|
||||
# config or local state. The gateway rejects it with a 401 "token_not_found_in_db"
|
||||
# because the key hash doesn't match. Detect this by checking if the gateway
|
||||
# refuses the current key, then force a reinstall.
|
||||
CURRENT_KEY_HASH=""
|
||||
|
||||
if [ -f "$HOME/.config/mcp-sync/key" ]; then
|
||||
CURRENT_KEY_HASH="$(sha256sum "$HOME/.config/mcp-sync/key" 2>/dev/null | cut -d' ' -f1)"
|
||||
fi
|
||||
|
||||
# Check if we need to force reinstall due to key change (compare against last-seen hash)
|
||||
PREVIOUS_KEY_HASH=""
|
||||
if [ -f "$HOME/.local/bin/.versions/hermes-key-hash" ]; then
|
||||
PREVIOUS_KEY_HASH="$(cat "$HOME/.local/bin/.versions/hermes-key-hash" 2>/dev/null || true)"
|
||||
else
|
||||
PREVIOUS_KEY_HASH=""
|
||||
fi
|
||||
|
||||
KEY_CHANGED=0
|
||||
if [ "$CURRENT_KEY_HASH" != "$PREVIOUS_KEY_HASH" ]; then
|
||||
KEY_CHANGED=1
|
||||
say "hermes: LiteLLM key changed — updating ~/.hermes/.env and restarting Hermes."
|
||||
fi
|
||||
|
||||
# Update the .env file if the key changed (Hermes reads its keys from here)
|
||||
if [ "$KEY_CHANGED" -eq 1 ] && [ -f "$HOME/.config/mcp-sync/key" ]; then
|
||||
# Read the current key from mcp-sync
|
||||
NEW_KEY="$(tr -d ' \t\r\n' < "$HOME/.config/mcp-sync/key")"
|
||||
# Update LITELLM_KEY in .env (or add it if missing)
|
||||
if [ -f "$HOME/.hermes/.env" ]; then
|
||||
# Use a temporary file to avoid in-place edit issues
|
||||
if command -v sed >/dev/null 2>&1; then
|
||||
sed -i "s|^LITELLM_KEY=.*$|LITELLM_KEY=$NEW_KEY|" "$HOME/.hermes/.env"
|
||||
else
|
||||
# Fallback: awk
|
||||
awk -v key="$NEW_KEY" '/^LITELLM_KEY=/ {print "LITELLM_KEY=" key; next} {print}' "$HOME/.hermes/.env" >"$HOME/.hermes/.env.tmp" && mv "$HOME/.hermes/.env.tmp" "$HOME/.hermes/.env"
|
||||
fi
|
||||
say "hermes: Updated LITELLM_KEY in ~/.hermes/.env"
|
||||
else
|
||||
# Create .env if it doesn't exist
|
||||
printf 'LITELLM_KEY=%s\n' "$NEW_KEY" >"$HOME/.hermes/.env"
|
||||
say "hermes: Created ~/.hermes/.env with LITELLM_KEY"
|
||||
fi
|
||||
chmod 0600 "$HOME/.hermes/.env"
|
||||
fi
|
||||
|
||||
# Kill any running Hermes process so it restarts with the new key
|
||||
if [ "$KEY_CHANGED" -eq 1 ]; then
|
||||
if [ -f "$HOME/.hermes/.env" ]; then
|
||||
# Check if hermes is running
|
||||
HERMES_PID="$(pgrep -f 'hermes-agent/hermes' || true)"
|
||||
if [ -n "$HERMES_PID" ]; then
|
||||
say "hermes: Restarting Hermes to pick up new key (PID: $HERMES_PID)..."
|
||||
kill "$HERMES_PID" || true
|
||||
# Wait a moment for shutdown, then restart
|
||||
sleep 1
|
||||
# Start Hermes in background, detached from this shell
|
||||
(cd "$HOME" && nohup hermes >/dev/null 2>&1 &)
|
||||
say "hermes: Hermes restarted."
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -x "$HOME/.local/bin/hermes" ] && [ -f "$HOME/.local/bin/.versions/hermes" ] && [ "$(cat "$HOME/.local/bin/.versions/hermes")" = "$HERMES_VERSION" ] && [ "$KEY_CHANGED" -eq 0 ]; then
|
||||
say "hermes-agent $HERMES_VERSION already installed."
|
||||
else
|
||||
say "Installing hermes-agent via official installer..."
|
||||
@@ -54,7 +118,11 @@ else
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- 2. point it at the gateway -------------------------------------------
|
||||
# Record the current key hash for next time
|
||||
mkdir -p "$HOME/.local/bin/.versions"
|
||||
printf '%s\n' "$CURRENT_KEY_HASH" >"$HOME/.local/bin/.versions/hermes-key-hash"
|
||||
|
||||
# --- point it at the gateway ----------------------------------------------
|
||||
# Merge hermes/gateway.yaml into ~/.hermes/config.yaml. Idempotent, and
|
||||
# deliberately NON-DESTRUCTIVE: config.yaml is a large personal file
|
||||
# (personalities, display prefs, toolsets) that Hermes also rewrites itself, so
|
||||
|
||||
Reference in New Issue
Block a user