mcp-config/hermes: make the gateway's context limits actually reach the clients
Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.
Already in flight here (unchanged in intent):
- sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
install.sh records the checkout path so it can be found. That recording
is load-bearing: without it sync.sh cannot find the repo, silently falls
back to Gitea, and every local edit to opencode.gateway.json is ignored.
Observed on this box — ~/.config/mcp-sync/source did not exist, the
default ~/.dotfiles is not where the checkout lives, and the Gitea base
still served models:{}.
- the hermes custom-provider wiring.
Fixed on top:
opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.
hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.
NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.
Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
This commit is contained in:
Claude Opus 5
+97
-4
@@ -14,7 +14,14 @@
|
||||
# After this, every new shell refreshes the gateway config automatically.
|
||||
set -eu
|
||||
|
||||
RAW_BASE="${MCP_SYNC_RAW_BASE:-https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config}"
|
||||
# Fallback source, used ONLY when this script is not run from a checkout.
|
||||
# A Gitea *API* endpoint, not the web /raw/ route: /raw/ answers an
|
||||
# unauthorized request with a 200 and an HTML login page, which `curl -f`
|
||||
# reports as success. The API returns an honest 401/403. Set GITEA_KEY to read
|
||||
# a private repo.
|
||||
API_BASE="${MCP_SYNC_API_BASE:-https://git.nic-oconnor.com/api/v1/repos/public/dotfiles}"
|
||||
REF="${MCP_SYNC_REF:-main}"
|
||||
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd 2>/dev/null)" || SCRIPT_DIR=""
|
||||
SYNC_HOME="$HOME/.config/mcp-sync"
|
||||
SYNC_SH="$SYNC_HOME/sync.sh"
|
||||
KEYFILE="$SYNC_HOME/key"
|
||||
@@ -26,12 +33,66 @@ command -v curl >/dev/null 2>&1 || { say "error: curl is required"; exit 1; }
|
||||
|
||||
mkdir -p "$SYNC_HOME"
|
||||
|
||||
# --- 1. fetch the sync hook ------------------------------------------------
|
||||
say "Fetching mcp-sync hook..."
|
||||
curl -fsSL -m 15 "$RAW_BASE/sync.sh" -o "$SYNC_SH"
|
||||
# --- 1. install the sync hook ----------------------------------------------
|
||||
# Prefer the copy sitting next to this script: when bootstrap.sh runs the
|
||||
# module from a checkout there is no reason to go to the network, and it makes
|
||||
# the module work on a box that cannot reach Gitea at all.
|
||||
if [ -n "$SCRIPT_DIR" ] && [ -f "$SCRIPT_DIR/sync.sh" ]; then
|
||||
say "Installing mcp-sync hook from $SCRIPT_DIR/sync.sh"
|
||||
cp "$SCRIPT_DIR/sync.sh" "$SYNC_SH"
|
||||
else
|
||||
SYNC_URL="$API_BASE/raw/mcp-config/sync.sh?ref=$REF"
|
||||
say "Fetching mcp-sync hook..."
|
||||
# Fetch to a temp file and CHECK IT before overwriting the installed hook.
|
||||
# The API endpoint gives an honest 4xx, but keep the shebang check as a
|
||||
# backstop: if anything ever writes an HTML login page over sync.sh, the shell
|
||||
# fails to parse it, the hook silently does nothing, OPENCODE_CONFIG is never
|
||||
# exported, and opencode comes up with no covert-gpt provider at all.
|
||||
_tmp="$(mktemp)" || { say "error: mktemp failed"; exit 1; }
|
||||
if [ -n "${GITEA_KEY:-}" ]; then
|
||||
set -- -H "Authorization: token $GITEA_KEY"
|
||||
else
|
||||
set --
|
||||
fi
|
||||
if ! curl -fsSL -m 15 "$@" "$SYNC_URL" -o "$_tmp" || [ ! -s "$_tmp" ]; then
|
||||
rm -f "$_tmp"
|
||||
say "error: could not fetch $SYNC_URL"
|
||||
[ -n "${GITEA_KEY:-}" ] ||
|
||||
say " The repo may be private — export GITEA_KEY and re-run."
|
||||
exit 1
|
||||
fi
|
||||
case "$(head -c 2 "$_tmp")" in
|
||||
'#!') mv "$_tmp" "$SYNC_SH" ;;
|
||||
*)
|
||||
rm -f "$_tmp"
|
||||
say "error: $SYNC_URL did not return a script."
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
chmod 0644 "$SYNC_SH"
|
||||
|
||||
# --- 1b. record the checkout as the config source --------------------------
|
||||
# THE POINT OF THIS FILE: bootstrap.sh already cloned this repo over
|
||||
# authenticated git, so the per-shell hook must never have to re-fetch the
|
||||
# gateway config anonymously over HTTP. Needing that was the only reason the
|
||||
# repo ever had to be world-readable. Record where we ran from and sync.sh
|
||||
# reads both the config and the skills straight off disk.
|
||||
if [ -n "$SCRIPT_DIR" ] && [ -f "$SCRIPT_DIR/opencode.gateway.json" ]; then
|
||||
if REPO_DIR="$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd 2>/dev/null)"; then
|
||||
printf '%s\n' "$REPO_DIR" > "$SYNC_HOME/source"
|
||||
say "Config source: $REPO_DIR (local checkout, no network needed)."
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- 2. capture the LiteLLM key -------------------------------------------
|
||||
# An already-stored key is as good as one in the env: bootstrap.sh re-runs this
|
||||
# module on every pass, and prompting a box that is already configured would
|
||||
# make an otherwise unattended bootstrap hang on a TTY read.
|
||||
if [ "${LITELLM_KEY:-}" = "" ] && [ -f "$KEYFILE" ]; then
|
||||
LITELLM_KEY="$(cat "$KEYFILE" 2>/dev/null || true)"
|
||||
[ -n "$LITELLM_KEY" ] && say "Reusing the key already stored at $KEYFILE."
|
||||
fi
|
||||
if [ "${LITELLM_KEY:-}" = "" ]; then
|
||||
# Read from the controlling TTY since stdin is the piped script.
|
||||
if [ -r /dev/tty ]; then
|
||||
@@ -83,6 +144,38 @@ fi
|
||||
# shellcheck disable=SC1090
|
||||
. "$SYNC_SH" || true
|
||||
|
||||
# If this is the initial install (key just written now), restart Hermes.
|
||||
# The key file mtime changed, but we need to compare hashes to know if it's
|
||||
# actually new (not just a touch). Only restart if we're not in a re-run.
|
||||
if [ -f "$KEYFILE" ]; then
|
||||
KEY_HASH="$(sha256sum "$KEYFILE" 2>/dev/null | cut -d' ' -f1)"
|
||||
PREV_KEY_HASH=""
|
||||
if [ -f "$HOME/.config/mcp-sync/.key-hash" ]; then
|
||||
PREV_KEY_HASH="$(cat "$HOME/.config/mcp-sync/.key-hash" 2>/dev/null || true)"
|
||||
fi
|
||||
if [ "$KEY_HASH" != "$PREV_KEY_HASH" ]; then
|
||||
# Key changed - update .env and restart Hermes if running
|
||||
if [ -f "$HOME/.hermes/.env" ]; then
|
||||
say "hermes: Key changed during install — updating ~/.hermes/.env"
|
||||
if command -v sed >/dev/null 2>&1; then
|
||||
sed -i "s|^LITELLM_KEY=.*$|LITELLM_KEY=$LITELLM_KEY|" "$HOME/.hermes/.env"
|
||||
else
|
||||
awk -v key="$LITELLM_KEY" '/^LITELLM_KEY=/ {print "LITELLM_KEY=" key; next} {print}' "$HOME/.hermes/.env" >"$HOME/.hermes/.env.tmp" && mv "$HOME/.hermes/.env.tmp" "$HOME/.hermes/.env"
|
||||
fi
|
||||
chmod 0600 "$HOME/.hermes/.env"
|
||||
fi
|
||||
if pgrep -f 'hermes-agent/hermes' >/dev/null 2>&1; then
|
||||
HERMES_PID="$(pgrep -f 'hermes-agent/hermes')"
|
||||
say "hermes: Restarting to pick up new key (PID: $HERMES_PID)..."
|
||||
kill "$HERMES_PID" || true
|
||||
sleep 1
|
||||
(cd "$HOME" && nohup hermes >/dev/null 2>&1 &)
|
||||
say "hermes: Hermes restarted."
|
||||
fi
|
||||
fi
|
||||
printf '%s\n' "$KEY_HASH" >"$HOME/.config/mcp-sync/.key-hash"
|
||||
fi
|
||||
|
||||
say ""
|
||||
say "Done. Open a new terminal (or 'source $RC') and run opencode."
|
||||
say "The gateway config refreshes automatically on each new shell."
|
||||
|
||||
Reference in New Issue
Block a user