mcp-config/hermes: make the gateway's context limits actually reach the clients
Folds in the in-flight local-checkout work and fixes the context accounting
on top of it. Both halves are the same bug seen from two ends: clients were
sending requests larger than the backend's window and getting hard
ContextWindowExceededError 400s, intermittently, in a way that looked like
the model or the GPU misbehaving.
Already in flight here (unchanged in intent):
- sync.sh prefers the LOCAL CHECKOUT over re-downloading from Gitea, and
install.sh records the checkout path so it can be found. That recording
is load-bearing: without it sync.sh cannot find the repo, silently falls
back to Gitea, and every local edit to opencode.gateway.json is ignored.
Observed on this box — ~/.config/mcp-sync/source did not exist, the
default ~/.dotfiles is not where the checkout lives, and the Gitea base
still served models:{}.
- the hermes custom-provider wiring.
Fixed on top:
opencode had NO context limit on ANY gateway model, not a wrong one. Two
faults compounded. /model/info 403s for a day-to-day virtual key (only
management-scoped keys may read it), which sync.sh treats as best-effort and
falls back to names-only — as designed and as commented. But the merge then
did `prov["models"] = models`, replacing the map wholesale, so limits
declared in the base config were discarded too and there was no floor left.
Base specs are now a floor that live /model/info overrides field by field:
discovery still wins where present, so this adds no drift, and a 403 no
longer strips the limits. Verified: ornith 195584/32768, the qwens
261120/32768.
hermes' model map named `qwen3-vl-30b-a3b` at 262144 — a model the gateway
stopped serving when Ornith replaced it — while ornith-1.0-35b, the model
actually on cg-1 AND the only one with a window smaller than its neighbours
(196608), had no entry at all and inherited Hermes' default. The one model
needing a lower ceiling was the one not declaring one, which is exactly the
shape that fails only once a session grows.
NOTE for other boxes: hermes reads ~/.hermes/config.yaml, which install.sh
produces by MERGING gateway.yaml. Editing gateway.yaml alone changes nothing
until install.sh re-runs — this machine's live config was patched by hand.
Not addressed here, deliberately: the gateway-side context_window_fallback
(ai-k8s-iac). It is wired but inert while cg-2 is scaled down, so preventing
the oversized request client-side is what actually works today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
This commit is contained in:
Claude Opus 5
+162
-26
@@ -10,10 +10,15 @@
|
||||
# every failure path just returns, leaving the last-good config in place.
|
||||
#
|
||||
# What it does:
|
||||
# 1. Fetch the shared opencode.gateway.json from Gitea (the single source of
|
||||
# truth: the covert-gpt provider + the client-side MCP servers). The file
|
||||
# is identical for all users — EVERY credential (LiteLLM key, Gitea PAT,
|
||||
# Proxmox token) is injected via {env:VAR} and never baked into the file.
|
||||
# 1. Read the shared opencode.gateway.json (the single source of truth: the
|
||||
# covert-gpt provider + the client-side MCP servers). PREFERRED SOURCE IS
|
||||
# THE LOCAL CHECKOUT — bootstrap.sh already cloned this repo over
|
||||
# authenticated git, so re-downloading a file that is sitting on disk is
|
||||
# both redundant and, historically, the ONLY reason the repo had to be
|
||||
# world-readable. A box with no checkout falls back to the Gitea API with
|
||||
# $GITEA_KEY. The file is identical for all users — EVERY credential
|
||||
# (LiteLLM key, Gitea PAT, Proxmox token) is injected via {env:VAR} and
|
||||
# never baked into the file.
|
||||
# 2. Auto-discover models: query the gateway's /v1/models with the user's
|
||||
# key and fill the provider's "models" map. New models exposed on the
|
||||
# gateway appear on every box at the next sync — no config edit anywhere.
|
||||
@@ -37,18 +42,32 @@
|
||||
# resolves.
|
||||
|
||||
# --- config (overridable via env before sourcing) -------------------------
|
||||
MCP_SYNC_RAW_URL="${MCP_SYNC_RAW_URL:-https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config/opencode.gateway.json}"
|
||||
# The local checkout is the preferred source for BOTH the config and the
|
||||
# skills; these URLs are the no-checkout fallback only. They are Gitea *API*
|
||||
# endpoints on purpose: the web /raw/ route answers an unauthorized request
|
||||
# with a 200 and an HTML login page, which `curl -f` treats as success and
|
||||
# happily writes over the file it was fetching. The API answers with an honest
|
||||
# 401/403, so -f can be trusted.
|
||||
MCP_SYNC_API_BASE="${MCP_SYNC_API_BASE:-https://git.nic-oconnor.com/api/v1/repos/public/dotfiles}"
|
||||
MCP_SYNC_REF="${MCP_SYNC_REF:-main}"
|
||||
MCP_SYNC_RAW_URL="${MCP_SYNC_RAW_URL:-$MCP_SYNC_API_BASE/raw/mcp-config/opencode.gateway.json?ref=$MCP_SYNC_REF}"
|
||||
MCP_SYNC_HOME="${MCP_SYNC_HOME:-$HOME/.config/mcp-sync}"
|
||||
# Where the checkout is. install.sh records the tree it ran from; failing that
|
||||
# we look where bootstrap.sh clones to.
|
||||
MCP_SYNC_REPO="${MCP_SYNC_REPO:-}"
|
||||
MCP_SYNC_REPO_FILE="${MCP_SYNC_REPO_FILE:-$MCP_SYNC_HOME/source}"
|
||||
MCP_SYNC_DOTFILES_DIR="${MCP_SYNC_DOTFILES_DIR:-$HOME/.dotfiles}"
|
||||
MCP_SYNC_DEST="${MCP_SYNC_DEST:-$HOME/.config/opencode/opencode.gateway.json}"
|
||||
MCP_SYNC_KEYFILE="${MCP_SYNC_KEYFILE:-$MCP_SYNC_HOME/key}"
|
||||
# Provider whose models are auto-discovered, and the gateway base URL.
|
||||
MCP_SYNC_PROVIDER="${MCP_SYNC_PROVIDER:-covert-gpt}"
|
||||
MCP_SYNC_GATEWAY="${MCP_SYNC_GATEWAY:-https://llm.covert-gpt.com}"
|
||||
# Skills. A skill is a folder of files, so unlike the config there is no single
|
||||
# raw URL to pull — we take the repo tarball and lift out its skills/ subtree.
|
||||
# Set MCP_SYNC_SKILLS_SRC to a local directory (or file://) to sync from a
|
||||
# working checkout instead; see zsh/zshrc.local.example.
|
||||
MCP_SYNC_SKILLS_SRC="${MCP_SYNC_SKILLS_SRC:-https://git.nic-oconnor.com/public/dotfiles/archive/main.tar.gz}"
|
||||
# Skills. Taken from the checkout like the config; with no checkout we fall
|
||||
# back to the repo tarball and lift out its skills/ subtree (a skill is a folder
|
||||
# of files, so unlike the config there is no single raw URL to pull). Set
|
||||
# MCP_SYNC_SKILLS_SRC to a local directory (or file://) to pin it to one tree;
|
||||
# see zsh/zshrc.local.example.
|
||||
MCP_SYNC_SKILLS_SRC="${MCP_SYNC_SKILLS_SRC:-$MCP_SYNC_API_BASE/archive/$MCP_SYNC_REF.tar.gz}"
|
||||
MCP_SYNC_SKILLS_SUBDIR="${MCP_SYNC_SKILLS_SUBDIR:-skills}"
|
||||
MCP_SYNC_SKILLS_DEST="${MCP_SYNC_SKILLS_DEST:-$MCP_SYNC_HOME/skills}"
|
||||
# Hermes: the SAME MCP servers, rendered into Hermes' own schema. Set
|
||||
@@ -69,6 +88,67 @@ _mcp_sync_export_key() {
|
||||
[ -n "$LITELLM_KEY" ] && export LITELLM_KEY
|
||||
}
|
||||
|
||||
# --- locate the local checkout (the preferred source) ----------------------
|
||||
# Prints the repo root on stdout, or fails. Order: explicit override, the path
|
||||
# install.sh recorded, then bootstrap.sh's clone location. A directory only
|
||||
# counts if it actually holds the config, so a half-deleted tree falls through
|
||||
# to the network rather than deploying nothing.
|
||||
_mcp_sync_find_repo() {
|
||||
for _cand in "$MCP_SYNC_REPO" \
|
||||
"$([ -f "$MCP_SYNC_REPO_FILE" ] && cat "$MCP_SYNC_REPO_FILE" 2>/dev/null)" \
|
||||
"$MCP_SYNC_DOTFILES_DIR"; do
|
||||
[ -n "$_cand" ] || continue
|
||||
if [ -f "$_cand/mcp-config/opencode.gateway.json" ]; then
|
||||
printf '%s\n' "$_cand"
|
||||
unset _cand
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
unset _cand
|
||||
return 1
|
||||
}
|
||||
|
||||
# git, authenticated with the Gitea PAT when we have one, so a PRIVATE repo
|
||||
# works. The token goes through the environment rather than `git -c ...`
|
||||
# because argv is world-readable in `ps`.
|
||||
_mcp_sync_git() {
|
||||
if [ -n "${GITEA_KEY:-}" ]; then
|
||||
GIT_CONFIG_COUNT=1 \
|
||||
GIT_CONFIG_KEY_0=http.extraHeader \
|
||||
GIT_CONFIG_VALUE_0="Authorization: token $GITEA_KEY" \
|
||||
git "$@"
|
||||
else
|
||||
git "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
# Refresh the checkout so a pushed config change actually lands. ONLY for the
|
||||
# bootstrap-managed clone: a recorded DEV checkout (someone's ~/source/dotfiles)
|
||||
# must never be pulled out from under them by a shell hook. Skipped unless the
|
||||
# tree is clean and sitting on its default branch; quiet and non-fatal always.
|
||||
_mcp_sync_pull() {
|
||||
[ "$1" = "$MCP_SYNC_DOTFILES_DIR" ] || return 0
|
||||
[ -d "$1/.git" ] || return 0
|
||||
command -v git >/dev/null 2>&1 || return 0
|
||||
[ -z "$(git -C "$1" status --porcelain 2>/dev/null)" ] || return 0
|
||||
_branch="$(git -C "$1" symbolic-ref --short -q HEAD 2>/dev/null)" || return 0
|
||||
_default="$(git -C "$1" symbolic-ref --short -q refs/remotes/origin/HEAD 2>/dev/null)"
|
||||
_default="${_default#origin/}"
|
||||
[ -n "$_default" ] || _default=main
|
||||
[ "$_branch" = "$_default" ] || { unset _branch _default; return 0; }
|
||||
unset _branch _default
|
||||
_mcp_sync_git -C "$1" pull --ff-only --quiet >/dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
# curl, with the Gitea PAT when we have one. Used only on the no-checkout path.
|
||||
_mcp_sync_curl() {
|
||||
if [ -n "${GITEA_KEY:-}" ]; then
|
||||
curl -fsS -m "${3:-8}" -H "Authorization: token $GITEA_KEY" "$1" -o "$2" 2>/dev/null
|
||||
else
|
||||
curl -fsS -m "${3:-8}" "$1" -o "$2" 2>/dev/null
|
||||
fi
|
||||
}
|
||||
|
||||
# --- refresh config + discover models (throttled, non-fatal) ---------------
|
||||
_mcp_sync_refresh() {
|
||||
mkdir -p "$(dirname "$MCP_SYNC_DEST")" 2>/dev/null || return 0
|
||||
@@ -81,16 +161,42 @@ _mcp_sync_refresh() {
|
||||
fi
|
||||
fi
|
||||
|
||||
command -v curl >/dev/null 2>&1 || return 0
|
||||
# 1. Get the shared base config. THE CHECKOUT WINS: it is already on disk,
|
||||
# was fetched over authenticated git, and needs no network at all. Only a
|
||||
# box with no checkout goes to Gitea. On total failure we fall back to the
|
||||
# currently deployed file so model discovery can still refresh it.
|
||||
_base=""
|
||||
_repo="$(_mcp_sync_find_repo)" || _repo=""
|
||||
if [ -n "$_repo" ]; then
|
||||
_mcp_sync_pull "$_repo"
|
||||
if [ -s "$_repo/mcp-config/opencode.gateway.json" ]; then
|
||||
_base="$(mktemp 2>/dev/null)" || return 0
|
||||
cp "$_repo/mcp-config/opencode.gateway.json" "$_base" 2>/dev/null ||
|
||||
{ rm -f "$_base"; _base=""; }
|
||||
fi
|
||||
fi
|
||||
|
||||
# 1. Fetch the shared base config. On failure fall back to the currently
|
||||
# deployed file so model discovery can still refresh it.
|
||||
_base="$(mktemp 2>/dev/null)" || return 0
|
||||
# -f: fail on HTTP error; -s: silent; -m: hard timeout so a hung network
|
||||
# can't stall the shell.
|
||||
if ! curl -fsS -m 8 "$MCP_SYNC_RAW_URL" -o "$_base" 2>/dev/null || [ ! -s "$_base" ]; then
|
||||
rm -f "$_base"; _base=""
|
||||
[ -f "$MCP_SYNC_DEST" ] || return 0
|
||||
if [ -z "$_base" ]; then
|
||||
command -v curl >/dev/null 2>&1 || return 0
|
||||
_base="$(mktemp 2>/dev/null)" || return 0
|
||||
if ! _mcp_sync_curl "$MCP_SYNC_RAW_URL" "$_base" || [ ! -s "$_base" ]; then
|
||||
rm -f "$_base"; _base=""
|
||||
[ -f "$MCP_SYNC_DEST" ] || return 0
|
||||
elif [ "$(head -c 1 "$_base")" != "{" ]; then
|
||||
# A 200 that is not the config. The API endpoints we default to return an
|
||||
# honest 401/403, so this only fires for an override still pointing at
|
||||
# Gitea's web /raw/ route, which answers with a 200 HTML login page that
|
||||
# -f cannot detect. The python step below would reject it and keep the
|
||||
# last-good file — correct, but SILENTLY, leaving the config frozen with
|
||||
# nothing to show for it. Say so instead.
|
||||
rm -f "$_base"; _base=""
|
||||
printf 'mcp-sync: %s did not return JSON (Gitea login page?).\n' "$MCP_SYNC_RAW_URL" >&2
|
||||
printf 'mcp-sync: set GITEA_KEY, or point MCP_SYNC_REPO at a checkout.\n' >&2
|
||||
# Bump the deployed file's mtime so the TTL throttles this warning to once
|
||||
# an hour instead of printing it at the top of every single shell.
|
||||
[ -f "$MCP_SYNC_DEST" ] && touch "$MCP_SYNC_DEST" 2>/dev/null
|
||||
[ -f "$MCP_SYNC_DEST" ] || return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# 2. Discover the gateway's models with the user's key (best-effort).
|
||||
@@ -189,10 +295,26 @@ try:
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Per-model specs declared in the BASE config are a FLOOR; live /model/info
|
||||
# overrides them field by field. This matters because the /model/info fetch
|
||||
# above is best-effort and 403s for any key not scoped to management routes —
|
||||
# which is the normal case for a day-to-day virtual key. Before this merge that
|
||||
# fallback silently produced `{}` for every model, i.e. no context limit at
|
||||
# all, and opencode then ran the whole gateway on its built-in defaults and
|
||||
# walked straight into ContextWindowExceededError 400s on the one model whose
|
||||
# window is smaller than those defaults. Declaring limits in the base file did
|
||||
# not help either, because `prov["models"] = models` replaced the map wholesale.
|
||||
# Live data still wins wherever it is present, so this reintroduces no drift.
|
||||
base_models = {}
|
||||
try:
|
||||
base_models = (cfg.get("provider", {}).get(provider) or {}).get("models") or {}
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
models = {}
|
||||
try:
|
||||
data = json.loads(os.environ.get("MCP_SYNC_MODELS") or "{}")
|
||||
models = {m["id"]: dict(info.get(m["id"], {}))
|
||||
models = {m["id"]: {**(base_models.get(m["id"]) or {}), **dict(info.get(m["id"], {}))}
|
||||
for m in data.get("data", []) if m.get("id")}
|
||||
except Exception:
|
||||
pass
|
||||
@@ -393,8 +515,13 @@ _mcp_sync_skills() {
|
||||
_tree="${MCP_SYNC_SKILLS_SRC#file://}"
|
||||
;;
|
||||
*)
|
||||
if command -v curl >/dev/null 2>&1 && command -v tar >/dev/null 2>&1 &&
|
||||
curl -fsS -m 15 "$MCP_SYNC_SKILLS_SRC" -o "$_work/repo.tgz" 2>/dev/null &&
|
||||
# Same rule as the config: the checkout is the source of truth when there
|
||||
# is one, so no tarball download and no anonymous read required.
|
||||
_repo="$(_mcp_sync_find_repo)" || _repo=""
|
||||
if [ -n "$_repo" ] && [ -d "$_repo/$MCP_SYNC_SKILLS_SUBDIR" ]; then
|
||||
_tree="$_repo/$MCP_SYNC_SKILLS_SUBDIR"
|
||||
elif command -v curl >/dev/null 2>&1 && command -v tar >/dev/null 2>&1 &&
|
||||
_mcp_sync_curl "$MCP_SYNC_SKILLS_SRC" "$_work/repo.tgz" 15 &&
|
||||
[ -s "$_work/repo.tgz" ] &&
|
||||
mkdir -p "$_work/x" && tar -xzf "$_work/repo.tgz" -C "$_work/x" 2>/dev/null
|
||||
then
|
||||
@@ -440,10 +567,14 @@ _mcp_sync_skills
|
||||
# as "my new model isn't there" when in fact the gateway is fine and the local
|
||||
# config is just stale. Run `mcp-sync` to pull it immediately.
|
||||
#
|
||||
# Defined for the interactive shell (zsh/bash), which is the only place sync.sh
|
||||
# is sourced; the hyphen in the name is fine there.
|
||||
# The name is `mcp-sync`, but the DEFINITION below uses an underscore: a hyphen
|
||||
# is not a legal function name in POSIX sh, and dash rejects it at PARSE time --
|
||||
# i.e. it aborts the whole file, not just this line. install.sh sources sync.sh
|
||||
# under /bin/sh (dash on Debian-family), so a bare `mcp-sync() {` here silently
|
||||
# killed install.sh's "deploy the config now" step for every install. An alias
|
||||
# is resolved lazily, so it costs nothing to parse and keeps the typed name.
|
||||
MCP_SYNC_SELF="${MCP_SYNC_SELF:-$HOME/.config/mcp-sync/sync.sh}"
|
||||
mcp-sync() {
|
||||
mcp_sync() {
|
||||
[ -f "$MCP_SYNC_SELF" ] || {
|
||||
printf 'mcp-sync: %s not found\n' "$MCP_SYNC_SELF" >&2; return 1
|
||||
}
|
||||
@@ -459,5 +590,10 @@ models = cfg.get("provider", {}).get(os.environ["MCP_SYNC_PY_PROVIDER"], {}).get
|
||||
print(f"mcp-sync: {len(models)} models -> {', '.join(models) or '(none)'}")
|
||||
PYEOF
|
||||
}
|
||||
alias mcp-sync=mcp_sync 2>/dev/null || true
|
||||
|
||||
unset -f _mcp_sync_export_key _mcp_sync_refresh _mcp_sync_hermes _mcp_sync_skills 2>/dev/null || true
|
||||
unset -f _mcp_sync_export_key _mcp_sync_refresh _mcp_sync_hermes _mcp_sync_skills \
|
||||
_mcp_sync_find_repo _mcp_sync_pull _mcp_sync_curl _mcp_sync_git 2>/dev/null || true
|
||||
# This file is SOURCED into the user's interactive shell, so scratch variables
|
||||
# would otherwise linger in their environment for the rest of the session.
|
||||
unset _base _repo _models _info _work _tree _cand _branch _default 2>/dev/null || true
|
||||
|
||||
Reference in New Issue
Block a user