hermes: wire into dotfiles; fix opencode's generated context limits
Two client-side fixes against the covert-gpt LiteLLM gateway, both traced from opencode throwing intermittent ContextWindowExceededError 400s. opencode (mcp-config/sync.sh): spec_for() mapped LiteLLM's /model/info onto opencode's schema 1:1, but the two mean different things by "output". LiteLLM's max_output_tokens is the whole window (vLLM caps TOTAL context, not output separately, which is correct for LiteLLM); opencode's limit.output is a RESERVE it subtracts from context to pick a compaction point. Copying one to the other told opencode to reserve the entire 262144 window, so it targeted an input of context-output and the server counted one token more than opencode did -- landing on context+1 and a hard 400, every time (measured: 196609 in + 65536 out = 262145 vs a 262144 window; the retries shrinking max_tokens by 65 never helped because the input grew to compensate). Capping the reserve alone would NOT have fixed it -- opencode would still sit exactly on the boundary, just at a different input size. So: cap the reserve (OUTPUT_RESERVE_CAP) and hold a margin back from context (CONTEXT_SAFETY_MARGIN). Generated at sync time, so it fixes every gateway model at once, including third-party ones. hermes (hermes/gateway.yaml, hermes/install.sh): Hermes reached the gateway by pretending it WAS OpenRouter -- the LiteLLM key in OPENROUTER_API_KEY, the gateway in OPENROUTER_BASE_URL. Requests routed fine, but fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and ignores OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model catalog -- models this gateway cannot serve -- while the gateway's own models, including cg-1's, never appeared at all. Replaced with a named custom provider. install.sh now merges gateway.yaml into ~/.hermes/config.yaml, so a fresh box gets a Hermes that can actually talk to the gateway instead of one pointed at nothing. The merge is idempotent and narrow: only the covert-gpt entry is touched, model.default/provider are seeded only when unset, and a box still on the masquerade is migrated off it (fingerprint: provider openrouter + base_url pointing at this gateway; a genuine OpenRouter user never has that, and is left alone). No secrets: gateway.yaml names the key via key_env, and mcp-config/sync.sh is what exports it. Hermes needs no context safety margin -- it omits max_tokens entirely, so vLLM bounds output by whatever context is left. Also folds in the in-flight hermes bootstrap wiring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -0,0 +1,55 @@
|
||||
# hermes/gateway.yaml — the covert-gpt LiteLLM gateway, as a Hermes custom
|
||||
# provider. Merged into ~/.hermes/config.yaml by hermes/install.sh.
|
||||
#
|
||||
# NO SECRETS, EVER: the key is referenced by NAME via `key_env: LITELLM_KEY`,
|
||||
# never by value. mcp-config/sync.sh exports LITELLM_KEY into every interactive
|
||||
# shell from ~/.config/mcp-sync/key (0600), and Hermes — launched from such a
|
||||
# shell — resolves it at runtime. So this file is safe in a PUBLIC repo, and
|
||||
# `hermes/install.sh` has no key to prompt for. Run mcp-config/install.sh first;
|
||||
# without it LITELLM_KEY is unset and Hermes falls back to "no-key-required",
|
||||
# which the gateway rejects with a 401.
|
||||
#
|
||||
# WHY a custom provider instead of `model.provider: openrouter` + a base_url:
|
||||
# that older shape pointed Hermes at the gateway by pretending it WAS OpenRouter
|
||||
# (OPENROUTER_API_KEY held the LiteLLM key, OPENROUTER_BASE_URL held the gateway
|
||||
# URL). Requests reached the gateway, but the MODEL PICKER did not: Hermes'
|
||||
# fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and ignores
|
||||
# OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model catalog —
|
||||
# models this gateway cannot serve — while the gateway's own models never
|
||||
# appeared. A named custom provider probes THIS base_url instead.
|
||||
#
|
||||
# NOTE ON COMMENTS: Hermes rewrites ~/.hermes/config.yaml itself (save_config)
|
||||
# and does not preserve comments, so none of this survives into the deployed
|
||||
# file. That is exactly why the reasoning lives HERE, in the repo, rather than
|
||||
# inline in the generated config.
|
||||
custom_providers:
|
||||
- name: covert-gpt
|
||||
base_url: https://llm.covert-gpt.com/v1
|
||||
key_env: LITELLM_KEY
|
||||
api_mode: chat_completions
|
||||
# `discover_models` is deliberately left at its default (true): with a key
|
||||
# present Hermes probes the gateway's /v1/models and REPLACES the picker's
|
||||
# list with the live catalog, so new gateway models show up with no edit
|
||||
# here. The `models:` map below is NOT the picker list — it exists purely to
|
||||
# carry per-model context_length overrides, which are read from config
|
||||
# independently of discovery.
|
||||
models:
|
||||
# These MUST track each node's --max-model-len in ai-k8s-iac
|
||||
# (charts/inference/values.yaml > vllm.nodes[].extraArgs). They are a
|
||||
# separate copy of that number; there is no single source across repos.
|
||||
#
|
||||
# Unlike opencode, Hermes needs NO safety margin here. opencode reserves
|
||||
# `limit.output` and targets an input of exactly context - output, landing
|
||||
# one token over the window (a hard 400); Hermes omits max_tokens entirely
|
||||
# unless model.max_tokens is set, so vLLM just bounds output by whatever
|
||||
# context remains. Do NOT add model.max_tokens to work around anything: it
|
||||
# is GLOBAL, would apply to every provider (including Anthropic), and only
|
||||
# shrinks the effective input budget Hermes compacts against.
|
||||
qwen3-vl-30b-a3b: # cg-1 — multimodal, tool calling
|
||||
context_length: 262144
|
||||
supports_vision: true
|
||||
qwen3-coder-next: # cg-2 — agentic coder
|
||||
context_length: 262144
|
||||
# Models the gateway proxies to third parties (kimi, glm, …) are
|
||||
# intentionally absent: discovery still lists them, and inventing a
|
||||
# context_length we have not verified is worse than letting Hermes probe.
|
||||
Executable
+199
@@ -0,0 +1,199 @@
|
||||
#!/usr/bin/env sh
|
||||
# hermes/install.sh — install Hermes Agent by Nous Research, and point it at
|
||||
# the covert-gpt LiteLLM gateway.
|
||||
#
|
||||
# WHY: Hermes requires Python 3.11+ and creates a self-contained venv at
|
||||
# ~/.hermes. The official installer handles the complexity of managing its
|
||||
# own uv-based Python environment. This module simply runs the official
|
||||
# installer with --skip-setup so it completes in non-interactive mode.
|
||||
# Everything under $HOME, survives Kasm redeploys.
|
||||
#
|
||||
# It then merges hermes/gateway.yaml into ~/.hermes/config.yaml, so a fresh box
|
||||
# gets a Hermes that can actually talk to the gateway. Without that step Hermes
|
||||
# installs and then points at nothing — the asymmetry mcp-config/ does not have,
|
||||
# since opencode rebuilds its config from Gitea on every shell via sync.sh.
|
||||
#
|
||||
# No secrets are handled here: gateway.yaml references the key by NAME
|
||||
# (key_env: LITELLM_KEY) and mcp-config/sync.sh exports it. Run
|
||||
# mcp-config/install.sh first.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
|
||||
REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
|
||||
|
||||
# shellcheck source=../lib/fetch.sh
|
||||
. "$REPO_ROOT/lib/fetch.sh"
|
||||
|
||||
say() { printf '%s\n' "$*" >&2; }
|
||||
|
||||
# Hermes Agent version (synced with PyPI/npm).
|
||||
HERMES_VERSION="v0.18.2"
|
||||
|
||||
# --- 1. the binary --------------------------------------------------------
|
||||
# Check if hermes is already installed at the correct version using the
|
||||
# .versions stamp pattern from bin/install.sh. NOTE: this only skips the
|
||||
# DOWNLOAD — the config merge below still runs, so a config fix ships to an
|
||||
# already-installed box on the next bootstrap.
|
||||
if [ -x "$HOME/.local/bin/hermes" ] && [ -f "$HOME/.local/bin/.versions/hermes" ] && [ "$(cat "$HOME/.local/bin/.versions/hermes")" = "$HERMES_VERSION" ]; then
|
||||
say "hermes-agent $HERMES_VERSION already installed."
|
||||
else
|
||||
say "Installing hermes-agent via official installer..."
|
||||
# Run the official installer in non-interactive mode, skipping the setup
|
||||
# wizard. The installer creates ~/.hermes/venvs/hermes with its own
|
||||
# uv-managed Python.
|
||||
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash -s -- --non-interactive --skip-setup
|
||||
|
||||
# Verify installation by checking the binary exists AND produces expected output.
|
||||
if command -v hermes >/dev/null 2>&1; then
|
||||
mkdir -p "$HOME/.local/bin/.versions"
|
||||
printf '%s\n' "$HERMES_VERSION" >"$HOME/.local/bin/.versions/hermes"
|
||||
say "Installed hermes-agent $HERMES_VERSION successfully."
|
||||
else
|
||||
say "ERROR: hermes CLI not found after install"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- 2. point it at the gateway -------------------------------------------
|
||||
# Merge hermes/gateway.yaml into ~/.hermes/config.yaml. Idempotent, and
|
||||
# deliberately NON-DESTRUCTIVE: config.yaml is a large personal file
|
||||
# (personalities, display prefs, toolsets) that Hermes also rewrites itself, so
|
||||
# this replaces ONLY the covert-gpt custom_providers entry and never rewrites
|
||||
# the rest. model.default / model.provider are seeded only when unset, so a
|
||||
# deliberate model choice is preserved across bootstraps.
|
||||
#
|
||||
# PyYAML is needed to merge. Prefer the system python3; fall back to Hermes' own
|
||||
# venv, which always has it. If neither works the config step is skipped with a
|
||||
# warning rather than failing bootstrap — the binary is still installed, and a
|
||||
# missing provider is a far smaller problem than a broken box.
|
||||
GATEWAY_YAML="$SCRIPT_DIR/gateway.yaml"
|
||||
HERMES_CONFIG="$HOME/.hermes/config.yaml"
|
||||
|
||||
PY=""
|
||||
for _c in python3 "$HOME/.hermes/venvs/hermes/bin/python" "$HOME/.hermes/hermes-agent/venv/bin/python"; do
|
||||
if command -v "$_c" >/dev/null 2>&1 && "$_c" -c 'import yaml' >/dev/null 2>&1; then PY="$_c"; break; fi
|
||||
done
|
||||
|
||||
if [ -z "$PY" ]; then
|
||||
say "WARNING: no python3 with PyYAML found — skipping Hermes gateway config."
|
||||
say " Install PyYAML and re-run $0 to finish wiring the gateway."
|
||||
elif [ ! -f "$GATEWAY_YAML" ]; then
|
||||
say "WARNING: $GATEWAY_YAML missing — skipping Hermes gateway config."
|
||||
else
|
||||
HERMES_GATEWAY_YAML="$GATEWAY_YAML" HERMES_CONFIG_PATH="$HERMES_CONFIG" "$PY" - <<'PYEOF'
|
||||
import os, sys, tempfile
|
||||
import yaml
|
||||
|
||||
gw_path = os.environ["HERMES_GATEWAY_YAML"]
|
||||
cfg_path = os.environ["HERMES_CONFIG_PATH"]
|
||||
|
||||
with open(gw_path) as f:
|
||||
gw = yaml.safe_load(f) or {}
|
||||
entries = gw.get("custom_providers") or []
|
||||
if not entries:
|
||||
print("hermes: gateway.yaml has no custom_providers — nothing to do", file=sys.stderr)
|
||||
raise SystemExit(0)
|
||||
|
||||
cfg = {}
|
||||
if os.path.exists(cfg_path):
|
||||
try:
|
||||
with open(cfg_path) as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
except Exception as e:
|
||||
# A corrupt config is the user's to fix; never overwrite it blind.
|
||||
print(f"hermes: {cfg_path} is not valid YAML ({e}) — skipping", file=sys.stderr)
|
||||
raise SystemExit(0)
|
||||
if not isinstance(cfg, dict):
|
||||
print(f"hermes: {cfg_path} is not a mapping — skipping", file=sys.stderr)
|
||||
raise SystemExit(0)
|
||||
|
||||
existing = cfg.get("custom_providers")
|
||||
if not isinstance(existing, list):
|
||||
existing = []
|
||||
|
||||
changed = []
|
||||
for entry in entries:
|
||||
name = entry.get("name")
|
||||
if not name:
|
||||
continue
|
||||
for i, cur in enumerate(existing):
|
||||
if isinstance(cur, dict) and cur.get("name") == name:
|
||||
if cur != entry:
|
||||
existing[i] = entry
|
||||
changed.append(f"updated custom_providers[{name}]")
|
||||
break
|
||||
else:
|
||||
existing.append(entry)
|
||||
changed.append(f"added custom_providers[{name}]")
|
||||
cfg["custom_providers"] = existing
|
||||
|
||||
# `default` is a BARE model id and `provider` is the custom:<name> form; both
|
||||
# are the user's to change, so they are seeded on a fresh box and otherwise left
|
||||
# alone — EXCEPT to migrate the old masquerade, below.
|
||||
primary = entries[0]["name"]
|
||||
model = cfg.get("model")
|
||||
if not isinstance(model, dict):
|
||||
model = {}
|
||||
|
||||
gw_url = (entries[0].get("base_url") or "").rstrip("/")
|
||||
cur_provider = (model.get("provider") or "").strip()
|
||||
cur_base = (model.get("base_url") or "").rstrip("/")
|
||||
|
||||
# MIGRATE the old masquerade: `provider: openrouter` + `base_url: <this
|
||||
# gateway>`. That shape routed requests here but left the picker listing
|
||||
# openrouter.ai's catalog, so leaving it in place would silently preserve the
|
||||
# exact bug this module exists to fix. `base_url` pointing at OUR gateway is the
|
||||
# fingerprint — a genuine OpenRouter user never has that. Any OTHER provider is
|
||||
# a deliberate choice and is never touched.
|
||||
if gw_url and cur_base == gw_url and cur_provider in ("", "openrouter", "custom"):
|
||||
if cur_provider != f"custom:{primary}":
|
||||
model["provider"] = f"custom:{primary}"
|
||||
changed.append(
|
||||
f"migrated model.provider {cur_provider or '<unset>'} -> custom:{primary}"
|
||||
)
|
||||
elif not cur_provider:
|
||||
model["provider"] = f"custom:{primary}"
|
||||
changed.append(f"set model.provider=custom:{primary}")
|
||||
|
||||
if not model.get("default"):
|
||||
first_model = next(iter((entries[0].get("models") or {})), None)
|
||||
if first_model:
|
||||
model["default"] = first_model
|
||||
changed.append(f"set model.default={first_model}")
|
||||
|
||||
# model.base_url is ignored once a named custom provider supplies it, and a
|
||||
# stale value here is an active footgun — drop it if it points at this gateway.
|
||||
if gw_url and cur_base == gw_url:
|
||||
model.pop("base_url", None)
|
||||
changed.append("removed dead model.base_url")
|
||||
cfg["model"] = model
|
||||
|
||||
if not changed:
|
||||
print("hermes: gateway config already current", file=sys.stderr)
|
||||
raise SystemExit(0)
|
||||
|
||||
d = os.path.dirname(cfg_path) or "."
|
||||
os.makedirs(d, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=d)
|
||||
with os.fdopen(fd, "w") as f:
|
||||
yaml.safe_dump(cfg, f, sort_keys=False, default_flow_style=False, allow_unicode=True)
|
||||
os.chmod(tmp, 0o600)
|
||||
os.replace(tmp, cfg_path)
|
||||
for c in changed:
|
||||
print(f"hermes: {c}", file=sys.stderr)
|
||||
PYEOF
|
||||
|
||||
# The provider is useless without the key. mcp-config/sync.sh exports it; warn
|
||||
# rather than fail, since bootstrap ordering may put mcp-config after this.
|
||||
if [ -z "${LITELLM_KEY:-}" ] && [ ! -f "$HOME/.config/mcp-sync/key" ]; then
|
||||
say "NOTE: LITELLM_KEY is unset and ~/.config/mcp-sync/key is missing."
|
||||
say " Run mcp-config/install.sh, or Hermes will 401 against the gateway."
|
||||
fi
|
||||
# The old OpenRouter masquerade actively breaks the model picker (it lists
|
||||
# openrouter.ai's catalog, not the gateway's), so flag it if it lingers.
|
||||
if [ -n "${OPENROUTER_API_KEY:-}" ] && [ "${OPENROUTER_API_KEY:-}" = "${LITELLM_KEY:-x}" ]; then
|
||||
say "NOTE: OPENROUTER_API_KEY is set to the LiteLLM key in your environment."
|
||||
say " That is the old masquerade; unset it in ~/.hermes/.env so the"
|
||||
say " model picker shows the gateway's models instead of openrouter.ai's."
|
||||
fi
|
||||
fi
|
||||
Reference in New Issue
Block a user