hermes: wire into dotfiles; fix opencode's generated context limits

Two client-side fixes against the covert-gpt LiteLLM gateway, both traced
from opencode throwing intermittent ContextWindowExceededError 400s.

opencode (mcp-config/sync.sh):
  spec_for() mapped LiteLLM's /model/info onto opencode's schema 1:1, but the
  two mean different things by "output". LiteLLM's max_output_tokens is the
  whole window (vLLM caps TOTAL context, not output separately, which is
  correct for LiteLLM); opencode's limit.output is a RESERVE it subtracts from
  context to pick a compaction point. Copying one to the other told opencode to
  reserve the entire 262144 window, so it targeted an input of context-output
  and the server counted one token more than opencode did -- landing on
  context+1 and a hard 400, every time (measured: 196609 in + 65536 out =
  262145 vs a 262144 window; the retries shrinking max_tokens by 65 never
  helped because the input grew to compensate).

  Capping the reserve alone would NOT have fixed it -- opencode would still sit
  exactly on the boundary, just at a different input size. So: cap the reserve
  (OUTPUT_RESERVE_CAP) and hold a margin back from context
  (CONTEXT_SAFETY_MARGIN). Generated at sync time, so it fixes every gateway
  model at once, including third-party ones.

hermes (hermes/gateway.yaml, hermes/install.sh):
  Hermes reached the gateway by pretending it WAS OpenRouter -- the LiteLLM key
  in OPENROUTER_API_KEY, the gateway in OPENROUTER_BASE_URL. Requests routed
  fine, but fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and
  ignores OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model
  catalog -- models this gateway cannot serve -- while the gateway's own models,
  including cg-1's, never appeared at all.

  Replaced with a named custom provider. install.sh now merges gateway.yaml into
  ~/.hermes/config.yaml, so a fresh box gets a Hermes that can actually talk to
  the gateway instead of one pointed at nothing. The merge is idempotent and
  narrow: only the covert-gpt entry is touched, model.default/provider are
  seeded only when unset, and a box still on the masquerade is migrated off it
  (fingerprint: provider openrouter + base_url pointing at this gateway; a
  genuine OpenRouter user never has that, and is left alone).

  No secrets: gateway.yaml names the key via key_env, and mcp-config/sync.sh is
  what exports it. Hermes needs no context safety margin -- it omits max_tokens
  entirely, so vLLM bounds output by whatever context is left.

Also folds in the in-flight hermes bootstrap wiring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

README.md

@@ -101,6 +101,20 @@ Variables modules look for:
 
   These are NOT used by `ai-k8s-iac` / `task up`: the LiteLLM gateway hosts no
   MCP servers and stores no MCP credentials. See `mcp-config/README.md`.
+- `hermes/` — installs Hermes Agent (Nous Research's self-improving AI agent)
+  via npm. Hermes uses npx to spawn and downloads the Python hermes-agent from
+  PyPI. Everything under $HOME, so it survives Kasm redeploys.
+
+  It then merges `hermes/gateway.yaml` into `~/.hermes/config.yaml`, wiring
+  Hermes to the covert-gpt LiteLLM gateway as a named custom provider. The merge
+  is idempotent and touches only that provider entry, so the rest of your
+  personal config is left alone; `model.default` / `model.provider` are seeded
+  only on a fresh box. Unlike `mcp-config/`, this deploys at BOOTSTRAP time, not
+  per-shell — re-run `hermes/install.sh` to pick up a gateway.yaml change.
+
+  Depends on `mcp-config/install.sh` having run: gateway.yaml names the key
+  (`key_env: LITELLM_KEY`) but never contains it, and `mcp-config/sync.sh` is
+  what exports LITELLM_KEY into your shell. Without it Hermes 401s.
 
 Secrets never go in this repo or in `zsh/zshrc` — that file is shared
 config applied to every box.

bootstrap.sh

@@ -66,6 +66,9 @@ run_module bin
 # Provides npx, which opencode needs to spawn the npm-based MCP servers in
 # mcp-config/. Without it every MCP server silently fails to start.
 run_module node
+# Hermes Agent (Nous Research) is distributed as an npm package with a Python
+# dependency. Runs after node/ ensures npx is available.
+run_module hermes
 run_module ansible
 # Runs last: needs rootless Docker already up (started at boot) so it can
 # delegate cgroups to it. No-ops where that does not apply.

hermes/gateway.yaml

@@ -0,0 +1,55 @@
+# hermes/gateway.yaml — the covert-gpt LiteLLM gateway, as a Hermes custom
+# provider. Merged into ~/.hermes/config.yaml by hermes/install.sh.
+#
+# NO SECRETS, EVER: the key is referenced by NAME via `key_env: LITELLM_KEY`,
+# never by value. mcp-config/sync.sh exports LITELLM_KEY into every interactive
+# shell from ~/.config/mcp-sync/key (0600), and Hermes — launched from such a
+# shell — resolves it at runtime. So this file is safe in a PUBLIC repo, and
+# `hermes/install.sh` has no key to prompt for. Run mcp-config/install.sh first;
+# without it LITELLM_KEY is unset and Hermes falls back to "no-key-required",
+# which the gateway rejects with a 401.
+#
+# WHY a custom provider instead of `model.provider: openrouter` + a base_url:
+# that older shape pointed Hermes at the gateway by pretending it WAS OpenRouter
+# (OPENROUTER_API_KEY held the LiteLLM key, OPENROUTER_BASE_URL held the gateway
+# URL). Requests reached the gateway, but the MODEL PICKER did not: Hermes'
+# fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and ignores
+# OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model catalog —
+# models this gateway cannot serve — while the gateway's own models never
+# appeared. A named custom provider probes THIS base_url instead.
+#
+# NOTE ON COMMENTS: Hermes rewrites ~/.hermes/config.yaml itself (save_config)
+# and does not preserve comments, so none of this survives into the deployed
+# file. That is exactly why the reasoning lives HERE, in the repo, rather than
+# inline in the generated config.
+custom_providers:
+  - name: covert-gpt
+    base_url: https://llm.covert-gpt.com/v1
+    key_env: LITELLM_KEY
+    api_mode: chat_completions
+    # `discover_models` is deliberately left at its default (true): with a key
+    # present Hermes probes the gateway's /v1/models and REPLACES the picker's
+    # list with the live catalog, so new gateway models show up with no edit
+    # here. The `models:` map below is NOT the picker list — it exists purely to
+    # carry per-model context_length overrides, which are read from config
+    # independently of discovery.
+    models:
+      # These MUST track each node's --max-model-len in ai-k8s-iac
+      # (charts/inference/values.yaml > vllm.nodes[].extraArgs). They are a
+      # separate copy of that number; there is no single source across repos.
+      #
+      # Unlike opencode, Hermes needs NO safety margin here. opencode reserves
+      # `limit.output` and targets an input of exactly context - output, landing
+      # one token over the window (a hard 400); Hermes omits max_tokens entirely
+      # unless model.max_tokens is set, so vLLM just bounds output by whatever
+      # context remains. Do NOT add model.max_tokens to work around anything: it
+      # is GLOBAL, would apply to every provider (including Anthropic), and only
+      # shrinks the effective input budget Hermes compacts against.
+      qwen3-vl-30b-a3b: # cg-1 — multimodal, tool calling
+        context_length: 262144
+        supports_vision: true
+      qwen3-coder-next: # cg-2 — agentic coder
+        context_length: 262144
+      # Models the gateway proxies to third parties (kimi, glm, …) are
+      # intentionally absent: discovery still lists them, and inventing a
+      # context_length we have not verified is worse than letting Hermes probe.

hermes/install.sh

@@ -0,0 +1,199 @@
+#!/usr/bin/env sh
+# hermes/install.sh — install Hermes Agent by Nous Research, and point it at
+# the covert-gpt LiteLLM gateway.
+#
+# WHY: Hermes requires Python 3.11+ and creates a self-contained venv at
+# ~/.hermes. The official installer handles the complexity of managing its
+# own uv-based Python environment. This module simply runs the official
+# installer with --skip-setup so it completes in non-interactive mode.
+# Everything under $HOME, survives Kasm redeploys.
+#
+# It then merges hermes/gateway.yaml into ~/.hermes/config.yaml, so a fresh box
+# gets a Hermes that can actually talk to the gateway. Without that step Hermes
+# installs and then points at nothing — the asymmetry mcp-config/ does not have,
+# since opencode rebuilds its config from Gitea on every shell via sync.sh.
+#
+# No secrets are handled here: gateway.yaml references the key by NAME
+# (key_env: LITELLM_KEY) and mcp-config/sync.sh exports it. Run
+# mcp-config/install.sh first.
+set -eu
+
+SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
+REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
+
+# shellcheck source=../lib/fetch.sh
+. "$REPO_ROOT/lib/fetch.sh"
+
+say() { printf '%s\n' "$*" >&2; }
+
+# Hermes Agent version (synced with PyPI/npm).
+HERMES_VERSION="v0.18.2"
+
+# --- 1. the binary --------------------------------------------------------
+# Check if hermes is already installed at the correct version using the
+# .versions stamp pattern from bin/install.sh. NOTE: this only skips the
+# DOWNLOAD — the config merge below still runs, so a config fix ships to an
+# already-installed box on the next bootstrap.
+if [ -x "$HOME/.local/bin/hermes" ] && [ -f "$HOME/.local/bin/.versions/hermes" ] && [ "$(cat "$HOME/.local/bin/.versions/hermes")" = "$HERMES_VERSION" ]; then
+  say "hermes-agent $HERMES_VERSION already installed."
+else
+  say "Installing hermes-agent via official installer..."
+  # Run the official installer in non-interactive mode, skipping the setup
+  # wizard. The installer creates ~/.hermes/venvs/hermes with its own
+  # uv-managed Python.
+  curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash -s -- --non-interactive --skip-setup
+
+  # Verify installation by checking the binary exists AND produces expected output.
+  if command -v hermes >/dev/null 2>&1; then
+    mkdir -p "$HOME/.local/bin/.versions"
+    printf '%s\n' "$HERMES_VERSION" >"$HOME/.local/bin/.versions/hermes"
+    say "Installed hermes-agent $HERMES_VERSION successfully."
+  else
+    say "ERROR: hermes CLI not found after install"
+    exit 1
+  fi
+fi
+
+# --- 2. point it at the gateway -------------------------------------------
+# Merge hermes/gateway.yaml into ~/.hermes/config.yaml. Idempotent, and
+# deliberately NON-DESTRUCTIVE: config.yaml is a large personal file
+# (personalities, display prefs, toolsets) that Hermes also rewrites itself, so
+# this replaces ONLY the covert-gpt custom_providers entry and never rewrites
+# the rest. model.default / model.provider are seeded only when unset, so a
+# deliberate model choice is preserved across bootstraps.
+#
+# PyYAML is needed to merge. Prefer the system python3; fall back to Hermes' own
+# venv, which always has it. If neither works the config step is skipped with a
+# warning rather than failing bootstrap — the binary is still installed, and a
+# missing provider is a far smaller problem than a broken box.
+GATEWAY_YAML="$SCRIPT_DIR/gateway.yaml"
+HERMES_CONFIG="$HOME/.hermes/config.yaml"
+
+PY=""
+for _c in python3 "$HOME/.hermes/venvs/hermes/bin/python" "$HOME/.hermes/hermes-agent/venv/bin/python"; do
+  if command -v "$_c" >/dev/null 2>&1 && "$_c" -c 'import yaml' >/dev/null 2>&1; then PY="$_c"; break; fi
+done
+
+if [ -z "$PY" ]; then
+  say "WARNING: no python3 with PyYAML found — skipping Hermes gateway config."
+  say "         Install PyYAML and re-run $0 to finish wiring the gateway."
+elif [ ! -f "$GATEWAY_YAML" ]; then
+  say "WARNING: $GATEWAY_YAML missing — skipping Hermes gateway config."
+else
+  HERMES_GATEWAY_YAML="$GATEWAY_YAML" HERMES_CONFIG_PATH="$HERMES_CONFIG" "$PY" - <<'PYEOF'
+import os, sys, tempfile
+import yaml
+
+gw_path = os.environ["HERMES_GATEWAY_YAML"]
+cfg_path = os.environ["HERMES_CONFIG_PATH"]
+
+with open(gw_path) as f:
+    gw = yaml.safe_load(f) or {}
+entries = gw.get("custom_providers") or []
+if not entries:
+    print("hermes: gateway.yaml has no custom_providers — nothing to do", file=sys.stderr)
+    raise SystemExit(0)
+
+cfg = {}
+if os.path.exists(cfg_path):
+    try:
+        with open(cfg_path) as f:
+            cfg = yaml.safe_load(f) or {}
+    except Exception as e:
+        # A corrupt config is the user's to fix; never overwrite it blind.
+        print(f"hermes: {cfg_path} is not valid YAML ({e}) — skipping", file=sys.stderr)
+        raise SystemExit(0)
+if not isinstance(cfg, dict):
+    print(f"hermes: {cfg_path} is not a mapping — skipping", file=sys.stderr)
+    raise SystemExit(0)
+
+existing = cfg.get("custom_providers")
+if not isinstance(existing, list):
+    existing = []
+
+changed = []
+for entry in entries:
+    name = entry.get("name")
+    if not name:
+        continue
+    for i, cur in enumerate(existing):
+        if isinstance(cur, dict) and cur.get("name") == name:
+            if cur != entry:
+                existing[i] = entry
+                changed.append(f"updated custom_providers[{name}]")
+            break
+    else:
+        existing.append(entry)
+        changed.append(f"added custom_providers[{name}]")
+cfg["custom_providers"] = existing
+
+# `default` is a BARE model id and `provider` is the custom:<name> form; both
+# are the user's to change, so they are seeded on a fresh box and otherwise left
+# alone — EXCEPT to migrate the old masquerade, below.
+primary = entries[0]["name"]
+model = cfg.get("model")
+if not isinstance(model, dict):
+    model = {}
+
+gw_url = (entries[0].get("base_url") or "").rstrip("/")
+cur_provider = (model.get("provider") or "").strip()
+cur_base = (model.get("base_url") or "").rstrip("/")
+
+# MIGRATE the old masquerade: `provider: openrouter` + `base_url: <this
+# gateway>`. That shape routed requests here but left the picker listing
+# openrouter.ai's catalog, so leaving it in place would silently preserve the
+# exact bug this module exists to fix. `base_url` pointing at OUR gateway is the
+# fingerprint — a genuine OpenRouter user never has that. Any OTHER provider is
+# a deliberate choice and is never touched.
+if gw_url and cur_base == gw_url and cur_provider in ("", "openrouter", "custom"):
+    if cur_provider != f"custom:{primary}":
+        model["provider"] = f"custom:{primary}"
+        changed.append(
+            f"migrated model.provider {cur_provider or '<unset>'} -> custom:{primary}"
+        )
+elif not cur_provider:
+    model["provider"] = f"custom:{primary}"
+    changed.append(f"set model.provider=custom:{primary}")
+
+if not model.get("default"):
+    first_model = next(iter((entries[0].get("models") or {})), None)
+    if first_model:
+        model["default"] = first_model
+        changed.append(f"set model.default={first_model}")
+
+# model.base_url is ignored once a named custom provider supplies it, and a
+# stale value here is an active footgun — drop it if it points at this gateway.
+if gw_url and cur_base == gw_url:
+    model.pop("base_url", None)
+    changed.append("removed dead model.base_url")
+cfg["model"] = model
+
+if not changed:
+    print("hermes: gateway config already current", file=sys.stderr)
+    raise SystemExit(0)
+
+d = os.path.dirname(cfg_path) or "."
+os.makedirs(d, exist_ok=True)
+fd, tmp = tempfile.mkstemp(dir=d)
+with os.fdopen(fd, "w") as f:
+    yaml.safe_dump(cfg, f, sort_keys=False, default_flow_style=False, allow_unicode=True)
+os.chmod(tmp, 0o600)
+os.replace(tmp, cfg_path)
+for c in changed:
+    print(f"hermes: {c}", file=sys.stderr)
+PYEOF
+
+  # The provider is useless without the key. mcp-config/sync.sh exports it; warn
+  # rather than fail, since bootstrap ordering may put mcp-config after this.
+  if [ -z "${LITELLM_KEY:-}" ] && [ ! -f "$HOME/.config/mcp-sync/key" ]; then
+    say "NOTE: LITELLM_KEY is unset and ~/.config/mcp-sync/key is missing."
+    say "      Run mcp-config/install.sh, or Hermes will 401 against the gateway."
+  fi
+  # The old OpenRouter masquerade actively breaks the model picker (it lists
+  # openrouter.ai's catalog, not the gateway's), so flag it if it lingers.
+  if [ -n "${OPENROUTER_API_KEY:-}" ] && [ "${OPENROUTER_API_KEY:-}" = "${LITELLM_KEY:-x}" ]; then
+    say "NOTE: OPENROUTER_API_KEY is set to the LiteLLM key in your environment."
+    say "      That is the old masquerade; unset it in ~/.hermes/.env so the"
+    say "      model picker shows the gateway's models instead of openrouter.ai's."
+  fi
+fi

mcp-config/sync.sh

@@ -134,13 +134,28 @@ except Exception:
 # actually reports are emitted; a model LiteLLM knows nothing about (a
 # self-hosted vLLM entry with no model_info in the gateway's model_list) simply
 # stays {}, which is what every model got before this existed.
+# opencode's limit.output is a RESERVE it subtracts from limit.context to decide
+# when to compact — NOT a server-side cap. LiteLLM reports max_output_tokens as
+# the whole window for self-hosted vLLM (vLLM caps TOTAL context, not output
+# separately — see charts/inference/values.yaml model_info), so copying it
+# verbatim made opencode reserve the entire window and target an input of
+# context - output. Cap the reserve at a generous agent-output budget instead.
+OUTPUT_RESERVE_CAP = 32768
+# opencode targets input == context - output EXACTLY, and the server counts one
+# more token than opencode does, landing at context + 1 and getting a hard 400
+# (ContextWindowExceededError). Observed on qwen3-coder-next: 196609 in + 65536
+# out = 262145 vs a 262144 window, one token over, every time. Hold a margin
+# back so opencode's target can never sit on the boundary.
+CONTEXT_SAFETY_MARGIN = 1024
+
+
 def spec_for(mi):
     spec = {}
     limit = {}
     if mi.get("max_input_tokens"):
-        limit["context"] = mi["max_input_tokens"]
+        limit["context"] = max(1, mi["max_input_tokens"] - CONTEXT_SAFETY_MARGIN)
     if mi.get("max_output_tokens"):
-        limit["output"] = mi["max_output_tokens"]
+        limit["output"] = min(mi["max_output_tokens"], OUTPUT_RESERVE_CAP)
     if limit:
         spec["limit"] = limit
     # LiteLLM prices PER TOKEN; opencode (like models.dev) uses $ per 1M tokens.

zsh/zshrc.local.example

@@ -16,6 +16,11 @@
 # setting it here. Only set it here on a box without mcp-sync:
 # export LITELLM_KEY=sk-...
 #
+# Hermes Agent uses the same key and endpoint. The base_url is configured
+# in ~/.hermes/config.yaml to point to your covert-gpt gateway
+# (https://llm.covert-gpt.com). Run hermes setup or update config.yaml
+# to set the default model (see ~/.hermes/config.yaml for available models).
+
 # To develop mcp-config/ or skills/ against a working checkout instead of
 # Gitea's main, point the sync at it and source it directly:
 # MCP_SYNC_RAW_URL="file://$HOME/source/dotfiles/mcp-config/opencode.gateway.json"