hermes: wire into dotfiles; fix opencode's generated context limits
Two client-side fixes against the covert-gpt LiteLLM gateway, both traced from opencode throwing intermittent ContextWindowExceededError 400s. opencode (mcp-config/sync.sh): spec_for() mapped LiteLLM's /model/info onto opencode's schema 1:1, but the two mean different things by "output". LiteLLM's max_output_tokens is the whole window (vLLM caps TOTAL context, not output separately, which is correct for LiteLLM); opencode's limit.output is a RESERVE it subtracts from context to pick a compaction point. Copying one to the other told opencode to reserve the entire 262144 window, so it targeted an input of context-output and the server counted one token more than opencode did -- landing on context+1 and a hard 400, every time (measured: 196609 in + 65536 out = 262145 vs a 262144 window; the retries shrinking max_tokens by 65 never helped because the input grew to compensate). Capping the reserve alone would NOT have fixed it -- opencode would still sit exactly on the boundary, just at a different input size. So: cap the reserve (OUTPUT_RESERVE_CAP) and hold a margin back from context (CONTEXT_SAFETY_MARGIN). Generated at sync time, so it fixes every gateway model at once, including third-party ones. hermes (hermes/gateway.yaml, hermes/install.sh): Hermes reached the gateway by pretending it WAS OpenRouter -- the LiteLLM key in OPENROUTER_API_KEY, the gateway in OPENROUTER_BASE_URL. Requests routed fine, but fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and ignores OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model catalog -- models this gateway cannot serve -- while the gateway's own models, including cg-1's, never appeared at all. Replaced with a named custom provider. install.sh now merges gateway.yaml into ~/.hermes/config.yaml, so a fresh box gets a Hermes that can actually talk to the gateway instead of one pointed at nothing. The merge is idempotent and narrow: only the covert-gpt entry is touched, model.default/provider are seeded only when unset, and a box still on the masquerade is migrated off it (fingerprint: provider openrouter + base_url pointing at this gateway; a genuine OpenRouter user never has that, and is left alone). No secrets: gateway.yaml names the key via key_env, and mcp-config/sync.sh is what exports it. Hermes needs no context safety margin -- it omits max_tokens entirely, so vLLM bounds output by whatever context is left. Also folds in the in-flight hermes bootstrap wiring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -101,6 +101,20 @@ Variables modules look for:
|
||||
|
||||
These are NOT used by `ai-k8s-iac` / `task up`: the LiteLLM gateway hosts no
|
||||
MCP servers and stores no MCP credentials. See `mcp-config/README.md`.
|
||||
- `hermes/` — installs Hermes Agent (Nous Research's self-improving AI agent)
|
||||
via npm. Hermes uses npx to spawn and downloads the Python hermes-agent from
|
||||
PyPI. Everything under $HOME, so it survives Kasm redeploys.
|
||||
|
||||
It then merges `hermes/gateway.yaml` into `~/.hermes/config.yaml`, wiring
|
||||
Hermes to the covert-gpt LiteLLM gateway as a named custom provider. The merge
|
||||
is idempotent and touches only that provider entry, so the rest of your
|
||||
personal config is left alone; `model.default` / `model.provider` are seeded
|
||||
only on a fresh box. Unlike `mcp-config/`, this deploys at BOOTSTRAP time, not
|
||||
per-shell — re-run `hermes/install.sh` to pick up a gateway.yaml change.
|
||||
|
||||
Depends on `mcp-config/install.sh` having run: gateway.yaml names the key
|
||||
(`key_env: LITELLM_KEY`) but never contains it, and `mcp-config/sync.sh` is
|
||||
what exports LITELLM_KEY into your shell. Without it Hermes 401s.
|
||||
|
||||
Secrets never go in this repo or in `zsh/zshrc` — that file is shared
|
||||
config applied to every box.
|
||||
|
||||
Reference in New Issue
Block a user