noconnorandClaude Opus 4.8
e42807c054
Two client-side fixes against the covert-gpt LiteLLM gateway, both traced from opencode throwing intermittent ContextWindowExceededError 400s. opencode (mcp-config/sync.sh): spec_for() mapped LiteLLM's /model/info onto opencode's schema 1:1, but the two mean different things by "output". LiteLLM's max_output_tokens is the whole window (vLLM caps TOTAL context, not output separately, which is correct for LiteLLM); opencode's limit.output is a RESERVE it subtracts from context to pick a compaction point. Copying one to the other told opencode to reserve the entire 262144 window, so it targeted an input of context-output and the server counted one token more than opencode did -- landing on context+1 and a hard 400, every time (measured: 196609 in + 65536 out = 262145 vs a 262144 window; the retries shrinking max_tokens by 65 never helped because the input grew to compensate). Capping the reserve alone would NOT have fixed it -- opencode would still sit exactly on the boundary, just at a different input size. So: cap the reserve (OUTPUT_RESERVE_CAP) and hold a margin back from context (CONTEXT_SAFETY_MARGIN). Generated at sync time, so it fixes every gateway model at once, including third-party ones. hermes (hermes/gateway.yaml, hermes/install.sh): Hermes reached the gateway by pretending it WAS OpenRouter -- the LiteLLM key in OPENROUTER_API_KEY, the gateway in OPENROUTER_BASE_URL. Requests routed fine, but fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and ignores OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model catalog -- models this gateway cannot serve -- while the gateway's own models, including cg-1's, never appeared at all. Replaced with a named custom provider. install.sh now merges gateway.yaml into ~/.hermes/config.yaml, so a fresh box gets a Hermes that can actually talk to the gateway instead of one pointed at nothing. The merge is idempotent and narrow: only the covert-gpt entry is touched, model.default/provider are seeded only when unset, and a box still on the masquerade is migrated off it (fingerprint: provider openrouter + base_url pointing at this gateway; a genuine OpenRouter user never has that, and is left alone). No secrets: gateway.yaml names the key via key_env, and mcp-config/sync.sh is what exports it. Hermes needs no context safety margin -- it omits max_tokens entirely, so vLLM bounds output by whatever context is left. Also folds in the in-flight hermes bootstrap wiring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
47 lines
2.4 KiB
Plaintext
47 lines
2.4 KiB
Plaintext
# ~/.zshrc.local — per-machine customizations. Copy this file to
|
|
# ~/.zshrc.local and fill in what this box needs. It is sourced at the end
|
|
# of the managed ~/.zshrc, is never overwritten by bootstrap, and never
|
|
# lives in the repo — this is the ONLY place to put secrets.
|
|
#
|
|
# cp zsh/zshrc.local.example ~/.zshrc.local && chmod 600 ~/.zshrc.local
|
|
|
|
# --- Gitea -------------------------------------------------------------
|
|
# Personal access token for git.nic-oconnor.com (API/CLI use; HTTPS git
|
|
# auth itself goes through credential.helper store, see git/).
|
|
# export GITEA_KEY=...
|
|
|
|
# --- covert-gpt LiteLLM gateway -----------------------------------------
|
|
# LITELLM_KEY is normally exported by mcp-config's sync hook from
|
|
# ~/.config/mcp-sync/key — run mcp-config/install.sh once instead of
|
|
# setting it here. Only set it here on a box without mcp-sync:
|
|
# export LITELLM_KEY=sk-...
|
|
#
|
|
# Hermes Agent uses the same key and endpoint. The base_url is configured
|
|
# in ~/.hermes/config.yaml to point to your covert-gpt gateway
|
|
# (https://llm.covert-gpt.com). Run hermes setup or update config.yaml
|
|
# to set the default model (see ~/.hermes/config.yaml for available models).
|
|
|
|
# To develop mcp-config/ or skills/ against a working checkout instead of
|
|
# Gitea's main, point the sync at it and source it directly:
|
|
# MCP_SYNC_RAW_URL="file://$HOME/source/dotfiles/mcp-config/opencode.gateway.json"
|
|
# MCP_SYNC_SKILLS_SRC="$HOME/source/dotfiles/skills"
|
|
# . "$HOME/source/dotfiles/mcp-config/sync.sh"
|
|
|
|
# --- 1Password CLI -------------------------------------------------------
|
|
# Service-account token for non-interactive `op read` / `op run`.
|
|
# export OP_SERVICE_ACCOUNT_TOKEN=ops_...
|
|
|
|
# --- Proxmox MCP gateway (ai-k8s-iac) ------------------------------------
|
|
# Credentials for the Proxmox MCP server registered on the covert-gpt
|
|
# LiteLLM gateway. `task up` in ai-k8s-iac reads these env vars and stores
|
|
# them in the inference-mcp Secret; they are never committed.
|
|
# export PROXMOX_MCP_HOST="https://pve.local:8006"
|
|
# export PROXMOX_MCP_TOKEN_ID="root@pam!terragrunt-proxmox"
|
|
# export PROXMOX_MCP_TOKEN_SECRET="00000000-0000-0000-0000-000000000000"
|
|
# export PROXMOX_MCP_INSECURE_TLS="true"
|
|
|
|
# --- Anything else -------------------------------------------------------
|
|
# Aliases, PATH additions, kubeconfig defaults, etc. This file loads last,
|
|
# so it can override anything the managed zshrc sets.
|
|
# export KUBECONFIG="$HOME/.kube/homelab.yaml"
|