noconnorandClaude Opus 5
d42d0b29ff
OpenAI's /v1/models schema carries no context window, so a client that only
lists models cannot know how large a request it may send, and oversizing one
is a hard ContextWindowExceededError 400 — vLLM counts prompt + max_tokens
against its window. opencode already read the window from /v1/model/info.
Hermes and pi never consult that endpoint: they take it from static config,
which was hand-copied and drifted. Hermes named a model the gateway had
stopped serving while its replacement (the one model with a SMALLER window)
had no entry; pi's reserve was half its maxTokens. Both fail only once a
session grows, which is why it looked like the model or the GPU.
This renders one /v1/model/info response into all three clients. It is not a
plugin per harness and needs no upstream change — it extends the renderer that
already translates the MCP servers into Hermes' schema, with the same
manifest discipline.
SAME SOURCE, THREE DIFFERENT NUMBERS, deliberately — each client means
something different by "context":
opencode limit.context = window - 1024 reserves limit.output and targets
context - output, so needs headroom
Hermes context_length = window omits max_tokens entirely; vLLM
bounds output by what is left
pi maxTokens <= reserveTokens-2048 compacts at window - reserveTokens
(compaction.js shouldCompact) then
requests maxTokens
pi's maxTokens is derived FROM its reserve rather than the reserve being raised
to fit: reserveTokens is one global setting shared by every provider pi knows,
so raising it for this gateway would shrink everyone's usable context.
Safety properties, each tested in a sandbox against copies of the real
configs before touching live ones (11 cases):
- Ownership is per FIELD. A value is retracted only if this set it and the
gateway stopped declaring it. Hand-added models and fields are never
touched; a model removed upstream loses only what was rendered.
- Vision is retracted, not just added. The chart states that supports_vision
is what makes clients send image parts, so if a backend falls back to
language-only, the `image` input this added is taken back — but a user's own
`image` on an entry is left alone.
- Raw /v1/model/info is cached on each successful fetch, because refresh is
throttled and a throttled shell never fetches. A bad fetch (403 body, HTML
error page, empty list, rows without model_name) never replaces a good cache.
- Never invents a window: claude-* declares none and is skipped. A gateway
that suddenly declares no windows is treated as broken, not empty.
- One output cap (MCP_SYNC_OUTPUT_RESERVE_CAP) is read by opencode and pi, so
the clients cannot drift apart on it. Parses under dash as well as bash,
since install.sh sources this under /bin/sh.
First live run found a bug nobody knew about: pi's Claude entries carried
maxTokens 64000 against a 34816 reserve, so claude-haiku-4-5 sent
(200000 - 34816) + 64000 = 229184 tokens at a 200000 window — still broken
after the earlier manual reserve fix, which only covered 32768. Now 32768, and
every served model satisfies the invariant. It also gave Hermes limits for the
Claude routes it had none for.
hermes/gateway.yaml: the static map is marked as a first-boot seed that the
next sync overwrites, so nobody hand-edits it believing it is authoritative.
The source of truth is model_info.max_input_tokens in ai-k8s-iac's
charts/inference/values.yaml. README documents all of the above.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoDK4wwB1Twty5gwiriYLf
dotfiles
Configures fresh VMs (RHEL-family or Debian-family) with a consistent
environment. Run ./bootstrap.sh (or curl-pipe it) to apply everything;
each module also runs standalone and is safe to re-run.
Quick start
On a fresh box, apply everything with one command:
curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/bootstrap.sh | sh
When piped like this (no local checkout), bootstrap.sh clones the repo to
~/.dotfiles and runs from there, so git must be installed first. Re-running
the command updates the checkout (git pull) and re-applies every module.
Override the destination or source with DOTFILES_DIR / DOTFILES_REPO.
Already have a clone? Just run it directly:
./bootstrap.sh
Modules
-
zsh/— installs zsh + oh-my-zsh, drops a managed~/.zshrc(robbyrussell theme, git plugin), and sets zsh as the login shell. The managed rc sources~/.zshrc.locallast for per-machine customizations (see below). -
git/— sets gituser.name/user.emailand configurescredential.helper store(plaintext~/.git-credentials, 0600) for HTTPS auth. -
docker/— installs full (rootful) Docker Engine from Docker's upstream package repo (get.docker.com on Debian-family; the docker-ce repo wired up directly on RHEL-family, since the convenience script rejects EL rebuilds like Oracle Linux by name), enables the service (systemd hosts), and adds the invoking user to thedockergroup (takes effect at next login). No-ops where a rootless Docker daemon is already running (e.g. Kasm) so it never fights that setup. Uses sudo. -
bin/— installs userland devops CLIs as self-contained, statically linked release binaries into~/.local/bin: jq, kubectl, helm, k9s, kubectx/kubens, stern, yq, OpenTofu, terragrunt, task (go-task, the Taskfile runner), the 1Password CLIopfor secrets storage, the opencode coding agent, zarf (DevSecOps for Offline and Airgapped Kubernetes), and packer (HashiCorp's infrastructure build tool). kind is installed only when full (non-rootless) Docker is available — it doesn't work against rootless Docker, so it is skipped on Kasm. No sudo, nothing outside$HOME, so tools persist across Kasm redeploys with no reinstall. Idempotent: each tool records its version under~/.local/bin/.versionsand is skipped when the pinned version is already present.zsh/zshrcputs~/.local/binonPATH.opencode is configured by
mcp-config/: it points at the covert-gpt LiteLLM gateway (https://llm.covert-gpt.com/v1) for models. MCP tools do NOT come from the gateway — it hosts no MCP servers, deliberately; they run client-side under your own credentials (seemcp-config/README.md). Runmcp-config/install.shonce to store your LiteLLM key and enable config sync — without it, opencode installs but can't reach the gateway. -
ansible/— installs Ansible into a virtualenv at~/.local/opt/ansible(it's Python, not a single binary) and symlinks its CLIs into~/.local/bin. Persists across redeploys; only rebuilt when the venv is missing or broken. Building it may installpython3/python3-venvvia the system package manager (the one place this module uses sudo). -
docker-rootless-cgroup/— on systemd-less hosts (e.g. Kasm), hand-delegates cgroup v2 controllers to the rootless Docker daemon so per-container cgroups work (rootless Docker otherwise comes up withCgroupDriver=none). Uses sudo for host cgroup writes and restarts the daemon; no-ops on systemd or non-rootless hosts. Re-run at each bootstrap (the cgroup layout resets on redeploy); its PATH shim lives in$HOME. -
mcp-config/— shared MCP gateway config sync for laptops (LiteLLM / opencode). Run bybootstrap.sh, and standalone too. The config and skills are read from the local checkout, not re-downloaded from Gitea, so this repo does not have to be public for a shell to work; see its own README.
Per-machine customizations and secrets
~/.zshrc is managed — bootstrap overwrites it, so never hand-edit it.
Per-machine settings and secrets go in ~/.zshrc.local, which the managed
rc sources last (so it can override anything) and bootstrap never touches:
cp zsh/zshrc.local.example ~/.zshrc.local && chmod 600 ~/.zshrc.local
Variables modules look for:
-
GITEA_KEY— personal access token forgit.nic-oconnor.com. Used for API/CLI work, and bybootstrap.sh/mcp-config/to read this repo when it is private and there is no checkout yet. With a checkout on disk nothing here needs it; HTTPS git auth otherwise goes throughgit/'s credential helper. -
LITELLM_KEY— covert-gpt gateway key used by opencode. Normally you don't set this by hand: runmcp-config/install.shonce and its shell hook exports it from~/.config/mcp-sync/key. Set it in~/.zshrc.localonly on a box without mcp-sync. -
OP_SERVICE_ACCOUNT_TOKEN— 1Password service-account token for non-interactiveop read/op run. -
GITEA_KEY,PROXMOX_TOKEN_ID,PROXMOX_TOKEN_SECRET— your own credentials for the client-side MCP servers inmcp-config/. opencode reads them from your environment ({env:VAR}) and passes them to the MCP server it spawns, so the tools act as you. Keep them in~/.zshrc.local; they are never written into any config file.These are NOT used by
ai-k8s-iac/task up: the LiteLLM gateway hosts no MCP servers and stores no MCP credentials. Seemcp-config/README.md. -
hermes/— installs Hermes Agent (Nous Research's self-improving AI agent) via npm. Hermes uses npx to spawn and downloads the Python hermes-agent from PyPI. Everything under $HOME, so it survives Kasm redeploys.It then merges
hermes/gateway.yamlinto~/.hermes/config.yaml, wiring Hermes to the covert-gpt LiteLLM gateway as a named custom provider. The merge is idempotent and touches only that provider entry, so the rest of your personal config is left alone;model.default/model.providerare seeded only on a fresh box. Unlikemcp-config/, this deploys at BOOTSTRAP time, not per-shell — re-runhermes/install.shto pick up a gateway.yaml change.Depends on
mcp-config/install.shhaving run: gateway.yaml names the key (key_env: LITELLM_KEY) but never contains it, andmcp-config/sync.shis what exports LITELLM_KEY into your shell. Without it Hermes 401s.
Secrets never go in this repo or in zsh/zshrc — that file is shared
config applied to every box.
lib/os-detect.sh provides shared package-manager detection (dnf/yum
vs apt); lib/fetch.sh provides arch detection and binary-download
helpers. Both are used by module install scripts.