noconnorandClaude Opus 4.8
e42807c054
Two client-side fixes against the covert-gpt LiteLLM gateway, both traced from opencode throwing intermittent ContextWindowExceededError 400s. opencode (mcp-config/sync.sh): spec_for() mapped LiteLLM's /model/info onto opencode's schema 1:1, but the two mean different things by "output". LiteLLM's max_output_tokens is the whole window (vLLM caps TOTAL context, not output separately, which is correct for LiteLLM); opencode's limit.output is a RESERVE it subtracts from context to pick a compaction point. Copying one to the other told opencode to reserve the entire 262144 window, so it targeted an input of context-output and the server counted one token more than opencode did -- landing on context+1 and a hard 400, every time (measured: 196609 in + 65536 out = 262145 vs a 262144 window; the retries shrinking max_tokens by 65 never helped because the input grew to compensate). Capping the reserve alone would NOT have fixed it -- opencode would still sit exactly on the boundary, just at a different input size. So: cap the reserve (OUTPUT_RESERVE_CAP) and hold a margin back from context (CONTEXT_SAFETY_MARGIN). Generated at sync time, so it fixes every gateway model at once, including third-party ones. hermes (hermes/gateway.yaml, hermes/install.sh): Hermes reached the gateway by pretending it WAS OpenRouter -- the LiteLLM key in OPENROUTER_API_KEY, the gateway in OPENROUTER_BASE_URL. Requests routed fine, but fetch_openrouter_models hardcodes openrouter.ai/api/v1/models and ignores OPENROUTER_BASE_URL, so the picker listed OpenRouter's ~342-model catalog -- models this gateway cannot serve -- while the gateway's own models, including cg-1's, never appeared at all. Replaced with a named custom provider. install.sh now merges gateway.yaml into ~/.hermes/config.yaml, so a fresh box gets a Hermes that can actually talk to the gateway instead of one pointed at nothing. The merge is idempotent and narrow: only the covert-gpt entry is touched, model.default/provider are seeded only when unset, and a box still on the masquerade is migrated off it (fingerprint: provider openrouter + base_url pointing at this gateway; a genuine OpenRouter user never has that, and is left alone). No secrets: gateway.yaml names the key via key_env, and mcp-config/sync.sh is what exports it. Hermes needs no context safety margin -- it omits max_tokens entirely, so vLLM bounds output by whatever context is left. Also folds in the in-flight hermes bootstrap wiring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
84 lines
3.5 KiB
Bash
Executable File
84 lines
3.5 KiB
Bash
Executable File
#!/usr/bin/env sh
|
|
# bootstrap.sh — entrypoint for configuring a fresh VM (RHEL-family or
|
|
# Debian-family) with this repo's dotfiles. Safe to re-run.
|
|
#
|
|
# curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/bootstrap.sh | sh
|
|
# (or, from a checked-out clone: ./bootstrap.sh)
|
|
set -eu
|
|
|
|
REPO_URL="${DOTFILES_REPO:-https://git.nic-oconnor.com/public/dotfiles.git}"
|
|
DOTFILES_DIR="${DOTFILES_DIR:-$HOME/.dotfiles}"
|
|
|
|
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
|
|
|
|
# When curl-piped into `sh`, there is no checkout on disk: $0 is not a real
|
|
# path, so the module scripts alongside this file are missing. Clone (or
|
|
# update) the repo, then re-exec bootstrap.sh from the checkout.
|
|
if [ ! -f "$SCRIPT_DIR/zsh/install.sh" ]; then
|
|
if ! command -v git >/dev/null 2>&1; then
|
|
echo "bootstrap: git is required to self-clone; install git first" >&2
|
|
exit 1
|
|
fi
|
|
if [ -d "$DOTFILES_DIR/.git" ]; then
|
|
echo "bootstrap: updating existing checkout at $DOTFILES_DIR"
|
|
git -C "$DOTFILES_DIR" pull --ff-only
|
|
else
|
|
echo "bootstrap: cloning $REPO_URL into $DOTFILES_DIR"
|
|
git clone "$REPO_URL" "$DOTFILES_DIR"
|
|
fi
|
|
exec "$DOTFILES_DIR/bootstrap.sh"
|
|
fi
|
|
|
|
# python3 is a hard dependency of this setup, not an optional extra:
|
|
# - lib/fetch.sh needs python3 (or unzip) to extract .zip release assets;
|
|
# without either, the 1Password CLI install fails.
|
|
# - mcp-config/sync.sh needs it to inject the gateway's discovered models;
|
|
# without it opencode comes up with an EMPTY model list.
|
|
# Minimal Debian/RHEL images ship neither python3 nor unzip, so install it
|
|
# before bin/ runs. Non-fatal: an unsupported OS should degrade, not abort.
|
|
if ! command -v python3 >/dev/null 2>&1 && ! command -v unzip >/dev/null 2>&1; then
|
|
# shellcheck source=lib/os-detect.sh
|
|
. "$SCRIPT_DIR/lib/os-detect.sh"
|
|
echo "bootstrap: installing python3 (needed for zip assets and model discovery)"
|
|
pkg_install python3 || echo "bootstrap: warning: could not install python3" >&2
|
|
fi
|
|
|
|
# run_module <path> — run one module; on failure, warn and keep going.
|
|
#
|
|
# WHY: these used to be bare calls, so under `set -eu` the FIRST failing module
|
|
# aborted bootstrap and every later module silently never ran — one unsupported
|
|
# distro in docker/ was enough to leave a box with no CLIs, no npx and no
|
|
# ansible. The modules are independent, so a failure in one is not a reason to
|
|
# skip the rest. (bin/install.sh does the same for individual tools; this is
|
|
# that guard one level up.)
|
|
FAILED=""
|
|
run_module() {
|
|
if ( set -e; "$SCRIPT_DIR/$1/install.sh" ); then return 0; fi
|
|
FAILED="$FAILED $1"
|
|
echo "bootstrap: WARNING: $1 module failed; continuing." >&2
|
|
}
|
|
|
|
run_module zsh
|
|
run_module git
|
|
# Runs before bin/ so the kind install gate can see the rootful daemon.
|
|
run_module docker
|
|
run_module bin
|
|
# Provides npx, which opencode needs to spawn the npm-based MCP servers in
|
|
# mcp-config/. Without it every MCP server silently fails to start.
|
|
run_module node
|
|
# Hermes Agent (Nous Research) is distributed as an npm package with a Python
|
|
# dependency. Runs after node/ ensures npx is available.
|
|
run_module hermes
|
|
run_module ansible
|
|
# Runs last: needs rootless Docker already up (started at boot) so it can
|
|
# delegate cgroups to it. No-ops where that does not apply.
|
|
run_module docker-rootless-cgroup
|
|
|
|
if [ -n "$FAILED" ]; then
|
|
echo "" >&2
|
|
echo "bootstrap: WARNING: these modules did NOT complete:$FAILED" >&2
|
|
echo "bootstrap: everything else was applied. Fix the cause and re-run." >&2
|
|
exit 1
|
|
fi
|
|
echo "bootstrap: all modules applied." >&2
|