.agents/: add skills for working on this repo

Five skills covering this repo's own conventions: adding a bootstrap
module, adding a CLI to bin/, bumping a pinned version, adding an MCP
server, and verifying a module change.

Lives in .agents/skills/ because opencode discovers that path natively
alongside .opencode/skills/ and .claude/skills/, making it the most
tool-neutral of the three. Distinct from the repo's skills/ tree, which
ships to every covert-gpt user via sync.sh; these stay in the checkout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.agents/README.md

@@ -0,0 +1,76 @@
+# .agents — skills for working *on* this repo
+
+Task-scoped instructions a coding agent loads on demand when changing this
+repo: how to add a module, bump a pin, wire an MCP server, verify a change.
+
+`.agents/skills/` is a real discovery path, not a local convention — opencode
+searches it natively, alongside `.opencode/skills/` and `.claude/skills/`. That
+makes it the most vendor-neutral of the three, which is why the skills live
+here rather than under a tool-specific directory.
+
+## Not to be confused with `skills/`
+
+Two different things, easy to mix up:
+
+| | `skills/` (repo root) | `.agents/skills/` |
+|---|---|---|
+| Audience | every covert-gpt **user** | whoever is **editing this repo** |
+| Shipped? | yes — `mcp-config/sync.sh` deploys it to `~/.config/mcp-sync/skills` | no, stays in the checkout |
+| Content | general-purpose skills for daily work | this repo's own conventions |
+
+A skill about *how bootstrap modules are structured* belongs here. A skill
+about *how to write a Terraform module* belongs in `skills/`.
+
+## Layout
+
+Per the [opencode skills spec](https://opencode.ai/docs/skills/): one folder
+per skill, each containing a `SKILL.md`.
+
+```
+.agents/skills/
+  add-bootstrap-module/SKILL.md
+  add-cli-tool/SKILL.md
+  add-mcp-server/SKILL.md
+  bump-pinned-version/SKILL.md
+  verify-module/SKILL.md
+```
+
+Frontmatter requires `name` and `description`:
+
+- **`name`** — 1–64 chars, `^[a-z0-9]+(-[a-z0-9]+)*$`, and it **must match the
+  folder name**. A mismatch is the usual reason a skill doesn't show up.
+- **`description`** — 1–1024 chars. This is the only part the model sees when
+  deciding whether to load the skill, so write it to say **when to use this**,
+  not just what it is.
+
+`license`, `compatibility`, and `metadata` are optional; unknown fields are
+ignored. These skills set none of them — `compatibility: opencode` in
+particular would undercut the point of using a neutral directory.
+
+## Other agents
+
+opencode needs no configuration. For Claude Code, point its conventional path
+at the neutral one:
+
+```sh
+mkdir -p .claude && ln -sfn ../.agents/skills .claude/skills
+```
+
+Most other agents accept a skills directory; give them `.agents/skills`.
+Failing that, the files are plain Markdown — `@`-mention one directly.
+
+## Verifying
+
+```sh
+opencode debug skill      # lists every skill opencode sees, with its path
+```
+
+If one is missing, check the folder name against the frontmatter `name`.
+
+## Always-on conventions
+
+These skills deliberately do *not* restate the repo's baseline rules (POSIX
+`sh`, `set -eu`, idempotent, no sudo outside the modules that declare it, no
+secrets in-repo). Those apply to every change, so they belong in a root
+`AGENTS.md` that is always in context — not in a skill that loads only
+sometimes. Each skill assumes them and covers only what is specific to its task.

.agents/skills/add-bootstrap-module/SKILL.md

@@ -0,0 +1,71 @@
+---
+name: add-bootstrap-module
+description: Use when adding a new top-level module to this dotfiles repo (a new directory with an install.sh that bootstrap.sh runs) — e.g. installing a new runtime, configuring a new service, or dropping a new managed rc file. Covers the module contract, ordering in bootstrap.sh, and the README entry.
+---
+
+# Adding a bootstrap module
+
+A module is a top-level directory with an `install.sh` that bootstrap runs. It
+must be safe to run standalone *and* to re-run on an already-configured box.
+
+## The contract
+
+Every module's `install.sh` satisfies all of these. Copy the shape from
+`node/install.sh` — it is the newest and cleanest example.
+
+1. **POSIX `sh`, not bash.** `#!/usr/bin/env sh` + `set -eu`. No `[[`, no
+   arrays, no `local`. These boxes are RHEL- *and* Debian-family; don't assume
+   bash is the `/bin/sh`.
+2. **A `WHY:` header comment.** Not what the script does — the code says that.
+   Explain the failure it prevents or the constraint that forced the design.
+   `node/install.sh` is the model: it explains that a missing `npx` surfaces
+   only as a dead MCP server, which is easy to misdiagnose.
+3. **Locate the repo without depending on `$PWD`:**
+   ```sh
+   SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
+   REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
+   ```
+4. **Reuse `lib/`.** `lib/os-detect.sh` for `pkg_install` (handles dnf/yum vs
+   apt); `lib/fetch.sh` for `fetch_arch`, `have_version`/`mark_version`, and
+   the `install_*_bin` helpers. Source with a shellcheck directive:
+   ```sh
+   # shellcheck source=../lib/fetch.sh
+   . "$REPO_ROOT/lib/fetch.sh"
+   ```
+5. **Idempotent, and it verifies rather than assumes.** Check the tool *runs*,
+   not just that the file exists — a Kasm redeploy can leave a present-but-
+   broken tree behind. `node/` and `ansible/` both guard this way.
+6. **Log to stderr** via a `say()` helper, so stdout stays usable.
+7. **Prefer `$HOME`.** Everything under `~/.local` survives a Kasm redeploy
+   (which wipes `/` but keeps `$HOME`) and needs no sudo. If the module *must*
+   use sudo, say so in its header comment and in the README — the repo tracks
+   which modules do.
+
+## Wiring it up
+
+**`bootstrap.sh`** — add the call. Order is load-bearing and each existing
+out-of-order entry carries a comment explaining why (`docker/` before `bin/`
+so kind's gate sees the daemon; `docker-rootless-cgroup/` last because it
+needs the rootless daemon already up). If your module has an ordering
+constraint, add it *with* a comment; if it doesn't, append it near related
+modules.
+
+**`README.md`** — add a bullet to the `## Modules` list. Match the existing
+voice: what it installs, where it puts it, whether it uses sudo, and any
+host it no-ops on.
+
+**`~/.zshrc.local`** — if the module reads a secret or per-machine value, add
+the variable to `zsh/zshrc.local.example` and to the README's variable list.
+Never commit the value itself.
+
+## Non-systemd hosts
+
+Kasm has no systemd. Anything that would `systemctl enable` must degrade
+gracefully — see `docker/install.sh`, which detects the situation and no-ops
+rather than failing the whole bootstrap run.
+
+## Before you're done
+
+Run the `verify-module` skill. At minimum: `shellcheck`, run standalone, then
+run a second time and confirm the output says "already present" rather than
+redoing the work.

.agents/skills/add-cli-tool/SKILL.md

@@ -0,0 +1,72 @@
+---
+name: add-cli-tool
+description: Use when adding a userland CLI binary to bin/install.sh (jq, kubectl, helm, k9s, stern, yq, tofu, task, op, opencode, …). Covers the install_<name> function pattern, arch tokens, the .versions idempotency stamp, and when a tool does NOT belong in bin/.
+---
+
+# Adding a CLI tool to `bin/`
+
+`bin/install.sh` installs self-contained release binaries into `~/.local/bin`:
+no sudo, nothing outside `$HOME`, so tools persist across Kasm redeploys.
+
+## Does it belong here?
+
+`bin/` is for **single, statically linked binaries**. If the tool is a tree
+(interpreter + libraries), it does not fit `install_*_bin` and needs its own
+module that unpacks into `~/.local/opt/<name>` and symlinks the CLIs — that is
+exactly why `ansible/` and `node/` exist. Prefer static builds: after a Kasm
+redeploy the root filesystem's shared libraries are gone, and a dynamically
+linked binary in `$HOME` will break.
+
+## The pattern
+
+One `install_<name>` function, pinned version, guard, install, stamp:
+
+```sh
+# <name> — one line on what it is. Note any asset-naming quirk.
+install_<name>() {
+  ver=v1.2.3
+  have_version <name> "$ver" && { fsay "<name> $ver already installed."; return 0; }
+  install_tarball_bin \
+    "https://github.com/<org>/<name>/releases/download/$ver/<name>_linux_$DEB.tar.gz" \
+    "<path/inside/archive>" <name>
+  mark_version <name> "$ver"
+}
+```
+
+Then call it at the bottom of the file with the others.
+
+**Keep `ver` byte-identical to the upstream tag.** Some tools tag `v1.2.3`,
+some `1.2.3` (jq uses a bare `1.8.2`). `have_version` does a string compare
+against the stamp in `~/.local/bin/.versions`, so a mismatch just means the
+tool silently reinstalls on every bootstrap.
+
+## Arch tokens
+
+`bin/install.sh` precomputes both spellings — pick the one the upstream asset
+actually uses, don't call `uname` yourself:
+
+- `$ARCH` → `x86_64` / `aarch64` (rust-style release names)
+- `$DEB` → `amd64` / `arm64` (Go/Debian-style — most devops tools)
+
+## Picking the helper
+
+- `install_raw_bin <url> <dest>` — the asset *is* the binary (jq, kubectl, yq).
+- `install_tarball_bin <url> <path-in-archive> <dest>` — `.tar.gz` (helm, k9s).
+- `install_zip_bin <url> <path-in-archive> <dest>` — `.zip` (the 1Password CLI).
+
+Asset names are inconsistent and this is where it usually goes wrong: k9s
+capitalizes `Linux`, helm nests under `linux-$DEB/`. Fetch the release page and
+read the real asset name rather than guessing — a 404 from `curl -fsSL` will
+abort bootstrap under `set -eu`.
+
+## Conditional installs
+
+A tool that only works in some environments gets a gate, like kind — installed
+only when a **rootful** Docker daemon answers, because it does not work against
+rootless Docker on Kasm. Reuse `docker_full_available()` rather than writing a
+new probe.
+
+## Finally
+
+Add the tool to the `bin/` bullet in `README.md`. Then run it twice and confirm
+the second run prints "already installed."

.agents/skills/add-mcp-server/SKILL.md

@@ -0,0 +1,70 @@
+---
+name: add-mcp-server
+description: Use when adding or changing an MCP server in mcp-config/opencode.gateway.json (gitea, proxmox, …). Covers the {env:VAR} secret rule, the npx/node dependency, the shared-vs-personal boundary, and how to tell a dead server apart from a model that just can't call tools.
+---
+
+# Adding an MCP server
+
+`mcp-config/opencode.gateway.json` is the **shared** opencode config, synced to
+every covert-gpt user by `mcp-config/sync.sh`. Read `mcp-config/README.md`
+before changing it.
+
+## Secrets: `{env:VAR}`, never a literal
+
+```jsonc
+"environment": {
+  "GITEA_HOST": "https://git.nic-oconnor.com",
+  "GITEA_ACCESS_TOKEN": "{env:GITEA_KEY}"   // <- resolved from the user's shell
+}
+```
+
+This file is shared and committed. A literal token here leaks it to every user
+and into git history. The `{env:VAR}` indirection is what lets one shared
+config give each person tools that act **as them** — opencode resolves the var
+from the invoking user's environment and passes it to the server it spawns.
+
+When you add a server that needs a credential:
+
+1. Reference it as `{env:NEW_VAR}` in the config.
+2. Add `NEW_VAR` to `zsh/zshrc.local.example`.
+3. Document it in the root `README.md` variable list **and** in
+   `mcp-config/README.md`.
+
+Users keep the value in `~/.zshrc.local` (mode 0600), which bootstrap never
+overwrites. It is never written into any config file.
+
+## Shared vs personal
+
+This tree is identical for every user. A server that only makes sense for one
+person's cluster, kubeconfig, or local paths belongs in that user's own
+`~/.config/opencode/`, which sync never touches. Same rule as `skills/`.
+
+## It needs `npx`
+
+Servers are spawned as `npx -y <pkg>@<version>`, so `node/install.sh` must have
+run. **Pin the version** — `npx -y` otherwise runs whatever the registry serves,
+and this process receives the user's tokens. See the `bump-pinned-version` skill.
+
+## Debugging: dead server vs. bad model
+
+The failure modes look alike from the chat window, so distinguish them first:
+
+- **Server dead** — the tool isn't in the model's tool list at all. Usual cause:
+  `npx` missing from `PATH`, a bad package name, or an unset env var. opencode
+  reports this only as absent tools, which reads like a config problem.
+- **Server fine, model weak** — the tool *is* called and the server answers,
+  including with errors like `user 'x' does not exist`. That round trip proves
+  MCP works. A model that invents required arguments (owner, repo) rather than
+  discovering them, or retries the same wrong call verbatim, is a tool-use
+  capability limit, not a config bug. Smaller / non-agentic models (e.g. VL or
+  low-active-param MoE variants) do this; coder-tuned models generally don't.
+
+Check the tool list before touching the config:
+
+```sh
+opencode debug skill      # skills opencode can see
+npx -y <pkg>@<ver>        # does the server even start standalone?
+```
+
+Fix argument-guessing in `AGENTS.md` (give the model the default owner/host),
+not in `opencode.gateway.json`.

.agents/skills/bump-pinned-version/SKILL.md

@@ -0,0 +1,64 @@
+---
+name: bump-pinned-version
+description: Use when upgrading a pinned version anywhere in this repo — a CLI in bin/install.sh, NODE_VERSION in node/install.sh, or an npm-pinned MCP server in mcp-config/opencode.gateway.json. Covers where the pins live, how to verify the new release actually exists, and the supply-chain reasoning behind pinning.
+---
+
+# Bumping a pinned version
+
+Everything in this repo is version-pinned on purpose. Nothing floats.
+
+## Where the pins live
+
+| Pin | Location | Shape |
+|---|---|---|
+| CLI tools | `bin/install.sh` | `ver=v1.2.3` inside each `install_<name>()` |
+| Node.js | `node/install.sh` | `NODE_VERSION="v24.18.0"` |
+| MCP servers | `mcp-config/opencode.gateway.json` | `["npx", "-y", "pkg@0.0.10"]` |
+| oh-my-zsh, Docker | `zsh/`, `docker/` | installed from upstream scripts — not pinned |
+
+Grep before assuming: `grep -rnE 'ver=|_VERSION=|@[0-9]+\.[0-9]+' --include='*.sh' --include='*.json'`
+
+## Why these are pinned
+
+`npx -y <pkg>` executes whatever the registry serves *at that moment*. The MCP
+servers receive your Gitea and Proxmox tokens (`{env:GITEA_KEY}` and friends
+are passed into the spawned process), so an unpinned server is a live
+supply-chain path to your credentials. `node/install.sh` says this out loud in
+its header. A version bump is a **supply-chain decision**, not a chore — when
+bumping an MCP server or Node itself, skim the upstream changelog for a
+maintainer change or a new network dependency, and say what you found.
+
+## Doing the bump
+
+1. **Verify the release exists before editing.** Don't trust the model's
+   memory of version numbers — check upstream (`gh release list`, the GitHub
+   releases page, `npm view <pkg> versions`, or nodejs.org/dist).
+2. **Verify the asset URL still resolves.** Projects rename assets between
+   releases. Build the exact URL the script will request and check it:
+   ```sh
+   curl -fsSLI "https://github.com/derailed/k9s/releases/download/v0.51.0/k9s_Linux_amd64.tar.gz"
+   ```
+   A 404 here becomes a hard bootstrap failure under `set -eu`.
+3. **Match the upstream tag exactly** — `v1.2.3` vs `1.2.3` matters, because
+   `have_version` string-compares against the `.versions` stamp. Get it wrong
+   and the tool reinstalls on every single bootstrap, silently.
+4. **Bump one thing per commit.** When a box breaks after a bootstrap, the
+   commit log should say which upgrade did it.
+
+## Verifying
+
+The `.versions` stamp is what makes the bump observable:
+
+```sh
+cat ~/.local/bin/.versions/<tool>    # old version
+./bin/install.sh                     # should download, not skip
+cat ~/.local/bin/.versions/<tool>    # new version
+./bin/install.sh                     # should now say "already installed"
+```
+
+If the second run re-downloads, the `ver=` string and the stamp disagree — that
+is the tag-format bug from step 3.
+
+For an MCP server bump, restart opencode and confirm the server actually comes
+up. A dead MCP server is reported only as missing tools, so it looks like a
+model problem rather than a version problem.

.agents/skills/verify-module/SKILL.md

@@ -0,0 +1,78 @@
+---
+name: verify-module
+description: Use before committing any change to a module's install.sh or to bootstrap.sh, to prove the change actually works. Covers the idempotency re-run check, shellcheck with the right shell dialect, and testing against both RHEL- and Debian-family containers.
+---
+
+# Verifying a module change
+
+Nothing here is verified by a test suite — the only proof a module works is
+running it. Three checks, cheapest first.
+
+## 1. shellcheck
+
+Every script is POSIX `sh`, so tell shellcheck that. Its default assumes bash
+and will happily wave through `[[ ]]` and `local`, which break on a box where
+`/bin/sh` is dash.
+
+```sh
+shellcheck -s sh bootstrap.sh */install.sh lib/*.sh
+```
+
+`lib/` is sourced, not executed, so keep the `# shellcheck source=../lib/fetch.sh`
+directives above each `.` line or shellcheck can't follow them.
+
+## 2. Idempotency — the re-run check
+
+This is the one that catches real bugs. Run the module twice:
+
+```sh
+./<module>/install.sh    # does the work
+./<module>/install.sh    # must SKIP, not redo it
+```
+
+The second run has to say "already installed" / "already present". If it
+re-downloads, the guard is broken — most often a pinned `ver=` that doesn't
+match the string stamped in `~/.local/bin/.versions/<tool>` (`v1.2.3` vs
+`1.2.3`). That failure is silent: everything still works, it just refetches on
+every bootstrap forever.
+
+Guards must check the tool **runs**, not that a path exists. A Kasm redeploy
+leaves broken trees behind, and an existence check will happily skip past one.
+
+## 3. Both distro families
+
+`lib/os-detect.sh` branches on dnf/yum vs apt. If your change touches
+`pkg_install` or anything OS-conditional, exercise both:
+
+```sh
+docker run --rm -v "$PWD:/repo" -w /repo debian:13 sh -c \
+  'apt-get update -qq && apt-get install -y -qq sudo curl git >/dev/null && ./<module>/install.sh'
+
+docker run --rm -v "$PWD:/repo" -w /repo rockylinux:9 sh -c \
+  'dnf install -y -q sudo curl git >/dev/null && ./<module>/install.sh'
+```
+
+Run each container's command twice in the same shell to get the idempotency
+check on both families at once.
+
+Note this runs as root in a container, which is *not* how a real box runs it —
+it won't catch a missing-sudo problem. It's a portability check, not a
+faithful rehearsal.
+
+## Non-systemd hosts
+
+Kasm has no systemd and no rootful Docker. Anything that would `systemctl` must
+no-op rather than abort — you can't fully rehearse this in a container, so read
+the code path and confirm the guard exists (`docker/install.sh` and
+`docker-rootless-cgroup/install.sh` are the references).
+
+## Full run
+
+Finally, `./bootstrap.sh` end to end. It runs modules in a deliberate order and
+`set -eu` means any module's failure aborts the rest — a module that passes
+standalone can still break the chain if it was inserted at the wrong point.
+
+## Report honestly
+
+If you only ran shellcheck and the re-run check, say that. Don't describe a
+change as verified on both distro families unless you actually ran both.

README.md

@@ -40,8 +40,9 @@ Already have a clone? Just run it directly:
 - `bin/` — installs userland devops CLIs as self-contained, statically
   linked release binaries into `~/.local/bin`: jq, kubectl, helm, k9s,
   kubectx/kubens, stern, yq, OpenTofu, terragrunt, task (go-task, the
-  Taskfile runner), the 1Password CLI `op` for secrets storage, and the
-  opencode coding agent. kind is installed
+  Taskfile runner), the 1Password CLI `op` for secrets storage, the
+  opencode coding agent, and zarf (DevSecOps for Offline and Airgapped
+  Kubernetes). kind is installed
   only when full (non-rootless) Docker is available — it doesn't work
   against rootless Docker, so it is skipped on Kasm. No sudo, nothing
   outside `$HOME`, so tools persist across Kasm redeploys with no
@@ -89,6 +90,14 @@ Variables modules look for:
   only on a box without mcp-sync.
 - `OP_SERVICE_ACCOUNT_TOKEN` — 1Password service-account token for
   non-interactive `op read` / `op run`.
+- `GITEA_KEY`, `PROXMOX_MCP_TOKEN_ID`, `PROXMOX_MCP_TOKEN_SECRET` — **your own**
+  credentials for the client-side MCP servers in `mcp-config/`. opencode reads
+  them from your environment (`{env:VAR}`) and passes them to the MCP server it
+  spawns, so the tools act as *you*. Keep them in `~/.zshrc.local`; they are
+  never written into any config file.
+
+  These are NOT used by `ai-k8s-iac` / `task up`: the LiteLLM gateway hosts no
+  MCP servers and stores no MCP credentials. See `mcp-config/README.md`.
 
 Secrets never go in this repo or in `zsh/zshrc` — that file is shared
 config applied to every box.

bin/install.sh

@@ -156,6 +156,19 @@ install_opencode() {
   mark_version opencode "$ver"
 }
 
+# zarf — a DevSecOps tool for Offline and Airgapped Kubernetes (single static binary).
+install_zarf() {
+  ver=v0.81.0
+  have_version zarf "$ver" && { fsay "zarf $ver already installed."; return 0; }
+  case "$DEB" in
+    amd64) _zarf_arch=amd64 ;;
+    *) _zarf_arch=arm64 ;;
+  esac
+  install_raw_bin \
+    "https://github.com/zarf-dev/zarf/releases/download/$ver/zarf_${ver}_Linux_$_zarf_arch" zarf
+  mark_version zarf "$ver"
+}
+
 # task — go-task, the Taskfile.yml task runner. Binary is at the archive root.
 install_task() {
   ver=v3.52.0
@@ -194,5 +207,6 @@ install_task
 install_terragrunt
 install_1p
 install_opencode
+install_zarf
 
 fsay "Done. Tools in $BIN_DIR (added to PATH by zsh/zshrc)."

bootstrap.sh

@@ -34,6 +34,9 @@ fi
 # Runs before bin/ so the kind install gate can see the rootful daemon.
 "$SCRIPT_DIR/docker/install.sh"
 "$SCRIPT_DIR/bin/install.sh"
+# Provides npx, which opencode needs to spawn the npm-based MCP servers in
+# mcp-config/. Without it every MCP server silently fails to start.
+"$SCRIPT_DIR/node/install.sh"
 "$SCRIPT_DIR/ansible/install.sh"
 # Runs last: needs rootless Docker already up (started at boot) so it can
 # delegate cgroups to it. No-ops where that does not apply.

mcp-config/README.md

@@ -1,8 +1,39 @@
-# mcp-config — shared MCP gateway config for laptops
+# mcp-config — shared opencode config for laptops
 
-Single source of truth for the **shared** MCP servers every covert-gpt user
-gets through the LiteLLM gateway (`llm.covert-gpt.com`). Users install once;
-their config then refreshes automatically on each new shell.
+Single source of truth for what opencode sees: the **covert-gpt model provider**
+(LiteLLM at `llm.covert-gpt.com`) and the **MCP servers**, which run
+**client-side**. Users install once; their config then refreshes automatically
+on each new shell.
+
+## MCP runs on the CLIENT, never on the gateway
+
+The LiteLLM gateway hosts **no MCP servers** — deliberately. Every API worth
+exposing (Gitea, Proxmox, UniFi, NetBird, k8s) is **user-scoped**, so hosting it
+centrally means one shared token per server: every user acts as the same
+identity and attribution is gone.
+
+LiteLLM *can* do per-request BYOK (env values written `${X-Header}` are filled
+from the caller's headers — verified working for proxmox on v1.87.0), but only
+for servers that tolerate a credential-less spawn. LiteLLM builds its
+tool→server mapping from a **headerless** background listing, and `gitea-mcp`
+hard-exits without a token, so it never registers and its tool calls 404. The
+gateway could therefore host only *some* servers — and a split brain (some MCPs
+server-side, some client-side) is worse than either. So **all** MCP servers are
+defined here, each reading the user's **own** credentials from their own
+environment. The config is identical for everyone; only the secrets differ.
+
+Credentials are referenced as `{env:VAR}` and are **never** written into the
+config. Each user exports them in `~/.zshrc.local`:
+
+| MCP server | env vars |
+|---|---|
+| `gitea`   | `GITEA_KEY` |
+| `proxmox` | `PROXMOX_MCP_TOKEN_ID`, `PROXMOX_MCP_TOKEN_SECRET` |
+
+**Requires `npx`** — the MCP servers are npm packages that opencode spawns as
+`npx -y <pkg>@<version>`. `bootstrap.sh` installs Node for this (`node/`).
+Without it every server dies with "command not found", which opencode surfaces
+only as a dead server — easy to misread as a broken config.
 
 This repo lives in Gitea at `public/dotfiles`. (It is mirrored from the
 `ai-k8s-iac` repo under `mcp-config/`; edit it there or here and keep them in
@@ -10,16 +41,17 @@ sync — the laptops pull from Gitea's `main`.)
 
 ## How it works
 
-- `opencode.gateway.json` — the shared base config: the `covert-gpt` model
-  provider (LiteLLM's OpenAI-compatible endpoint at
+- `opencode.gateway.json` — the shared base config:
+  - the `covert-gpt` model provider (LiteLLM's OpenAI-compatible endpoint at
     `https://llm.covert-gpt.com/v1`, models map left empty — it is filled by
-  discovery at sync time) and a single MCP entry for the gateway's
-  **aggregate** endpoint `https://llm.covert-gpt.com/mcp/` (trailing slash
-  required; the bare path 307-redirects, which MCP clients don't follow).
-  That one endpoint exposes every MCP server the user's key can access, so
-  new gateway servers need no client config change at all. **Identical for
-  every user**: the per-user key is injected via `{env:LITELLM_KEY}`, never
-  written into the file.
+    discovery at sync time), and
+  - the **MCP servers**, each `type: local`, spawned by opencode as
+    `npx -y <pkg>@<pinned-version>` on the user's own machine.
+
+  **Identical for every user**: every credential (the LiteLLM key, the Gitea
+  PAT, the Proxmox token) is injected via `{env:VAR}` and never written into
+  the file. Versions are pinned — `npx` runs whatever the registry serves, and
+  that process gets your tokens.
 - `sync.sh` — sourced from each user's shell rc. On a new shell (throttled
   ~1h) it pulls the base config from Gitea, queries the gateway's
   `/v1/models` with the user's key to **auto-discover models** (opencode
@@ -50,34 +82,45 @@ own kubeconfig). Sync never touches those.
 
 ## User setup (one time)
 
-Each user needs a LiteLLM **virtual key** scoped to the gateway servers (see
-`charts/inference/MCP-GATEWAY.md` in ai-k8s-iac for how keys are minted). Then:
+Each user needs a LiteLLM **virtual key** (for models) plus their **own**
+credentials for each MCP server. Run `bootstrap.sh` (it installs Node, which
+provides the `npx` the MCP servers need), then:
 
 ```sh
 curl -fsSL https://git.nic-oconnor.com/public/dotfiles/raw/branch/main/mcp-config/install.sh | sh
 # (prompts for the sk-... key; or pass it: ... | LITELLM_KEY=sk-... sh)
 ```
 
-Open a new terminal, run `opencode`, and the gateway tools are there.
+Then export your own MCP credentials in `~/.zshrc.local`:
 
-## Adding a gateway server or model (admin)
+```sh
+export GITEA_KEY=...                  # your Gitea PAT (repo + issue scopes)
+export PROXMOX_MCP_TOKEN_ID=...       # e.g. alice@pve!mcp
+export PROXMOX_MCP_TOKEN_SECRET=...   # the token UUID
+```
 
-Register it on the gateway (`charts/inference/values.yaml` →
-`proxy_config.mcp_servers` for MCP servers, `model_list` for models),
-deploy, smoke-test. That's it — clients auto-discover both: MCP servers
-through the aggregate `/mcp/` endpoint, models via the `/v1/models` query
-in `sync.sh`. Users pick them up within the sync TTL (or immediately on a
-new shell). `opencode.gateway.json` here only needs editing for structural
-changes (a new provider, endpoint URLs).
+Open a new terminal, run `opencode`, and the tools are there — acting as **you**.
+
+## Adding an MCP server (admin)
+
+Add it to `mcp.<name>` in `opencode.gateway.json` as `type: local`, with a
+**pinned** package version and `{env:VAR}` refs for every secret, then push to
+`main`. Users pick it up within the sync TTL (or immediately on a new shell).
+
+Two hard requirements:
+- It must be an **npm package** (opencode spawns it with `npx`).
+- Its credentials must be **per-user**, via `{env:VAR}`. Never commit a token,
+  and never introduce a shared one — that is the whole reason MCP is not on the
+  gateway.
+
+Models are different: they live on the gateway
+(`charts/inference/values.yaml` → `model_list`) and are auto-discovered by the
+`/v1/models` query in `sync.sh`. No client change needed.
 
 ## Adding a skill (admin)
 
-Skills have no gateway side. Add a folder under `skills/` and push to `main` —
-see `skills/README.md`. Verify with `opencode debug skill`.
-
-For **per-user / per-target** tools (cluster access, etc.) do NOT add them
-here — those run client-side with the user's own credentials. See
-`charts/inference/MCP-GATEWAY.md`.
+Add a folder under `skills/` and push to `main` — see `skills/README.md`.
+Verify with `opencode debug skill`.
 
 ## Notes / caveats
 

mcp-config/opencode.gateway.json

@@ -12,12 +12,25 @@
     }
   },
   "mcp": {
-    "covert-gpt": {
-      "type": "remote",
-      "url": "https://llm.covert-gpt.com/mcp/",
+    "gitea": {
+      "type": "local",
+      "command": ["npx", "-y", "gitea-mcp@0.0.10"],
       "enabled": true,
-      "headers": {
-        "Authorization": "Bearer {env:LITELLM_KEY}"
+      "environment": {
+        "GITEA_HOST": "https://git.nic-oconnor.com",
+        "GITEA_ACCESS_TOKEN": "{env:GITEA_KEY}"
+      }
+    },
+    "proxmox": {
+      "type": "local",
+      "command": ["npx", "-y", "proxmox-mcp-server@0.2.0"],
+      "enabled": true,
+      "environment": {
+        "PROXMOX_HOST": "https://prox-01.covert-cloud.com:8006",
+        "PROXMOX_TOKEN_ID": "{env:PROXMOX_MCP_TOKEN_ID}",
+        "PROXMOX_TOKEN_SECRET": "{env:PROXMOX_MCP_TOKEN_SECRET}",
+        "PROXMOX_INSECURE_TLS": "true",
+        "PVE_READONLY": "false"
       }
     }
   },

mcp-config/sync.sh

@@ -11,15 +11,17 @@
 #
 # What it does:
 #   1. Fetch the shared opencode.gateway.json from Gitea (the single source of
-#      truth: the covert-gpt provider + the aggregate MCP endpoint). The file
-#      is identical for all users — the per-user key is injected via
-#      {env:LITELLM_KEY}, never baked into the file.
+#      truth: the covert-gpt provider + the client-side MCP servers). The file
+#      is identical for all users — EVERY credential (LiteLLM key, Gitea PAT,
+#      Proxmox token) is injected via {env:VAR} and never baked into the file.
 #   2. Auto-discover models: query the gateway's /v1/models with the user's
 #      key and fill the provider's "models" map. New models exposed on the
 #      gateway appear on every box at the next sync — no config edit anywhere.
 #      (opencode cannot list models from a custom provider itself; the map
 #      must be materialized client-side.) MCP servers need no discovery step:
-#      the single /mcp/ endpoint aggregates every server the key can access.
+#      they are declared in the config and run locally under `npx`, each with
+#      the user's OWN credentials. The gateway hosts no MCP servers at all —
+#      see mcp-config/README.md for why (user-scoped APIs, no shared tokens).
 #   3. Deploy the repo's skills/ tree to ~/.config/mcp-sync/skills. Skills are
 #      plain files, so there is nothing on the gateway to discover — the repo
 #      IS the source of truth, and the tree is fetched whole (repo tarball).

node/install.sh

@@ -0,0 +1,62 @@
+#!/usr/bin/env sh
+# node/install.sh — install Node.js (and with it npx) into userland.
+#
+# WHY: the MCP servers in mcp-config/opencode.gateway.json are npm packages and
+# opencode spawns them as `npx -y <pkg>@<version>`. Without npx on PATH every
+# MCP server fails to start with "command not found" — and opencode reports that
+# only as a dead server, so it is easy to misread as a broken config. The MCP
+# ecosystem is overwhelmingly npx-based, so this also unblocks future servers.
+#
+# Node is a tree (bin/node + lib/node_modules/npm), not a single binary, so it
+# cannot use bin/install.sh's install_tarball_bin. It follows the ansible module
+# instead: unpack under ~/.local/opt and symlink the CLIs into ~/.local/bin.
+# Everything is under $HOME — no sudo, and it survives a Kasm redeploy.
+set -eu
+
+SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
+REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
+
+# shellcheck source=../lib/fetch.sh
+. "$REPO_ROOT/lib/fetch.sh"
+
+say() { printf '%s\n' "$*" >&2; }
+
+# Pinned LTS. Bump deliberately: npx executes whatever the registry serves, so
+# the runtime is part of the supply chain that sees your Gitea/Proxmox tokens.
+NODE_VERSION="v24.18.0"
+
+PREFIX="$HOME/.local/opt/node"
+BIN_DIR="$HOME/.local/bin"
+NODE_CLIS="node npm npx"
+
+case "$(fetch_arch)" in
+  x86_64)  NODE_ARCH=x64 ;;
+  aarch64) NODE_ARCH=arm64 ;;
+  *)       say "node: unsupported arch $(fetch_arch)"; exit 1 ;;
+esac
+
+# Verify it RUNS, not just that the file exists: a Kasm redeploy can leave a
+# present-but-broken tree behind (same failure the ansible venv guards against).
+if [ "$("$PREFIX/bin/node" --version 2>/dev/null || true)" = "$NODE_VERSION" ]; then
+  say "node $NODE_VERSION already present at $PREFIX."
+else
+  TARBALL="node-$NODE_VERSION-linux-$NODE_ARCH.tar.gz"
+  URL="https://nodejs.org/dist/$NODE_VERSION/$TARBALL"
+  TMP="$(mktemp -d)"
+  say "Installing node $NODE_VERSION ($NODE_ARCH) into $PREFIX..."
+  _fetch_dl "$URL" "$TMP/node.tar.gz"
+  rm -rf "$PREFIX"
+  mkdir -p "$PREFIX"
+  # The archive has a single top-level dir; strip it so bin/ lands at $PREFIX/bin.
+  tar -xzf "$TMP/node.tar.gz" -C "$PREFIX" --strip-components=1
+  rm -rf "$TMP"
+  say "Installed node $NODE_VERSION."
+fi
+
+# Symlink the CLIs into ~/.local/bin (on PATH via zsh/zshrc). npx must be here:
+# opencode spawns MCP servers with a bare `npx`, resolved from PATH.
+mkdir -p "$BIN_DIR"
+for cli in $NODE_CLIS; do
+  [ -x "$PREFIX/bin/$cli" ] && ln -sf "$PREFIX/bin/$cli" "$BIN_DIR/$cli"
+done
+say "Linked node CLIs ($NODE_CLIS) into $BIN_DIR."