secrets: read the 1Password vault once into a 0600 env file
Secrets were pulled in each box's hand-written ~/.zshrc.local with one `op read` per field on EVERY new shell: every terminal waited on 1Password, every shell broke when it was unreachable, and the loop lived outside the repo. New secrets/ module (after bin/, which provides op and jq): one vault per user and one item, both defaulting to the login name, whose field labels are the variable names. It asks for the service-account token once (hidden), fetches the item with a single `op item get`, and writes every valid-named field to ~/.config/dotfiles/secrets.env (0600), replaced atomically so a failed fetch keeps the previous copy. Values go through jq @sh, so quotes, $() and newlines in a secret can't break or inject. The token is kept in op.env (0600), which shells never source. bootstrap loads secrets.env right after the module, so mcp-config takes LITELLM_KEY from the env instead of prompting. A LITELLM_KEY rotated in the vault also updates ~/.config/mcp-sync/key, since sync.sh re-exports from that file after secrets.env loads and would otherwise win. The managed zshrc sources secrets.env (no op call at startup) and adds envr / envc / envd / envl / envs to read, upsert, delete, list and refresh; they load the token only in a subshell. Existing boxes still need the old op loop removed from ~/.zshrc.local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -53,6 +53,19 @@ Already have a clone? Just run it directly:
|
||||
redeploys with no reinstall. Idempotent: each tool records its version
|
||||
under `~/.local/bin/.versions` and is skipped when the pinned version
|
||||
is already present. `zsh/zshrc` puts `~/.local/bin` on `PATH`.
|
||||
- `secrets/` — reads your 1Password vault once and writes every field to
|
||||
`~/.config/dotfiles/secrets.env` (0600), which the managed `~/.zshrc`
|
||||
sources, so opening a shell never calls `op`. One vault per user and one
|
||||
item in it, both named after your login by default; each field's label is
|
||||
the variable name (`GITEA_KEY`, `LITELLM_KEY`, `PROXMOX_TOKEN_SECRET`, …).
|
||||
The first run prompts for the service-account token (and vault/item), and
|
||||
keeps them in `~/.config/dotfiles/op.env` (0600, not loaded into shells).
|
||||
Runs after `bin/` (needs `op` and `jq`); bootstrap loads the result so later
|
||||
modules get the keys without prompting. Day to day: `envr NAME` reads a
|
||||
field, `envc NAME [VALUE]` creates or updates one (prompts, hidden, without
|
||||
VALUE), `envd NAME` deletes one; both of those refresh the file and this
|
||||
shell. `envl [VAULT [ITEM]]` shows every field with its current value, and
|
||||
`envs` refreshes after a change made elsewhere. No sudo.
|
||||
|
||||
opencode is configured by `mcp-config/`: it points at the covert-gpt
|
||||
LiteLLM gateway (`https://llm.covert-gpt.com/v1`) for models. MCP tools do
|
||||
@@ -87,7 +100,8 @@ rc sources last (so it can override anything) and bootstrap never touches:
|
||||
cp zsh/zshrc.local.example ~/.zshrc.local && chmod 600 ~/.zshrc.local
|
||||
```
|
||||
|
||||
Variables modules look for:
|
||||
Variables modules look for. With `secrets/` set up, put these in your vault
|
||||
item instead of `~/.zshrc.local`:
|
||||
|
||||
- `GITEA_KEY` — personal access token for `git.nic-oconnor.com`. Used for
|
||||
API/CLI work, and by `bootstrap.sh`/`mcp-config/` to read this repo when it
|
||||
@@ -98,8 +112,9 @@ Variables modules look for:
|
||||
don't set this by hand: run `mcp-config/install.sh` once and its shell
|
||||
hook exports it from `~/.config/mcp-sync/key`. Set it in `~/.zshrc.local`
|
||||
only on a box without mcp-sync.
|
||||
- `OP_SERVICE_ACCOUNT_TOKEN` — 1Password service-account token for
|
||||
non-interactive `op read` / `op run`.
|
||||
- `OP_SERVICE_ACCOUNT_TOKEN` — 1Password service-account token. Not a shell
|
||||
variable any more: `secrets/install.sh` asks for it once and stores it in
|
||||
`~/.config/dotfiles/op.env`. Pass it in the env only for an unattended run.
|
||||
- `GITEA_KEY`, `PROXMOX_TOKEN_ID`, `PROXMOX_TOKEN_SECRET` — **your own**
|
||||
credentials for the client-side MCP servers in `mcp-config/`. opencode reads
|
||||
them from your environment (`{env:VAR}`) and passes them to the MCP server it
|
||||
|
||||
@@ -82,6 +82,15 @@ run_module git
|
||||
# Runs before bin/ so the kind install gate can see the rootful daemon.
|
||||
run_module docker
|
||||
run_module bin
|
||||
# Needs op and jq from bin/. Runs before everything that consumes a secret, and
|
||||
# its output is loaded into THIS process so later modules see the vault's
|
||||
# values: mcp-config/ takes LITELLM_KEY from the env instead of prompting for
|
||||
# it, and GITEA_KEY is there for anything that reads the repo over the API.
|
||||
run_module secrets
|
||||
if [ -f "$HOME/.config/dotfiles/secrets.env" ]; then
|
||||
# shellcheck source=/dev/null
|
||||
. "$HOME/.config/dotfiles/secrets.env"
|
||||
fi
|
||||
# Provides npx, which opencode needs to spawn the npm-based MCP servers in
|
||||
# mcp-config/. Without it every MCP server silently fails to start.
|
||||
run_module node
|
||||
|
||||
Executable
+155
@@ -0,0 +1,155 @@
|
||||
#!/usr/bin/env sh
|
||||
# secrets/install.sh — write this user's 1Password vault item to a local,
|
||||
# 0600 env file that the managed ~/.zshrc sources.
|
||||
#
|
||||
# WHY: secrets used to be pulled live in ~/.zshrc.local, one `op read` per
|
||||
# field on EVERY new shell. That made each terminal wait on 1Password, broke
|
||||
# every shell when the network or the service account was down, and meant each
|
||||
# box carried its own hand-written copy of the loop. Now the vault is read once
|
||||
# here (at bootstrap, or on demand with `envs`), and shells read a local file.
|
||||
#
|
||||
# Layout: one vault per user (default: the login name), holding one item
|
||||
# (default: also the login name) whose field LABELS are the variable names —
|
||||
# GITEA_KEY, LITELLM_KEY, PROXMOX_TOKEN_SECRET, ... Every such field is
|
||||
# exported. Add a secret by adding a field (`envc NAME`), not by editing
|
||||
# this repo.
|
||||
#
|
||||
# Files, all under ~/.config/dotfiles (0700), none in the repo:
|
||||
# op.env OP_SERVICE_ACCOUNT_TOKEN, OP_VAULT, OP_ITEM (0600). Kept so a
|
||||
# refresh needs no prompt. Deliberately NOT sourced by shells:
|
||||
# the token stays out of every process's environment, and only
|
||||
# the env* helpers in zsh/zshrc load it, in a subshell.
|
||||
# secrets.env the exported fields (0600), replaced atomically, so a failed
|
||||
# fetch leaves the previous copy in place.
|
||||
#
|
||||
# Prompts for the token on the controlling TTY the first time (bootstrap is
|
||||
# often piped into sh, so stdin is not the terminal). Without a TTY, pass
|
||||
# OP_SERVICE_ACCOUNT_TOKEN (and optionally OP_VAULT / OP_ITEM) in the env.
|
||||
# Needs op and jq, which bin/ installs. No sudo.
|
||||
set -eu
|
||||
|
||||
say() { printf '%s\n' "$*" >&2; }
|
||||
|
||||
CONF_DIR="${DOTFILES_CONFIG_DIR:-$HOME/.config/dotfiles}"
|
||||
OP_ENV="$CONF_DIR/op.env"
|
||||
SECRETS_ENV="$CONF_DIR/secrets.env"
|
||||
MCP_SYNC_KEYFILE="$HOME/.config/mcp-sync/key"
|
||||
|
||||
umask 077
|
||||
mkdir -p "$CONF_DIR"
|
||||
chmod 0700 "$CONF_DIR"
|
||||
|
||||
PATH="$HOME/.local/bin:$PATH"
|
||||
for _tool in op jq; do
|
||||
if ! "$_tool" --version >/dev/null 2>&1; then
|
||||
say "secrets: $_tool is not installed or does not run; run bin/install.sh first."
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# --- 1. resolve the service account, vault and item -------------------------
|
||||
# Precedence: the environment, then op.env, then a prompt (token) / defaults.
|
||||
_env_token="${OP_SERVICE_ACCOUNT_TOKEN:-}"
|
||||
_env_vault="${OP_VAULT:-}"
|
||||
_env_item="${OP_ITEM:-}"
|
||||
if [ -f "$OP_ENV" ]; then
|
||||
# shellcheck source=/dev/null
|
||||
. "$OP_ENV"
|
||||
fi
|
||||
OP_SERVICE_ACCOUNT_TOKEN="${_env_token:-${OP_SERVICE_ACCOUNT_TOKEN:-}}"
|
||||
OP_VAULT="${_env_vault:-${OP_VAULT:-}}"
|
||||
OP_ITEM="${_env_item:-${OP_ITEM:-}}"
|
||||
_user="$(id -un)"
|
||||
|
||||
# ask <prompt> <default> — read one line from the TTY; echoes the default on empty.
|
||||
ask() {
|
||||
printf '%s [%s]: ' "$1" "$2" >&2
|
||||
IFS= read -r _ans < /dev/tty || _ans=""
|
||||
printf '%s' "${_ans:-$2}"
|
||||
}
|
||||
|
||||
if [ -z "$OP_SERVICE_ACCOUNT_TOKEN" ]; then
|
||||
if [ -f "$SECRETS_ENV" ]; then
|
||||
say "secrets: no service-account token; keeping existing $SECRETS_ENV."
|
||||
exit 0
|
||||
fi
|
||||
# Probe in a subshell: in POSIX sh a failed redirect on a special builtin
|
||||
# like `:` exits the whole script, which would skip the message below.
|
||||
if ! ( : < /dev/tty ) 2>/dev/null; then
|
||||
say "secrets: no OP_SERVICE_ACCOUNT_TOKEN and no TTY to prompt."
|
||||
say " re-run as: OP_SERVICE_ACCOUNT_TOKEN=ops_... $0"
|
||||
exit 1
|
||||
fi
|
||||
printf '1Password service-account token (ops_..., input hidden): ' >&2
|
||||
stty -echo < /dev/tty 2>/dev/null || true
|
||||
IFS= read -r OP_SERVICE_ACCOUNT_TOKEN < /dev/tty || true
|
||||
stty echo < /dev/tty 2>/dev/null || true
|
||||
printf '\n' >&2
|
||||
OP_SERVICE_ACCOUNT_TOKEN="$(printf '%s' "$OP_SERVICE_ACCOUNT_TOKEN" | tr -d ' \t\r\n')"
|
||||
[ -n "$OP_SERVICE_ACCOUNT_TOKEN" ] || { say "secrets: no token entered."; exit 1; }
|
||||
[ -n "$OP_VAULT" ] || OP_VAULT="$(ask 'Vault' "$_user")"
|
||||
[ -n "$OP_ITEM" ] || OP_ITEM="$(ask 'Item' "$_user")"
|
||||
fi
|
||||
OP_VAULT="${OP_VAULT:-$_user}"
|
||||
OP_ITEM="${OP_ITEM:-$_user}"
|
||||
export OP_SERVICE_ACCOUNT_TOKEN
|
||||
|
||||
# --- 2. fetch the item once and render it -----------------------------------
|
||||
# One `op item get` instead of one `op read` per field. Labels that are not
|
||||
# valid shell identifiers (and the item's built-in notes field) are skipped
|
||||
# rather than eval'd; values go through jq's @sh, so quotes, $ and newlines in
|
||||
# a secret cannot break or inject into the file.
|
||||
if ! _json="$(op item get "$OP_ITEM" --vault "$OP_VAULT" --reveal --format json)"; then
|
||||
say "secrets: could not read item '$OP_ITEM' in vault '$OP_VAULT'."
|
||||
[ -f "$SECRETS_ENV" ] && say "secrets: keeping the previous $SECRETS_ENV."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
_skipped="$(printf '%s' "$_json" | jq -r '
|
||||
[.fields[] | select(.purpose != "NOTES")
|
||||
| select((.label // "") | test("^[A-Za-z_][A-Za-z0-9_]*$") | not) | .label]
|
||||
| join(", ")')"
|
||||
[ -z "$_skipped" ] || say "secrets: skipped fields whose labels are not variable names: $_skipped"
|
||||
|
||||
_tmp="$(mktemp "$CONF_DIR/secrets.env.XXXXXX")"
|
||||
trap 'rm -f "$_tmp"' EXIT
|
||||
{
|
||||
printf '# Generated by dotfiles secrets/install.sh from op://%s/%s — do not edit.\n' "$OP_VAULT" "$OP_ITEM"
|
||||
printf '# Refresh with: envs (change the vault with: envc NAME / envd NAME)\n'
|
||||
printf '%s' "$_json" | jq -r '
|
||||
.fields[]
|
||||
| select(.purpose != "NOTES")
|
||||
| select((.label // "") | test("^[A-Za-z_][A-Za-z0-9_]*$"))
|
||||
| select((.value // "") != "")
|
||||
| "export \(.label)=\(.value | @sh)"'
|
||||
} > "$_tmp"
|
||||
chmod 0600 "$_tmp"
|
||||
_count="$(grep -c '^export ' "$_tmp" || true)"
|
||||
if [ "$_count" -eq 0 ]; then
|
||||
say "secrets: item '$OP_ITEM' has no exportable fields; keeping the previous file."
|
||||
exit 1
|
||||
fi
|
||||
mv -f "$_tmp" "$SECRETS_ENV"
|
||||
trap - EXIT
|
||||
say "secrets: wrote $_count variables to $SECRETS_ENV (0600)."
|
||||
|
||||
# Remember the account only after it has actually worked.
|
||||
{
|
||||
printf 'OP_SERVICE_ACCOUNT_TOKEN=%s\n' "$(printf '%s' "$OP_SERVICE_ACCOUNT_TOKEN" | jq -Rr @sh)"
|
||||
printf 'OP_VAULT=%s\n' "$(printf '%s' "$OP_VAULT" | jq -Rr @sh)"
|
||||
printf 'OP_ITEM=%s\n' "$(printf '%s' "$OP_ITEM" | jq -Rr @sh)"
|
||||
} > "$OP_ENV"
|
||||
chmod 0600 "$OP_ENV"
|
||||
|
||||
# --- 3. keep mcp-sync's copy of the gateway key in step ----------------------
|
||||
# mcp-config/sync.sh exports LITELLM_KEY from its own key file on every shell,
|
||||
# AFTER secrets.env is loaded, so a key rotated in the vault would otherwise be
|
||||
# silently overridden by the stale copy. Only update a file that already exists:
|
||||
# creating it is mcp-config/install.sh's job.
|
||||
_litellm="$(printf '%s' "$_json" | jq -r '.fields[] | select(.label == "LITELLM_KEY") | .value // empty' | head -n 1)"
|
||||
if [ -n "$_litellm" ] && [ -f "$MCP_SYNC_KEYFILE" ] \
|
||||
&& [ "$(tr -d ' \t\r\n' < "$MCP_SYNC_KEYFILE")" != "$_litellm" ]; then
|
||||
printf '%s' "$_litellm" > "$MCP_SYNC_KEYFILE"
|
||||
chmod 0600 "$MCP_SYNC_KEYFILE"
|
||||
say "secrets: LITELLM_KEY changed; updated $MCP_SYNC_KEYFILE (re-run hermes/install.sh for Hermes)."
|
||||
fi
|
||||
@@ -74,6 +74,73 @@ grmb() {
|
||||
return $rc
|
||||
}
|
||||
|
||||
# Secrets from this user's 1Password vault, written by dotfiles/secrets/install.sh.
|
||||
# Reading a local 0600 file keeps shell startup off the network: nothing here
|
||||
# calls op. envc/envd refresh the file themselves; after editing the vault
|
||||
# anywhere else (the 1Password app, another box), run `envs`.
|
||||
_DOTFILES_CONF="$HOME/.config/dotfiles"
|
||||
[ -f "$_DOTFILES_CONF/secrets.env" ] && source "$_DOTFILES_CONF/secrets.env"
|
||||
|
||||
# envr NAME read one field straight from the vault
|
||||
# envc NAME [VALUE] create or update a field (prompts, hidden, when VALUE is
|
||||
# omitted, which keeps the secret out of shell history),
|
||||
# then refresh and reload this shell
|
||||
# envd NAME delete a field, refresh, and unset it in this shell
|
||||
# envl [VAULT [ITEM]] show every field and its current value (default: your
|
||||
# vault and item from op.env)
|
||||
# envs re-fetch the vault into secrets.env and reload this shell
|
||||
#
|
||||
# The service-account token lives in op.env and is loaded only inside these
|
||||
# subshells, so it never lands in the environment of everything you run.
|
||||
_dotfiles_op() {
|
||||
[ -f "$_DOTFILES_CONF/op.env" ] || { print -u2 "no $_DOTFILES_CONF/op.env; run: envs"; return 1; }
|
||||
( set -a; source "$_DOTFILES_CONF/op.env"; set +a; "$@" )
|
||||
}
|
||||
_dotfiles_env_name() {
|
||||
[[ "$1" =~ '^[A-Za-z_][A-Za-z0-9_]*$' ]] && return 0
|
||||
print -u2 "$2: '$1' is not a valid variable name"; return 2
|
||||
}
|
||||
envr() {
|
||||
_dotfiles_env_name "$1" envr || return
|
||||
_dotfiles_op sh -c 'op read "op://$OP_VAULT/$OP_ITEM/$1"' envr "$1"
|
||||
}
|
||||
envc() {
|
||||
emulate -L zsh
|
||||
local name=$1 value
|
||||
_dotfiles_env_name "$name" envc || return
|
||||
if (( $# >= 2 )); then
|
||||
value=$2
|
||||
else
|
||||
read -rs "value?$name: " && print
|
||||
fi
|
||||
# [concealed] makes it a password field; an existing field of that name is updated.
|
||||
_dotfiles_op sh -c 'op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[concealed]=$2" >/dev/null' \
|
||||
envc "$name" "$value" && envs
|
||||
}
|
||||
envd() {
|
||||
_dotfiles_env_name "$1" envd || return
|
||||
_dotfiles_op sh -c 'op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[delete]" >/dev/null' envd "$1" \
|
||||
&& unset "$1" && envs
|
||||
}
|
||||
envl() {
|
||||
_dotfiles_op sh -c '
|
||||
vault=${1:-$OP_VAULT} item=${2:-$OP_ITEM}
|
||||
printf "op://%s/%s\n" "$vault" "$item" >&2
|
||||
op item get "$item" --vault "$vault" --reveal --format json' envl "$@" \
|
||||
| jq -r '
|
||||
[.fields[] | select(.purpose != "NOTES")] as $f
|
||||
| ([$f[] | .label // "" | length] | max // 0) as $w
|
||||
| $f[]
|
||||
| ((.label // "") + " " * ($w - ((.label // "") | length))) as $name
|
||||
| ((.value // "") | split("\n")) as $lines
|
||||
| "\($name) \($lines[0] // "")",
|
||||
($lines[1:][] | (" " * ($w + 2)) + .)'
|
||||
}
|
||||
envs() {
|
||||
"${DOTFILES_DIR:-$HOME/.dotfiles}/secrets/install.sh" \
|
||||
&& source "$_DOTFILES_CONF/secrets.env"
|
||||
}
|
||||
|
||||
# Per-machine customizations (secrets, keys, local overrides) live in
|
||||
# ~/.zshrc.local, which bootstrap never touches. See zsh/zshrc.local.example
|
||||
# for the variables modules look for. Loaded last so it can override anything.
|
||||
|
||||
+10
-6
@@ -1,7 +1,9 @@
|
||||
# ~/.zshrc.local — per-machine customizations. Copy this file to
|
||||
# ~/.zshrc.local and fill in what this box needs. It is sourced at the end
|
||||
# of the managed ~/.zshrc, is never overwritten by bootstrap, and never
|
||||
# lives in the repo — this is the ONLY place to put secrets.
|
||||
# lives in the repo. Secrets normally come from your 1Password vault instead
|
||||
# (secrets/install.sh -> ~/.config/dotfiles/secrets.env); set one here only on
|
||||
# a box without 1Password, or to override the vault's value on this machine.
|
||||
#
|
||||
# cp zsh/zshrc.local.example ~/.zshrc.local && chmod 600 ~/.zshrc.local
|
||||
|
||||
@@ -30,9 +32,11 @@
|
||||
# Only the bootstrap-managed checkout (~/.dotfiles) is ever `git pull`-ed by the
|
||||
# hook, and only when clean — a tree you work in is read, never touched.
|
||||
|
||||
# --- 1Password CLI -------------------------------------------------------
|
||||
# Service-account token for non-interactive `op read` / `op run`.
|
||||
# export OP_SERVICE_ACCOUNT_TOKEN=ops_...
|
||||
# --- 1Password -----------------------------------------------------------
|
||||
# Nothing to set here. Do NOT loop over `op read` in this file: that ran on
|
||||
# every shell. secrets/install.sh stores the service-account token in
|
||||
# ~/.config/dotfiles/op.env and writes the vault's fields to secrets.env, which
|
||||
# the managed ~/.zshrc loads. Use envr / envc / envd (see zsh/zshrc).
|
||||
|
||||
# --- Proxmox -------------------------------------------------------------
|
||||
# One pair of credentials, several consumers: the proxmox MCP server
|
||||
@@ -40,8 +44,8 @@
|
||||
# rke2-devops-airgap, whose Taskfile composes PROXMOX_VE_API_TOKEN from them.
|
||||
# Unprefixed on purpose — these are Proxmox credentials, not MCP ones.
|
||||
#
|
||||
# The op-vault loop above already exports them from the `noconnor` item, so
|
||||
# these lines are only needed without 1Password. Never committed.
|
||||
# secrets.env already exports them from your vault item, so these lines are
|
||||
# only needed without 1Password. Never committed.
|
||||
# export PROXMOX_TOKEN_ID="root@pam!terragrunt-proxmox"
|
||||
# export PROXMOX_TOKEN_SECRET="00000000-0000-0000-0000-000000000000"
|
||||
#
|
||||
|
||||
Reference in New Issue
Block a user