Compare commits

...
Author SHA1 Message Date
noconnorandClaude Opus 5.5 9cb18a795d Coexist with a desktop that already manages its tools (Omarchy)
On an Omarchy laptop bootstrap left two of almost everything: VS Code from
both pacman and the tarball (two menu entries, one shared ~/.config/Code
profile), and kubectl/helm/k9s/kubectx/yq/jq/node/ansible in ~/.local/bin
behind mise and pacman copies that always won on PATH. The zsh helpers
(op*, grmb, secrets.env) also never reached the bash login shell.

- lib/fetch.sh: add `shadowed`, true when another copy of a tool comes
  earlier on PATH than ~/.local/bin. Always false where zshrc puts
  ~/.local/bin first, so VM installs are unchanged.
- bin/, node/, ansible/: skip tools that are shadowed.
- vscode/: defer to any system `code`, and remove a tarball copy this
  module installed earlier.
- bash/: new module; bash port of the zshrc helpers, sourced from
  ~/.bashrc via a marked block. No-ops without ~/.bashrc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 23:44:18 -04:00
noconnorandClaude Opus 5.5 41a90a10fb zsh: verify opc/opd against the vault and explain failures
op item edit can exit non-zero with "(404) Not Found" after an edit it
did save (seen with op 2.34.1 and 2.39.0 on an existing item), which
made opc report failure and skip the ops refresh. opc now reads the
field back and opd re-checks the item's labels, then prints either the
op:// reference it changed or what failed, op's message without its
timestamp prefix, and where to look next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:49:10 +00:00
noconnorandClaude Opus 5.5 5f0dec5270 zsh: let opc take NAME=VALUE and prompt when the value is missing
`opc NAME=VALUE` sets a field directly; `opc NAME` or `opc NAME=` prompts
with hidden input, and an empty answer changes nothing. The old
`opc NAME VALUE` form still works.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:43:27 +00:00
noconnorandClaude Opus 5.5 fe8b9a3ec0 claude-obsidian: drop committed bytecode; ignore __pycache__
The claude-obsidian merge carried a CPython 3.12 .pyc from running the
sync script in place. Bytecode is a build artifact; ignore it repo-wide.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 13:10:49 +00:00
noconnorandClaude Opus 5 72d85eb9ec vscode: install VS Code into ~/.local/opt from upstream's tarball
The box this was written on had a half-finished install: vscode.repo
present under /etc/yum.repos.d, `code` never installed, and nothing in
this repo recording that VS Code was meant to be there. Microsoft's
rpm/deb would also need root, write outside $HOME, and be wiped by a
Kasm redeploy. Upstream's portable tarball is the same build with none
of that, so the module keeps bin/'s contract: no sudo, nothing outside
$HOME, pinned rather than floating.

The Chromium sandbox survives a userland install — the bundled
chrome-sandbox is not setuid, so Electron falls back to the
unprivileged user-namespace sandbox. Verified by launching without
--no-sandbox, so dropping root costs nothing.

Adds install_tarball_tree to lib/fetch.sh: install_tarball_bin picks a
single binary out of an archive, and VS Code only runs from an intact
Electron tree. It stages beside the target and swaps in after
extraction succeeds, so an interrupted run leaves the old version
intact.

Two things the verification caught rather than the design:

  - argv.json lives at ~/.vscode/argv.json (product.json's
    dataFolderName), not under the user-data-dir. A copy written
    anywhere else is read by nothing, so the first attempt at
    suppressing the keyring prompt silently did nothing.

  - Run as root, `code --version` prints a "super user" warning
    instead of the version and still exits 0, so a bare run-check
    never matches and would re-download 325MB on every bootstrap.
    Root falls back to the .versions stamp.

Sets password-store=basic in argv.json so Electron's safeStorage stops
waking kwalletd, which prompts for an unlock on every launch because
kwallet-pam is not installed. Confirmed with dbus-monitor: no
org.kde.KWallet traffic on launch. The trade-off — tokens obfuscated
on disk rather than sealed behind a wallet password — and the route
back to the wallet are documented in the module and the README.

No-ops on headless hosts; DOTFILES_VSCODE overrides either way.

Verified per verify-module: shellcheck -s sh clean, idempotent re-run
on both paths, and fresh install plus re-run as root on oraclelinux:9
and debian:13.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 13:08:49 +00:00
noconnor 7bbd911365 Merge remote-tracking branch 'origin/claude-obsidian-sync' 2026-09-24 17:56:16 -04:00
noconnorandClaude Opus 5 b6d36ba113 hermes: put ~/.local/bin on PATH so the post-install check finds the CLI
Under the Coder agent (no managed zshrc in the parent) a successful
install still failed with "hermes CLI not found after install".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 13:46:09 -04:00
noconnorandClaude Opus 5 c0f1fc21b7 hermes: create ~/.hermes before writing .env on a fresh home
On a first run the key-changed branch writes ~/.hermes/.env before the
installer has created ~/.hermes, so it died with "Directory nonexistent"
and failed bootstrap (seen on a fresh Coder workspace).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 13:39:59 -04:00
noconnorandClaude Opus 5 86b8e51646 Remove the docker module: dotfiles no longer installs Docker
Installing Docker Engine is a host/image decision, not a per-user
environment one. In Coder workspaces (unprivileged pods, no systemd) the
module ran get.docker.com on every start for a daemon that can never
run there. Where Docker is wanted, install it on the host or bake it
into the image.

Also drops the zshrc newgrp-docker hook that existed only for this
module's group change, and the docs/skills references to docker/.
docker-rootless-cgroup/ (configures Kasm's existing rootless daemon)
and bin/'s kind gate (detects a daemon) stay.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 13:39:59 -04:00
noconnorandClaude Opus 5 ffedc84cfb secrets: widen the token mask to 10 stars and label first/last 10
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:59:02 -04:00
noconnorandClaude Opus 5 cb7ab17769 secrets: show the token masked (first/last 10 chars + length)
Makes a truncated or doubled paste visible at a glance without printing
the whole token.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:57:16 -04:00
noconnorandClaude Opus 5 5bc8a27f60 secrets: fail loudly on a bad 1Password token instead of a LiteLLM prompt
A bad token paste (or a vault/item that did not match the login-name
default) used to surface only as mcp-config asking for a LiteLLM key,
hiding that every vault key was missing. Now:

- the token is verified with `op whoami` and re-prompted on a bad paste
- vault and item are validated, auto-picked when the token sees only one,
  and otherwise prompted with the list it can see
- bootstrap skips mcp-config and hermes, with an error naming the cause,
  when no LITELLM_KEY came from the vault

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:55:27 -04:00
noconnorandClaude Opus 5 8ec3269f4a bin: install kubelogin as the kubectl oidc-login plugin
omnictl-generated kubeconfigs authenticate via `kubectl oidc-login
get-token`, so install int128/kubelogin as kubectl-oidc_login.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:35:15 -04:00
noconnor 46052cf573 Revert "claude-obsidian: Claude conversations -> Obsidian notes (+ MCP) (#1)"
This reverts commit d3393edfdd.
2026-09-17 16:23:54 -04:00
noconnor d3393edfdd claude-obsidian: Claude conversations -> Obsidian notes (+ MCP) (#1) 2026-09-17 20:18:09 +00:00
noconnorandClaude Opus 5 db3a3e6e82 claude-obsidian: don't count skipped empty sessions as still queued
The first full drain reported "105 written, 0 failed, 9 still queued" when
the 9 were empty sessions already recorded as skipped -- nothing was left.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KF322NzozeaFjpgWvh4F4g
2026-09-17 16:14:45 -04:00
noconnorandClaude Opus 5 376ec03428 claude-obsidian: unattended overnight runs -- lock, URL fallback, config fix
Needed to leave a full backlog drain running unattended:

- Run lock (flock): enabling the timer fired an immediate catch-up run
  (Persistent=true) while a manual run was about to start; without a
  lock the two would race on state.json and usage.jsonl.
- OBSIDIAN_API_URL accepts a comma-separated list, first reachable wins,
  so the permanent address and an interim port-forward share one config.
- Fix: inline "# comments" in config.env were parsed into values, so
  BUDGET_USD_PER_RUN=0.50  # ... crashed float(). The shipped
  config.env.example had exactly that shape on several lines -- anyone
  copying it would have hit the crash. A '#' inside a value (URL fragment)
  still survives. Tested against both the example and a real config.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KF322NzozeaFjpgWvh4F4g
2026-09-17 16:14:45 -04:00
noconnorandClaude Opus 5 2bfa7e9a2d claude-obsidian: meter and cap usage; cut per-session cost ~5.6x
The user asked for usage to be monitored closely, and measuring it turned
up that most of the spend was avoidable.

Metering: every model call is appended to usage.jsonl with BILLED tokens
(modelUsage -- the visible `usage` turn undercounted by half) and the
CLI's list-price cost. `usage` reports 5h/24h/7d. Each call's cost is
estimated first and refused if it would cross the per-run or rolling 24h
budget, so budgets cannot be overshot. Removed --max-budget-usd: it
aborts after spending and discards the result ($0.26 lost in one call).

Cost, measured on one session ($0.057 -> $0.010, same summary content):
- alwaysThinkingEnabled=false: extended thinking was ~85% of output.
- DISABLE_PROMPT_CACHING=1: one-shot prompts were being written to the
  1h cache at 2x input price.
- CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1: a hidden side request
  re-read the whole prompt, doubling billed input (9,638 vs 4,663).

Summaries are cached until their note is written, so a vault failure
never re-pays; --prefetch summarizes while the vault is unreachable.
Offline tests (fake claude binary): budget refusal pre-flight, cache
reuse with zero calls, map-reduce, env/argv levers, billed-token ledger.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KF322NzozeaFjpgWvh4F4g
2026-09-17 16:14:45 -04:00
noconnorandClaude Opus 5 11138d90a9 claude-obsidian: summarize Claude conversations into Obsidian notes
One note per Claude Code session (incremental, hourly user timer) and a
claude.ai data-export importer, written through the Obsidian Local REST API
plugin -- whose built-in /mcp/ server is also how Claude reads them back.

Transcripts are condensed (tool output dropped, one line per tool call,
~60x smaller), redacted, then summarized by headless `claude -p` on the
user's own plan. Three things learned the hard way are encoded here:

- --no-session-persistence, or every summary becomes a transcript that
  the next run summarizes.
- A LiteLLM gateway passing through the caller's Claude subscription has
  no key of its own, so a background job cannot use its claude-* models.
- Summaries share the plan's usage limits. A parallel ~100-session
  backfill exhausted a session limit in minutes. So one job, a per-run
  cap, and the run stops at the first usage-limit error (exit 75) instead
  of failing the whole queue; a stop flag, because Future.cancel() cannot
  stop work a thread already picked up. Covered by an offline test of
  stop / resume / idempotent rerun.

Nothing environment-specific is committed: hosts, keys and paths live in
~/.config/claude-obsidian/ (this repo is public).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KF322NzozeaFjpgWvh4F4g
2026-09-17 16:14:45 -04:00
noconnorandClaude Opus 5 7a7e987268 mcp-config: put the shell hook in ~/.zshrc.local on a fresh box
install.sh chose the rc file from $SHELL. On a fresh box bootstrap runs
from the user's original bash: zsh/ has already switched the login shell
to zsh, but $SHELL in that process still says bash. The hook went into
~/.bashrc, below zsh/'s auto-exec hook, so every new terminal exec'd into
zsh before reaching it, sync.sh never ran, OPENCODE_CONFIG was never
exported, and opencode came up unconfigured. Boxes already on zsh were
fine, which is why it only showed up on a new machine.

Now the dotfiles-managed ~/.zshrc, or zsh as the account's login shell
(getent), selects ~/.zshrc.local regardless of $SHELL. Standalone
`curl | sh` installs without these dotfiles behave as before.

Reproduced and verified in a clean ubuntu:24.04 container (zsh/ then
mcp-config/, dummy key): before, hook in .bashrc and OPENCODE_CONFIG
unset in a new zsh; after, hook in .zshrc.local and OPENCODE_CONFIG
points at a config with the covert-gpt provider and gitea/proxmox MCP
servers. A box already broken this way is repaired by re-running the
module, and re-runs leave a single hook.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 16:53:12 +00:00
noconnorandClaude Opus 5 0426ebc973 git: authenticate to our Gitea from $GITEA_KEY
git never reads GITEA_KEY on its own. With only `credential.helper
store`, auth depended on whatever was typed at a first prompt and cached
in ~/.git-credentials: it went stale when the token was rotated in the
vault, and a box that never pushed had no credential at all, so pushes
failed even with a key exported.

Add a credential helper scoped to https://git.nic-oconnor.com that hands
git $GITEA_KEY as the password (username from GITEA_USER, else the login
name). An empty helper entry first clears the inherited `store` for that
host, so a rejected token is never cached and a stale one can't win.
Other hosts keep `store`. With GITEA_KEY unset it prints nothing and git
prompts as before. Idempotent: re-runs leave one copy.

Verified: `git credential fill` returns exactly $GITEA_KEY for the host,
other hosts are unaffected, ~/.git-credentials stays absent. A dry-run
push still fails, because the key currently in the vault is rejected by
Gitea (401 on the API too); that is the token, not this wiring.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 16:36:20 +00:00
noconnorandClaude Opus 5 1159197e4b secrets: rename the vault helpers from env* to op*
They act on 1Password, not on the environment, so the prefix now says so:
opr (read), opc (create/update), opd (delete), opl (list), ops (refresh).
No behavior change. Checked the new names don't collide with any command
or oh-my-zsh alias, and verified opr/opl/ops against the real vault
(values compared, not printed).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 16:36:20 +00:00
26 changed files with 1765 additions and 231 deletions
+4 -4
View File
@@ -44,9 +44,9 @@ Every module's `install.sh` satisfies all of these. Copy the shape from
## Wiring it up
**`bootstrap.sh`** — add the call. Order is load-bearing and each existing
out-of-order entry carries a comment explaining why (`docker/` before `bin/`
so kind's gate sees the daemon; `docker-rootless-cgroup/` last because it
needs the rootless daemon already up). If your module has an ordering
out-of-order entry carries a comment explaining why (`secrets/` after `bin/`
because it needs `op`; `docker-rootless-cgroup/` last because it needs the
rootless daemon already up). If your module has an ordering
constraint, add it *with* a comment; if it doesn't, append it near related
modules.
@@ -61,7 +61,7 @@ Never commit the value itself.
## Non-systemd hosts
Kasm has no systemd. Anything that would `systemctl enable` must degrade
gracefully — see `docker/install.sh`, which detects the situation and no-ops
gracefully — see `docker-rootless-cgroup/install.sh`, which detects the situation and no-ops
rather than failing the whole bootstrap run.
## Before you're done
+1 -1
View File
@@ -14,7 +14,7 @@ Everything in this repo is version-pinned on purpose. Nothing floats.
| CLI tools | `bin/install.sh` | `ver=v1.2.3` inside each `install_<name>()` |
| Node.js | `node/install.sh` | `NODE_VERSION="v24.18.0"` |
| MCP servers | `mcp-config/opencode.gateway.json` | `["npx", "-y", "pkg@0.0.10"]` |
| oh-my-zsh, Docker | `zsh/`, `docker/` | installed from upstream scripts — not pinned |
| oh-my-zsh | `zsh/` | installed from upstream script — not pinned |
Grep before assuming: `grep -rnE 'ver=|_VERSION=|@[0-9]+\.[0-9]+' --include='*.sh' --include='*.json'`
+2 -2
View File
@@ -63,8 +63,8 @@ faithful rehearsal.
Kasm has no systemd and no rootful Docker. Anything that would `systemctl` must
no-op rather than abort — you can't fully rehearse this in a container, so read
the code path and confirm the guard exists (`docker/install.sh` and
`docker-rootless-cgroup/install.sh` are the references).
the code path and confirm the guard exists (`docker-rootless-cgroup/install.sh`
is the reference).
## Full run
+2
View File
@@ -0,0 +1,2 @@
__pycache__/
*.pyc
+48 -16
View File
@@ -29,19 +29,17 @@ Already have a clone? Just run it directly:
(robbyrussell theme, git plugin), and sets zsh as the login shell. The
managed rc sources `~/.zshrc.local` last for per-machine customizations
(see below).
- `bash/` — for hosts that keep bash as the login shell (Omarchy), copies a
bash port of the `zsh/zshrc` helpers (`opr`/`opc`/`opd`/`opl`/`ops`,
`grmb`, and `secrets.env` loading) to `~/.config/dotfiles/dotfiles.bash` and
adds one marked source line to `~/.bashrc`. Never owns `~/.bashrc`; no-ops
when there is none. No sudo.
- `git/` — sets git `user.name`/`user.email` and configures
`credential.helper store` (plaintext `~/.git-credentials`, 0600) for
HTTPS auth.
- `docker/` — installs full (rootful) Docker Engine from Docker's upstream
package repo (get.docker.com on Debian-family; the docker-ce repo wired up
directly on RHEL-family, since the convenience script rejects EL rebuilds
like Oracle Linux by name), enables the service (systemd hosts), and adds
the invoking user to the
`docker` group (takes effect at next login). No-ops where a rootless
Docker daemon is already running (e.g. Kasm) so it never fights that
setup. Uses sudo.
- `bin/` — installs userland devops CLIs as self-contained, statically
linked release binaries into `~/.local/bin`: jq, kubectl, helm, k9s,
linked release binaries into `~/.local/bin`: jq, kubectl, kubelogin (the
`kubectl oidc-login` plugin, for OIDC auth to Omni clusters), helm, k9s,
kubectx/kubens, stern, yq, OpenTofu, terragrunt, task (go-task, the
Taskfile runner), the 1Password CLI `op` for secrets storage, the
opencode coding agent, zarf (DevSecOps for Offline and Airgapped
@@ -49,23 +47,33 @@ Already have a clone? Just run it directly:
coder CLI for Coder workspaces.
kind is installed only when full (non-rootless) Docker is available —
it doesn't work against rootless Docker, so it is skipped on Kasm.
Dotfiles does not install Docker itself; install it on the host (or bake
it into the image) where you want it.
No sudo, nothing outside `$HOME`, so tools persist across Kasm
redeploys with no reinstall. Idempotent: each tool records its version
under `~/.local/bin/.versions` and is skipped when the pinned version
is already present. `zsh/zshrc` puts `~/.local/bin` on `PATH`.
A tool is also skipped when another copy comes *earlier* on `PATH` than
`~/.local/bin` (e.g. mise or pacman on Omarchy, where `~/.local/bin` is last),
since a copy there would never run. `node/` and `ansible/` follow the same
rule. On boxes where zshrc puts `~/.local/bin` first, nothing changes.
- `secrets/` — reads your 1Password vault once and writes every field to
`~/.config/dotfiles/secrets.env` (0600), which the managed `~/.zshrc`
sources, so opening a shell never calls `op`. One vault per user and one
item in it, both named after your login by default; each field's label is
the variable name (`GITEA_KEY`, `LITELLM_KEY`, `PROXMOX_TOKEN_SECRET`, …).
The first run prompts for the service-account token (and vault/item), and
keeps them in `~/.config/dotfiles/op.env` (0600, not loaded into shells).
The first run prompts for the service-account token and checks it with
1Password straight away, re-prompting on a bad paste. It then picks the vault
and item: the only one the token can see, or a prompt that lists them. It keeps
all three in `~/.config/dotfiles/op.env` (0600, not loaded into shells). If
that fails, bootstrap skips mcp-config/ and hermes/ with an error rather
than asking for the LiteLLM key by hand.
Runs after `bin/` (needs `op` and `jq`); bootstrap loads the result so later
modules get the keys without prompting. Day to day: `envr NAME` reads a
field, `envc NAME [VALUE]` creates or updates one (prompts, hidden, without
VALUE), `envd NAME` deletes one; both of those refresh the file and this
shell. `envl [VAULT [ITEM]]` shows every field with its current value, and
`envs` refreshes after a change made elsewhere. No sudo.
modules get the keys without prompting. Day to day: `opr NAME` reads a
field, `opc NAME [VALUE]` creates or updates one (prompts, hidden, without
VALUE), `opd NAME` deletes one; both of those refresh the file and this
shell. `opl [VAULT [ITEM]]` shows every field with its current value, and
`ops` refreshes after a change made elsewhere. No sudo.
opencode is configured by `mcp-config/`: it points at the covert-gpt
LiteLLM gateway (`https://llm.covert-gpt.com/v1`) for models. MCP tools do
@@ -79,6 +87,18 @@ Already have a clone? Just run it directly:
`~/.local/bin`. Persists across redeploys; only rebuilt when the venv is
missing or broken. Building it may install `python3`/`python3-venv` via
the system package manager (the one place this module uses sudo).
- `vscode/` — installs Visual Studio Code from upstream's portable tarball into
`~/.local/opt/vscode`, symlinks `code` into `~/.local/bin`, and writes a
user-level `.desktop` entry so it shows up in the application menu. **No
sudo, nothing outside `$HOME`** — the same contract as `bin/`, so it survives
a Kasm redeploy. Pinned via `VSCODE_VERSION` in the module; bump it to upgrade.
Also sets `password-store: basic` in `~/.vscode/argv.json` so KDE's kwallet
does not prompt for an unlock on every launch (see below). **No-ops on
headless hosts** — it's a ~400MB GUI app — override with `DOTFILES_VSCODE`.
**Defers to a system VS Code** (any `code` on `PATH` outside `~/.local/bin`,
e.g. Arch's `visual-studio-code-bin`): it skips the install and removes a
tarball copy it put there earlier, since two builds would show two menu
entries and share one `~/.config/Code` profile.
- `docker-rootless-cgroup/` — on systemd-less hosts (e.g. Kasm), hand-delegates
cgroup v2 controllers to the rootless Docker daemon so per-container cgroups
work (rootless Docker otherwise comes up with
@@ -112,6 +132,18 @@ item instead of `~/.zshrc.local`:
don't set this by hand: run `mcp-config/install.sh` once and its shell
hook exports it from `~/.config/mcp-sync/key`. Set it in `~/.zshrc.local`
only on a box without mcp-sync.
- `DOTFILES_VSCODE` — controls the `vscode/` module. Unset (default) installs
VS Code only when a graphical session is detected; `1` forces the install on
a headless host (e.g. for `code tunnel`); `0` skips it everywhere. A plain
shell/env var, not a vault secret.
On the keyring: VS Code's secret storage reaches for the OS keyring through
libsecret, which on KDE makes kwalletd prompt for an unlock before the editor
is usable — every session, since `kwallet-pam` is not installed. The module
sets `"password-store": "basic"` in `~/.vscode/argv.json` to stop that, which
means VS Code's stored tokens are obfuscated on disk rather than sealed
behind a wallet password. To use the wallet instead, install `kwallet-pam` so
it unlocks at login, then change that value to `"kwallet5"`.
- `OP_SERVICE_ACCOUNT_TOKEN` — 1Password service-account token. Not a shell
variable any more: `secrets/install.sh` asks for it once and stores it in
`~/.config/dotfiles/op.env`. Pass it in the env only for an unattended run.
+9
View File
@@ -15,6 +15,8 @@ REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
# shellcheck source=../lib/os-detect.sh
. "$REPO_ROOT/lib/os-detect.sh"
# shellcheck source=../lib/fetch.sh
. "$REPO_ROOT/lib/fetch.sh"
say() { printf '%s\n' "$*" >&2; }
@@ -22,6 +24,13 @@ VENV="$HOME/.local/opt/ansible"
BIN_DIR="$HOME/.local/bin"
ANSIBLE_CLIS="ansible ansible-playbook ansible-galaxy ansible-vault ansible-config ansible-doc ansible-inventory ansible-console ansible-pull"
# Another ansible (mise, pipx, a distro package) already wins on PATH, so this
# venv would never run -- see `shadowed` in lib/fetch.sh.
if _winner="$(shadowed ansible)"; then
say "ansible: $_winner comes first on PATH; skipping (a copy in $BIN_DIR would never run)."
exit 0
fi
# Verify it actually runs, not just that the file exists: a Kasm redeploy can
# wipe the system python3 the venv links to, leaving a present-but-broken
# venv that must be rebuilt.
+110
View File
@@ -0,0 +1,110 @@
# Managed by dotfiles/bash/install.sh — copied to ~/.config/dotfiles/dotfiles.bash
# and overwritten on the next bootstrap run. Edit bash/dotfiles.bash in the
# dotfiles repo instead.
#
# The bash port of the helpers in zsh/zshrc, for hosts that keep bash as the
# login shell (Omarchy ships its own bash config and the zsh module cannot
# install zsh there). Only the helpers: PATH, prompt and history stay with the
# host's own rc. Keep the two in sync — see zsh/zshrc for the full rationale
# behind each function.
# grmb <branch>... — delete remote branches; takes `origin/foo` or `foo`.
grmb() {
local remote=origin b rc=0
if [ "$#" -eq 0 ]; then
echo "grmb: usage: grmb <branch>... (origin/ prefix optional)" >&2
return 2
fi
git fetch --prune --quiet "$remote" || return 1
for b in "$@"; do
b=${b#"${remote}"/}
if git push "$remote" --delete "$b"; then
echo "deleted ${remote}/${b}"
else
echo "grmb: could not delete ${b} (already gone? check: git ls-remote --heads ${remote})" >&2
rc=1
fi
done
return $rc
}
# Secrets from this user's 1Password vault, written by dotfiles/secrets/install.sh.
# Reading a local 0600 file keeps shell startup off the network.
_DOTFILES_CONF="$HOME/.config/dotfiles"
[ -f "$_DOTFILES_CONF/secrets.env" ] && source "$_DOTFILES_CONF/secrets.env"
# opr NAME | opc NAME[=VALUE] | opc NAME VALUE | opd NAME | opl [VAULT [ITEM]] | ops
# Same behaviour as zsh/zshrc. The service-account token is loaded only inside
# the subshells, never into this shell's environment.
_dotfiles_op() {
[ -f "$_DOTFILES_CONF/op.env" ] || { echo "no $_DOTFILES_CONF/op.env; run: ops" >&2; return 1; }
( set -a; source "$_DOTFILES_CONF/op.env"; set +a; "$@" )
}
_dotfiles_env_name() {
[[ $1 =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] && return 0
echo "$2: '$1' is not a valid variable name" >&2; return 2
}
opr() {
_dotfiles_env_name "$1" opr || return
_dotfiles_op sh -c 'op read "op://$OP_VAULT/$OP_ITEM/$1"' opr "$1"
}
opc() {
local name=$1 value=
if [[ $1 == *=* ]]; then
# NAME=VALUE: split on the first '=' so values may contain '='
name=${1%%=*} value=${1#*=}
elif (( $# >= 2 )); then
value=$2
fi
_dotfiles_env_name "$name" opc || return
if [[ -z $value ]]; then
read -rsp "$name: " value && echo
[[ -n $value ]] || { echo "opc: empty value for $name; nothing changed" >&2; return 1; }
fi
# op's exit code is unreliable after an edit it did save; read the field back.
_dotfiles_op sh -c '
clean() { printf "%s" "${1:-(no output)}" | sed "s/^\[ERROR\] [0-9/]* [0-9:]* //"; }
ref="op://$OP_VAULT/$OP_ITEM/$1"
err=$(op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[concealed]=$2" 2>&1 >/dev/null)
if [ "$(op read "$ref" 2>/dev/null)" = "$2" ]; then
printf "opc: saved %s\n" "$ref" >&2
else
printf "opc: could not save %s\n op said: %s\n check vault, item and service-account write access with: opl\n" \
"$ref" "$(clean "$err")" >&2
exit 1
fi' opc "$name" "$value" && ops
}
opd() {
_dotfiles_env_name "$1" opd || return
_dotfiles_op sh -c '
clean() { printf "%s" "${1:-(no output)}" | sed "s/^\[ERROR\] [0-9/]* [0-9:]* //"; }
ref="op://$OP_VAULT/$OP_ITEM/$1"
err=$(op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[delete]" 2>&1 >/dev/null)
if ! item=$(op item get "$OP_ITEM" --vault "$OP_VAULT" --format json 2>&1); then
printf "opd: could not re-read %s to confirm the delete\n op said: %s\n" "$ref" "$(clean "$item")" >&2
exit 1
elif printf "%s" "$item" | jq -r ".fields[].label" | grep -qxF "$1"; then
printf "opd: could not delete %s\n op said: %s\n" "$ref" "$(clean "$err")" >&2
exit 1
fi
printf "opd: deleted %s\n" "$ref" >&2' opd "$1" \
&& unset "$1" && ops
}
opl() {
_dotfiles_op sh -c '
vault=${1:-$OP_VAULT} item=${2:-$OP_ITEM}
printf "op://%s/%s\n" "$vault" "$item" >&2
op item get "$item" --vault "$vault" --reveal --format json' opl "$@" \
| jq -r '
[.fields[] | select(.purpose != "NOTES")] as $f
| ([$f[] | .label // "" | length] | max // 0) as $w
| $f[]
| ((.label // "") + " " * ($w - ((.label // "") | length))) as $name
| ((.value // "") | split("\n")) as $lines
| "\($name) \($lines[0] // "")",
($lines[1:][] | (" " * ($w + 2)) + .)'
}
ops() {
"${DOTFILES_DIR:-$HOME/.dotfiles}/secrets/install.sh" \
&& source "$_DOTFILES_CONF/secrets.env"
}
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env sh
# bash/install.sh — give bash the dotfiles shell helpers (opr/opc/opd/opl/ops,
# grmb, and secrets.env loading).
#
# WHY: those helpers lived only in the managed ~/.zshrc, so on a host that
# keeps bash as its login shell they silently did not exist. Omarchy is that
# host: it ships its own ~/.bashrc and default rc, and zsh/ cannot install zsh
# there (no dnf/apt). secrets/ still wrote secrets.env, but nothing loaded it,
# so every vault-backed variable was simply unset in new shells.
#
# Unlike zsh/, this never owns ~/.bashrc -- the host's rc stays in charge. It
# copies bash/dotfiles.bash to ~/.config/dotfiles/ and adds one marked source
# line, the same pattern mcp-config/ uses. Skipped when there is no ~/.bashrc
# (a zsh-only box has no use for it). No sudo. Idempotent.
set -eu
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
say() { printf '%s\n' "$*" >&2; }
BASHRC="$HOME/.bashrc"
CONF_DIR="$HOME/.config/dotfiles"
DEST="$CONF_DIR/dotfiles.bash"
MARK_BEGIN="# >>> dotfiles helpers >>>"
MARK_END="# <<< dotfiles helpers <<<"
if [ ! -f "$BASHRC" ]; then
say "bash: no $BASHRC; skipping."
exit 0
fi
mkdir -p "$CONF_DIR"
chmod 700 "$CONF_DIR"
install -m 0644 "$SCRIPT_DIR/dotfiles.bash" "$DEST"
say "bash: wrote $DEST"
if grep -qxF "$MARK_BEGIN" "$BASHRC"; then
say "bash: $BASHRC already sources it."
else
cat >>"$BASHRC" <<EOF
$MARK_BEGIN
[ -f "\$HOME/.config/dotfiles/dotfiles.bash" ] && . "\$HOME/.config/dotfiles/dotfiles.bash"
$MARK_END
EOF
say "bash: added a source line to $BASHRC (open a new shell to pick it up)."
fi
+37 -17
View File
@@ -49,6 +49,18 @@ install_kubectl() {
mark_version kubectl "$ver"
}
# kubelogin — kubectl OIDC plugin, so omnictl-generated kubeconfigs can
# authenticate (they exec `kubectl oidc-login get-token`). Installed as
# kubectl-oidc_login, the name kubectl's plugin lookup expects.
install_oidc_login() {
ver=v1.36.4
have_version kubectl-oidc_login "$ver" && { fsay "kubectl-oidc_login $ver already installed."; return 0; }
install_zip_bin \
"https://github.com/int128/kubelogin/releases/download/$ver/kubelogin_linux_$DEB.zip" \
kubelogin kubectl-oidc_login
mark_version kubectl-oidc_login "$ver"
}
# helm — Kubernetes package manager (from get.helm.sh).
install_helm() {
ver=v4.2.2
@@ -226,34 +238,42 @@ install_coder() {
# The subshell re-enables `set -e` inside itself (it is disabled for a command
# being tested by `if`), so a failed download still aborts that tool's function
# before mark_version stamps a version that was never installed.
#
# try <install_fn> <binary> also skips the tool when another <binary> already
# wins on PATH (see `shadowed` in lib/fetch.sh): a copy here would never run.
FAILED=""
try() {
if _winner="$(shadowed "$2")"; then
fsay "$2: $_winner comes first on PATH; skipping (a copy in $BIN_DIR would never run)."
return 0
fi
if ( set -e; "$1" ); then return 0; fi
FAILED="$FAILED ${1#install_}"
fsay "WARNING: ${1#install_} failed to install; continuing."
}
try install_jq
try install_kubectl
try install_helm
try install_k9s
try install_jq jq
try install_kubectl kubectl
try install_oidc_login kubectl-oidc_login
try install_helm helm
try install_k9s k9s
if docker_full_available; then
try install_kind
try install_kind kind
else
fsay "Skipping kind: full (non-rootless) Docker not available."
fi
try install_kubectx
try install_kubens
try install_stern
try install_yq
try install_tofu
try install_task
try install_terragrunt
try install_1p
try install_opencode
try install_zarf
try install_packer
try install_coder
try install_kubectx kubectx
try install_kubens kubens
try install_stern stern
try install_yq yq
try install_tofu tofu
try install_task task
try install_terragrunt terragrunt
try install_1p op
try install_opencode opencode
try install_zarf zarf
try install_packer packer
try install_coder coder
if [ -n "$FAILED" ]; then
fsay ""
+25 -3
View File
@@ -66,7 +66,7 @@ fi
#
# WHY: these used to be bare calls, so under `set -eu` the FIRST failing module
# aborted bootstrap and every later module silently never ran — one unsupported
# distro in docker/ was enough to leave a box with no CLIs, no npx and no
# distro in one module was enough to leave a box with no CLIs, no npx and no
# ansible. The modules are independent, so a failure in one is not a reason to
# skip the rest. (bin/install.sh does the same for individual tools; this is
# that guard one level up.)
@@ -78,9 +78,9 @@ run_module() {
}
run_module zsh
# bash/ is zsh/'s helpers for hosts that keep bash (Omarchy); no-ops without ~/.bashrc.
run_module bash
run_module git
# Runs before bin/ so the kind install gate can see the rootful daemon.
run_module docker
run_module bin
# Needs op and jq from bin/. Runs before everything that consumes a secret, and
# its output is loaded into THIS process so later modules see the vault's
@@ -103,11 +103,33 @@ run_module node
# be repaired by re-running bootstrap. Losing ~/.zshrc.local (or the hook line
# in it) left opencode with no covert-gpt provider and no way to get it back
# short of curl-ing this module by hand.
#
# Both it and hermes/ are SKIPPED when the vault did not deliver LITELLM_KEY
# (and no box-local copy exists): mcp-config/ would otherwise fall back to a
# "paste your LiteLLM key" prompt, which reads like a LiteLLM problem and hides
# the real one — a bad 1Password token, vault or item, which also means every
# other vault key is missing.
if [ -z "${LITELLM_KEY:-}" ] && [ ! -s "$HOME/.config/mcp-sync/key" ]; then
echo "" >&2
echo "bootstrap: ERROR: no LITELLM_KEY from 1Password — skipping mcp-config and hermes." >&2
case " $FAILED " in
*" secrets "*) echo "bootstrap: the secrets module failed (see its messages above)." >&2 ;;
*) echo "bootstrap: the vault item has no LITELLM_KEY field; add it with: opc LITELLM_KEY" >&2 ;;
esac
echo "bootstrap: fix it with $SCRIPT_DIR/secrets/install.sh, then re-run bootstrap." >&2
echo "" >&2
FAILED="$FAILED mcp-config hermes"
else
run_module mcp-config
# Hermes Agent (Nous Research) is distributed as an npm package with a Python
# dependency. Runs after node/ ensures npx is available.
run_module hermes
fi
run_module ansible
# GUI editor, installed from upstream's tarball into ~/.local/opt (no sudo, same
# contract as bin/). No ordering constraints — it needs nothing from bin/,
# secrets/ or node/ — and no-ops on headless hosts.
run_module vscode
# Runs last: needs rootless Docker already up (started at boot) so it can
# delegate cgroups to it. No-ops where that does not apply.
run_module docker-rootless-cgroup
+113
View File
@@ -0,0 +1,113 @@
# claude-obsidian — Claude conversations → Obsidian notes
Summarizes every Claude conversation into one searchable Obsidian note: what happened, what
was decided and why, the facts and gotchas worth remembering, and what was left open.
- **Claude Code** sessions sync automatically and incrementally (hourly timer).
- **claude.ai** chats import from a data export (claude.ai has no API, so each export is manual).
Notes are written through the [Obsidian Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api)
plugin, so the vault can live anywhere the plugin is reachable. The same plugin exposes a
**built-in MCP server** (`/mcp/`), which is how Claude reads the notes back — no separate MCP
process.
```
~/.claude/projects/*/*.jsonl ─┐ ┌─ Claude/Code/2026/…md
claude.ai export .zip ────────┼─ condense → redact → summarize → PUT ────┼─ Claude/Web/2026/…md
└─ (headless claude -p, no transcript) └─ Claude/Index.md
```
## Install
1. In Obsidian, install **Local REST API**, enable its **HTTP server**, and make it reachable
from this machine. Put the API key in `~/.config/claude-obsidian/api-key` (`chmod 600`).
2. `./install.sh` — links the binary into `~/.local/bin`, enables the hourly user timer, and
creates `~/.config/claude-obsidian/config.env` from `config.env.example` if absent.
3. Edit `config.env` (at minimum `OBSIDIAN_API_URL`), then preview one note:
```bash
claude-obsidian-sync stats # sizes, no network
claude-obsidian-sync sync --dry-run --limit 1 # prints the note, writes nothing
```
## Use
```bash
claude-obsidian-sync sync # what the timer runs
claude-obsidian-sync sync --session <session-id> # one session, now, ignoring the settle window
claude-obsidian-sync import-claude-ai ~/Downloads/data-2026-09-13.zip
claude-obsidian-sync sync --prefetch --limit 3 # summarize into the local cache, vault untouched
claude-obsidian-sync usage # calls, billed tokens, spend: last 5h / 24h / 7d
```
Claude reads the vault through the plugin's MCP server:
```bash
claude mcp add --scope user --transport http obsidian http://<host>:27123/mcp/ \
--header "Authorization: Bearer $(cat ~/.config/claude-obsidian/api-key)"
```
## How it behaves — and why
- **Summaries run on your own Claude plan** via headless `claude -p` (`LLM_BACKEND=claude-cli`).
No API key to manage. An OpenAI-compatible backend (`LLM_BACKEND=openai`) exists for local
models. Note a LiteLLM gateway that fronts Claude by *passing through the caller's*
subscription has no key of its own, so a background job cannot use its `claude-*` models.
- **`--no-session-persistence` is load-bearing.** Without it every summary writes a new
transcript into `~/.claude/projects`, and the next run summarizes its own summarizing.
- **Usage is metered and capped.** Every model call is appended to
`~/.local/state/claude-obsidian/usage.jsonl` with *billed* tokens (the CLI's `modelUsage`, not
the visible turn) and `total_cost_usd` — API list-price equivalent, a faithful proxy for how much
was consumed; the plan's own meter is only visible in Claude Code's `/usage`. Before each call
its cost is **estimated** (+20% margin) and the call is refused if it would cross
`BUDGET_USD_PER_RUN` or the rolling `BUDGET_USD_PER_DAY` — so budgets are never overshot. Not
`claude -p --max-budget-usd`: that aborts *after* the tokens are spent and discards the result
(measured: $0.26 for nothing). Every run ends with a one-line usage report.
- **Three cost levers, measured on one session — $0.057 → $0.010, same summary content:**
`alwaysThinkingEnabled=false` (extended thinking was ~85% of output tokens),
`DISABLE_PROMPT_CACHING=1` (the CLI writes each prompt to the 1-hour cache at 2× input price, and
a one-shot call never reads it back), and `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` (otherwise a
hidden side request re-reads the entire prompt, doubling billed input). A whole ~110-session
backlog is about $2 at list price with all three.
- **Summaries are cached until their note is written**, so a vault outage never pays for the same
summary twice, and `--prefetch` can do the summarizing while the vault is unreachable.
- **It shares your plan's usage limits.** A first backfill of ~100 sessions in parallel hit a
session limit within minutes — which also limits *interactive* use. So: one job at a time,
`MAX_PER_RUN` sessions per run (newest first), and the run **stops at the first usage-limit
error** (exit 75, which the systemd unit treats as success) instead of failing the rest of
the queue. The next hourly run resumes. A 100-session backlog drains in about ten hours.
- **Secrets are redacted before anything leaves the machine**, and again in the note: private
keys, bearer tokens, `password=`-style assignments, JWTs, provider tokens, ≥48-char hex, and
random mixed-case strings. It over-redacts on purpose; git SHAs, md5s and paths survive.
The summarizer is also told never to reproduce secrets.
- **One run at a time** (a `flock` on `run.lock`): an hourly timer firing during a long manual run
exits immediately instead of racing it on the state and ledger files.
- **Several vault URLs** can be listed; the first that answers wins, so one config covers both a
permanent address and a temporary `kubectl port-forward`.
- **Incremental and idempotent.** State (`~/.local/state/claude-obsidian/state.json`) records
each session's size+mtime. A resumed session is re-summarized into the *same* note path. A
session is left alone until idle for `SETTLE_MINUTES`, so a live conversation isn't
summarized mid-flight. Empty sessions are recorded and skipped.
- **Condensed, not truncated.** Tool output is dropped and each tool call becomes one line;
prose is kept. Typical compression is ~60× (271 MB of transcripts → 4.4 MB). Sessions over
300 KB condensed are summarized in parts, then combined.
- **Subagent transcripts** (`<session>/subagents/**`) are part of their parent session and are
not synced separately.
## Note format
```yaml
type: claude-conversation
source: claude-code # or claude-ai
session_id: 3f9c2a1e-…
title: Fix flaky CI cache after runner upgrade
project: my-service # basename of the working directory
branch: …
started / ended: …
prompts: 1 # user prompts, not tool-loop records
summarized_by: claude-haiku-4-5-20251001
tags: [claude, claude-code, ci, caching, …]
```
followed by a summary callout and **Outcomes / Decisions / Facts & gotchas / Open threads**,
and a `claude --resume <id>` line. `Claude/Index.md` lists every note, newest first.
+798
View File
@@ -0,0 +1,798 @@
#!/usr/bin/env python3
"""claude-obsidian-sync -- summarize Claude conversations into an Obsidian vault.
Reads Claude Code transcripts (~/.claude/projects/<project>/<session>.jsonl) and claude.ai
data exports, condenses each conversation, redacts secrets, summarizes it with an
OpenAI-compatible LLM endpoint (e.g. LiteLLM), and writes one note per conversation into
Obsidian through the Local REST API plugin. Standard library only.
claude-obsidian-sync stats # sizes and counts, no network
claude-obsidian-sync sync [--dry-run] # Claude Code sessions (incremental)
claude-obsidian-sync import-claude-ai EXPORT.zip|conversations.json
Configuration comes from the environment, falling back to ~/.config/claude-obsidian/config.env
(KEY=VALUE lines). Nothing environment-specific lives in this file. See README.md.
"""
from __future__ import annotations
import argparse
import concurrent.futures as cf
import datetime as dt
import fcntl
import hashlib
import json
import os
import pathlib
import re
import shutil
import subprocess
import sys
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
import zipfile
HOME = pathlib.Path.home()
CONFIG_DIR = HOME / ".config" / "claude-obsidian"
# --------------------------------------------------------------------------- config
def load_config() -> dict:
cfg: dict[str, str] = {}
f = CONFIG_DIR / "config.env"
if f.exists():
for line in f.read_text().splitlines():
line = line.strip()
if line and not line.startswith("#") and "=" in line:
k, v = line.split("=", 1)
v = re.split(r"\s+#", v, maxsplit=1)[0] # inline comment; a '#' inside a value survives
cfg[k.strip()] = v.strip().strip('"').strip("'")
cfg.update({k: v for k, v in os.environ.items() if k in DEFAULTS or k.startswith("LLM_") or k.startswith("OBSIDIAN_")})
for k, v in DEFAULTS.items():
cfg.setdefault(k, v)
return cfg
DEFAULTS = {
"OBSIDIAN_API_URL": "http://127.0.0.1:27123",
"OBSIDIAN_API_KEY_FILE": str(CONFIG_DIR / "api-key"),
"LLM_BACKEND": "claude-cli", # claude-cli | openai
"CLAUDE_BIN": "claude",
"LLM_BASE_URL": "",
"LLM_API_KEY_FILE": "",
"LLM_API_KEY_ENV": "LITELLM_KEY",
"LLM_MODEL": "haiku",
"VAULT_FOLDER": "Claude",
"SETTLE_MINUTES": "30",
"CLAUDE_PROJECTS_DIR": str(HOME / ".claude" / "projects"),
"STATE_FILE": str(HOME / ".local" / "state" / "claude-obsidian" / "state.json"),
"JOBS": "1",
# Cap per run. Summaries draw on the same plan limits as interactive Claude use, so the
# timer drips a backlog instead of bursting it (a 107-session burst exhausted a session limit).
"MAX_PER_RUN": "10",
# Spend guards, in API list-price USD as reported by `claude -p` (total_cost_usd). Not the
# plan's own meter -- that is only visible in Claude Code's /usage -- but a faithful proxy
# for how much was consumed. Hitting one stops the run like a usage limit (exit 75).
"BUDGET_USD_PER_RUN": "0.50",
"BUDGET_USD_PER_DAY": "2.00",
# Pre-call estimate prices (USD per million tokens, list). A call is refused BEFORE it runs
# if the estimate would cross a budget. Deliberately NOT `claude -p --max-budget-usd`: that
# aborts after the tokens are spent and throws the result away (measured: $0.26 for nothing).
"LLM_PRICE_IN_PER_M": "1.0",
"LLM_PRICE_OUT_PER_M": "5.0",
}
def secret(cfg: dict, file_key: str, env_key: str | None = None) -> str:
path = cfg.get(file_key)
if path:
p = pathlib.Path(os.path.expanduser(path))
if p.exists():
return p.read_text().strip()
if env_key and os.environ.get(cfg.get(env_key, "") or ""):
return os.environ[cfg[env_key]].strip()
raise SystemExit(f"missing secret: set {file_key} (or {env_key})")
# --------------------------------------------------------------------------- redaction
# Applied to everything sent to the LLM AND to the note written. Over-redaction is the
# accepted failure mode: a summary missing a token is fine, a vault holding one is not.
_REDACTIONS = [
(re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----.*?-----END [A-Z ]*PRIVATE KEY-----", re.S), "[REDACTED PRIVATE KEY]"),
(re.compile(r"(?i)\b(bearer|token)\s+[A-Za-z0-9._~+/=-]{16,}"), r"\1 [REDACTED]"),
(re.compile(r"(?i)((?:password|passwd|pwd|secret|token|api[_-]?key|access[_-]?key|private[_-]?key|client[_-]?secret|NB_API_KEY|PAT)[\"']?\s*[:=]\s*[\"']?)([^\s\"'`,;]{6,})"), r"\1[REDACTED]"),
(re.compile(r"\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}"), "[REDACTED JWT]"),
(re.compile(r"\b(?:sk|pk|rk)-[A-Za-z0-9_-]{20,}|\bgh[pousr]_[A-Za-z0-9]{30,}|\bglpat-[A-Za-z0-9_-]{20,}|\bxox[abpr]-[A-Za-z0-9-]{10,}"), "[REDACTED TOKEN]"),
# >=48 hex: API keys / sha256 tokens. Deliberately leaves 40-hex git SHAs and 32-hex md5s.
(re.compile(r"\b[a-fA-F0-9]{48,}\b"), "[REDACTED HEX]"),
# Random mixed-case+digit strings (generated passwords). No '/' so paths survive.
(re.compile(r"(?<![A-Za-z0-9_./-])(?=[A-Za-z0-9+]*[0-9])(?=[A-Za-z0-9+]*[a-z])(?=[A-Za-z0-9+]*[A-Z])[A-Za-z0-9+]{20,}={0,2}(?![A-Za-z0-9_./-])"), "[REDACTED]"),
]
def redact(text: str) -> str:
for pat, repl in _REDACTIONS:
text = pat.sub(repl, text)
return text
# --------------------------------------------------------------------------- parsing
_NOISE_BLOCKS = re.compile(
r"<(system-reminder|local-command-stdout|local-command-stderr|local-command-caveat|task-notification)>.*?</\1>",
re.S,
)
_CMD = re.compile(r"<command-name>(.*?)</command-name>.*?(?:<command-args>(.*?)</command-args>)?", re.S)
MAX_MSG = 6000
def _clip(text: str) -> str:
return text if len(text) <= MAX_MSG else text[:4000] + "\n[...clipped...]\n" + text[-1500:]
def _clean_user(text: str) -> str:
text = _NOISE_BLOCKS.sub("", text)
m = _CMD.search(text)
if m:
args = (m.group(2) or "").strip()
text = f"{m.group(1).strip()} {args}".strip()
text = re.sub(r"</?command-(message|args|name)>", "", text)
return text.strip()
def _tool_line(block: dict) -> str:
name = block.get("name", "tool")
inp = block.get("input") or {}
desc = inp.get("description") or inp.get("command") or inp.get("file_path") or inp.get("query") \
or inp.get("url") or inp.get("prompt") or inp.get("pattern") or ""
desc = " ".join(str(desc).split())[:160]
return f"[{name}: {desc}]" if desc else f"[{name}]"
def parse_claude_code(path: pathlib.Path) -> dict:
meta = {"session_id": path.stem, "cwd": None, "branch": None, "first": None, "last": None,
"title": None, "version": None, "prompts": 0}
turns: list[list[str]] = []
for line in path.open(encoding="utf-8", errors="replace"):
try:
r = json.loads(line)
except json.JSONDecodeError:
continue
t = r.get("type")
if t == "ai-title" and r.get("aiTitle"):
meta["title"] = r["aiTitle"]
if t == "custom-title" and r.get("customTitle"):
meta["title"] = r["customTitle"]
if t not in ("user", "assistant") or r.get("isSidechain") or r.get("isMeta"):
continue
ts = r.get("timestamp")
if ts:
meta["first"] = meta["first"] or ts
meta["last"] = ts
meta["cwd"] = meta["cwd"] or r.get("cwd")
meta["branch"] = r.get("gitBranch") or meta["branch"]
meta["version"] = r.get("version") or meta["version"]
content = (r.get("message") or {}).get("content")
if t == "user":
if isinstance(content, str):
text = content
elif isinstance(content, list):
text = "\n".join(b.get("text", "") for b in content if b.get("type") == "text")
else:
continue
text = _clean_user(text)
if text:
role = "SUMMARY-OF-EARLIER-CONTEXT" if r.get("isCompactSummary") else "USER"
turns.append([role, _clip(text)])
meta["prompts"] += role == "USER"
else:
parts = []
for b in content if isinstance(content, list) else []:
if b.get("type") == "text" and b.get("text", "").strip():
parts.append(_clip(b["text"].strip()))
elif b.get("type") == "tool_use":
parts.append(_tool_line(b))
if not parts:
continue
if turns and turns[-1][0] == "ASSISTANT": # a tool loop is many records
turns[-1][1] += "\n" + "\n".join(parts)
else:
turns.append(["ASSISTANT", "\n".join(parts)])
meta["transcript"] = "\n\n".join(f"{role}: {text}" for role, text in turns)
return meta
def parse_claude_ai_export(src: pathlib.Path) -> list[dict]:
"""claude.ai 'Export data' -> conversations.json. Tolerant of field-name drift."""
if src.suffix == ".zip":
with zipfile.ZipFile(src) as z:
name = next((n for n in z.namelist() if n.endswith("conversations.json")), None)
if not name:
raise SystemExit(f"no conversations.json inside {src}")
data = json.loads(z.read(name))
else:
data = json.loads(src.read_text())
out = []
for c in data if isinstance(data, list) else data.get("conversations", []):
turns, count = [], 0
for m in c.get("chat_messages") or c.get("messages") or []:
text = m.get("text") or ""
if not text and isinstance(m.get("content"), list):
text = "\n".join(b.get("text", "") for b in m["content"] if b.get("type") == "text")
for a in m.get("attachments") or []:
if a.get("extracted_content"):
text += f"\n[attachment {a.get('file_name', '')}]: {a['extracted_content'][:1500]}"
if not text.strip():
continue
role = "USER" if (m.get("sender") or m.get("role")) in ("human", "user") else "ASSISTANT"
turns.append(f"{role}: {_clip(text.strip())}")
count += role == "USER"
if not turns:
continue
out.append({
"session_id": c.get("uuid") or hashlib.sha1(json.dumps(c, sort_keys=True).encode()).hexdigest(),
"title": c.get("name") or None, "first": c.get("created_at"), "last": c.get("updated_at"),
"cwd": None, "branch": None, "version": None, "prompts": count,
"transcript": "\n\n".join(turns),
})
return out
# --------------------------------------------------------------------------- LLM
SYSTEM = """You write concise knowledge-base notes summarizing a conversation between a user and Claude.
The reader is the same user, months later, searching their notes. Capture what happened and
what was learned, not a play-by-play. Be specific: name the systems, files, commands, errors
and decisions. Never include passwords, API keys, tokens or other secrets, even if present;
refer to them generically ("the API key"). Text marked [REDACTED] must stay redacted.
Reply with ONLY a JSON object, no prose, with these keys:
"title": short specific title (<= 70 chars)
"summary": 3-6 sentences
"outcomes": list of what was actually built, fixed or answered
"decisions": list of choices made and why
"facts": list of reusable facts, gotchas, root causes worth remembering
"open_threads": list of unfinished work or follow-ups
"tags": 3-8 lowercase kebab-case topic tags
Use empty lists where nothing applies."""
CHUNK_SYSTEM = """You are condensing one part of a long conversation between a user and Claude so that a
later step can summarize the whole. Output dense bullet notes: goals, actions taken, results,
errors and their causes, decisions, open questions. Be specific. Never reproduce secrets."""
CHUNK_CHARS = 300_000
class Ledger:
"""Append-only record of every model call: tokens and list-price cost. Also enforces budgets."""
def __init__(self, cfg: dict):
self.path = pathlib.Path(os.path.expanduser(cfg["STATE_FILE"])).parent / "usage.jsonl"
self.run_budget = float(cfg["BUDGET_USD_PER_RUN"])
self.day_budget = float(cfg["BUDGET_USD_PER_DAY"])
self.lock = threading.Lock()
self.run_cost = 0.0
self.run_calls = 0
self.run_in = self.run_out = 0
def entries(self, since: dt.datetime | None = None) -> list[dict]:
if not self.path.exists():
return []
out = []
for line in self.path.read_text().splitlines():
try:
e = json.loads(line)
except json.JSONDecodeError:
continue
if since is None or dt.datetime.fromisoformat(e["ts"]) >= since:
out.append(e)
return out
def spent_since(self, hours: float) -> float:
since = dt.datetime.now(dt.timezone.utc) - dt.timedelta(hours=hours)
return sum(e.get("cost_usd") or 0 for e in self.entries(since))
def check(self, estimate: float = 0.0) -> None:
"""Called BEFORE each model call with that call's estimated cost -- so a budget is never overshot."""
with self.lock:
if self.run_cost + estimate > self.run_budget:
raise BudgetExhausted(f"run budget ${self.run_budget:.2f}: ${self.run_cost:.3f} spent, "
f"next call est. ${estimate:.3f}")
day = self.spent_since(24)
if day + estimate > self.day_budget:
raise BudgetExhausted(f"24h budget ${self.day_budget:.2f}: ${day:.3f} spent, next call est. ${estimate:.3f}")
def record(self, label: str, model: str, usage: dict, cost: float | None) -> None:
e = {"ts": dt.datetime.now(dt.timezone.utc).isoformat(timespec="seconds"), "item": label, "model": model,
"input": (usage.get("input_tokens") or 0) + (usage.get("cache_creation_input_tokens") or 0)
+ (usage.get("cache_read_input_tokens") or 0),
"output": usage.get("output_tokens") or 0, "cost_usd": cost}
with self.lock:
self.path.parent.mkdir(parents=True, exist_ok=True)
with self.path.open("a") as f:
f.write(json.dumps(e) + "\n")
self.run_calls += 1
self.run_cost += cost or 0
self.run_in += e["input"]
self.run_out += e["output"]
def report(self) -> str:
return (f"usage this run: {self.run_calls} model calls, {self.run_in:,} in / {self.run_out:,} out tokens, "
f"${self.run_cost:.3f} list-price equiv | last 24h ${self.spent_since(24):.3f} of ${self.day_budget:.2f}")
class BudgetExhausted(RuntimeError):
"""A configured spend budget is reached -- stop the run the same way as a usage limit."""
class QuotaExhausted(RuntimeError):
"""The plan/API usage limit is hit. Every further call would fail the same way -- stop the run."""
_QUOTA = re.compile(r"(?i)hit your (?:session |usage |weekly )?limit|usage limit|quota|credit balance|resets? (?:at )?\d")
class LLM:
"""Two backends.
claude-cli (default): headless Claude Code on the user's own plan -- no API key to manage.
--no-session-persistence is load-bearing: without it every summary writes a new transcript
into ~/.claude/projects and the next sync would summarize its own summarizing.
openai: any OpenAI-compatible /chat/completions endpoint (LiteLLM, local models).
"""
def __init__(self, cfg: dict, ledger: "Ledger | None" = None):
self.ledger = ledger or Ledger(cfg)
self.price_in = float(cfg["LLM_PRICE_IN_PER_M"]) / 1e6
self.price_out = float(cfg["LLM_PRICE_OUT_PER_M"]) / 1e6
self.label = threading.local()
self.backend = cfg["LLM_BACKEND"]
self.model = cfg["LLM_MODEL"]
self.resolved = cfg["LLM_MODEL"]
if self.backend == "claude-cli":
self.bin = shutil.which(os.path.expanduser(cfg["CLAUDE_BIN"]))
if not self.bin:
raise SystemExit(f"claude CLI not found ({cfg['CLAUDE_BIN']})")
self.workdir = pathlib.Path(os.path.expanduser(cfg["STATE_FILE"])).parent # no project CLAUDE.md here
self.workdir.mkdir(parents=True, exist_ok=True)
elif self.backend == "openai":
if not cfg["LLM_BASE_URL"]:
raise SystemExit("LLM_BASE_URL is not set (see config.env.example)")
self.url = cfg["LLM_BASE_URL"].rstrip("/") + "/chat/completions"
self.key = secret(cfg, "LLM_API_KEY_FILE", "LLM_API_KEY_ENV")
else:
raise SystemExit(f"unknown LLM_BACKEND {self.backend!r}")
def estimate(self, system: str, user: str, max_tokens: int) -> float:
# ~3 chars/token over-estimates input (measured 3.1 on transcripts); assume the full output;
# +20% margin for anything unmeasured. Errs high on purpose: refusing early beats overshooting.
return 1.2 * ((len(system) + len(user)) / 3.0 * self.price_in + max_tokens * self.price_out)
def chat(self, system: str, user: str, max_tokens: int = 2500) -> str:
self.ledger.check(self.estimate(system, user, max_tokens))
return self._chat_cli(system, user) if self.backend == "claude-cli" else self._chat_openai(system, user, max_tokens)
def _chat_cli(self, system: str, user: str) -> str:
# Cost levers, measured on one session: $0.057 -> $0.015 with identical summary content.
# alwaysThinkingEnabled=false -- extended thinking was ~85% of output tokens
# DISABLE_PROMPT_CACHING=1 -- Claude Code writes prompts to the 1h cache at 2x input
# price; a one-shot call never reads it back
# CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 -- otherwise a hidden side request re-reads the
# whole prompt (~20 tokens out, title-generation shaped),
# doubling billed input: 9,638 vs 4,663 visible tokens.
cmd = [self.bin, "-p", "--model", self.model, "--no-session-persistence", "--tools", "",
"--strict-mcp-config", "--output-format", "json",
"--settings", json.dumps({"alwaysThinkingEnabled": False}), "--system-prompt", system]
env = dict(os.environ, DISABLE_PROMPT_CACHING="1", CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC="1")
last = ""
for attempt in range(5):
try:
proc = subprocess.run(cmd, input=user, capture_output=True, text=True, timeout=900, cwd=self.workdir, env=env)
except subprocess.TimeoutExpired:
last = "timeout"
else:
try:
d = json.loads(proc.stdout)
except json.JSONDecodeError:
d = {"is_error": True, "result": (proc.stdout or proc.stderr)[-300:]}
if d.get("usage") or d.get("total_cost_usd"): # failed calls can still cost
# modelUsage is what was BILLED across every request the CLI made; `usage` is only
# the visible turn and undercounted by half before the nonessential-traffic fix.
mu = list((d.get("modelUsage") or {}).values())
billed = {"input_tokens": sum((v.get("inputTokens") or 0) + (v.get("cacheReadInputTokens") or 0)
+ (v.get("cacheCreationInputTokens") or 0) for v in mu),
"output_tokens": sum(v.get("outputTokens") or 0 for v in mu)} if mu else (d.get("usage") or {})
self.ledger.record(getattr(self.label, "v", "?"), self.model, billed, d.get("total_cost_usd"))
if not d.get("is_error") and d.get("result"):
used = list((d.get("modelUsage") or {}).keys())
if used:
self.resolved = used[0]
return d["result"]
last = str(d.get("result") or proc.stderr)[-300:]
if _QUOTA.search(last):
raise QuotaExhausted(last.strip())
if not re.search(r"(?i)rate.?limit|overloaded|429|529|timeout|temporar", last):
break # not transient -- do not hammer
time.sleep(2 ** attempt * 10)
raise RuntimeError(f"claude -p failed: {last}")
def _chat_openai(self, system: str, user: str, max_tokens: int) -> str:
body = json.dumps({"model": self.model, "temperature": 0.2, "max_tokens": max_tokens,
"messages": [{"role": "system", "content": system},
{"role": "user", "content": user}]}).encode()
for attempt in range(5):
req = urllib.request.Request(self.url, data=body, method="POST", headers={
"Authorization": f"Bearer {self.key}", "Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=600) as r:
d = json.load(r)
u = d.get("usage") or {}
self.ledger.record(getattr(self.label, "v", "?"), self.model,
{"input_tokens": u.get("prompt_tokens"), "output_tokens": u.get("completion_tokens")}, None)
return d["choices"][0]["message"]["content"]
except urllib.error.HTTPError as e:
detail = e.read()[:300].decode(errors="replace")
if _QUOTA.search(detail):
raise QuotaExhausted(detail) from None
if e.code in (429, 500, 502, 503, 504, 529) and attempt < 4:
time.sleep(2 ** attempt * 5)
continue
raise RuntimeError(f"LLM HTTP {e.code}: {detail}") from None
except (urllib.error.URLError, TimeoutError) as e:
if attempt < 4:
time.sleep(2 ** attempt * 5)
continue
raise RuntimeError(f"LLM unreachable: {e}") from None
raise RuntimeError("LLM retries exhausted")
def summarize(self, meta: dict) -> dict:
text = redact(meta["transcript"])
if len(text) > CHUNK_CHARS: # map-reduce for long sessions
chunks, cur = [], []
size = 0
for part in text.split("\n\n"):
if size + len(part) > CHUNK_CHARS and cur:
chunks.append("\n\n".join(cur)); cur, size = [], 0
cur.append(part); size += len(part) + 2
chunks.append("\n\n".join(cur))
notes = [self.chat(CHUNK_SYSTEM, f"Part {i + 1} of {len(chunks)}:\n\n{c}", 3000)
for i, c in enumerate(chunks)]
text = "Condensed notes from each part of a long conversation, in order:\n\n" + \
"\n\n---\n\n".join(notes)
header = f"Working directory: {meta.get('cwd')}\nGit branch: {meta.get('branch')}\n" \
f"Existing title: {meta.get('title')}\n\n"
raw = self.chat(SYSTEM, header + text)
m = re.search(r"\{.*\}", raw, re.S)
if not m:
raise RuntimeError(f"LLM did not return JSON: {raw[:200]!r}")
out = json.loads(m.group(0))
for k in ("outcomes", "decisions", "facts", "open_threads", "tags"):
out[k] = [redact(str(x)) for x in (out.get(k) or []) if str(x).strip()]
out["summary"] = redact(str(out.get("summary", "")))
out["title"] = redact(str(out.get("title") or meta.get("title") or meta["session_id"]))
return out
# --------------------------------------------------------------------------- vault
class Vault:
def __init__(self, cfg: dict):
# OBSIDIAN_API_URL may list several URLs, comma-separated, tried in order at ping() -- e.g. the
# permanent address first and a port-forward second, so one config works either way.
self.candidates = [u.strip().rstrip("/") for u in cfg["OBSIDIAN_API_URL"].split(",") if u.strip()]
self.base = self.candidates[0]
self.key = secret(cfg, "OBSIDIAN_API_KEY_FILE")
def put(self, path: str, content: str) -> None:
req = urllib.request.Request(f"{self.base}/vault/{urllib.parse.quote(path)}",
data=content.encode(), method="PUT", headers={
"Authorization": f"Bearer {self.key}",
"Content-Type": "text/markdown"})
with urllib.request.urlopen(req, timeout=60) as r:
if r.status not in (200, 201, 204):
raise RuntimeError(f"vault PUT {path}: HTTP {r.status}")
def ping(self) -> None:
"""Checked BEFORE any summarizing, so an unreachable vault never spends usage limits."""
errors = []
for base in self.candidates:
req = urllib.request.Request(f"{base}/", headers={"Authorization": f"Bearer {self.key}"})
try:
with urllib.request.urlopen(req, timeout=10) as r:
info = json.load(r)
self.base = base
break
except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e:
errors.append(f"{base}: {getattr(e, 'reason', e)}")
else:
raise SystemExit("Obsidian REST API unreachable -- " + "; ".join(errors))
print(f"vault: {self.base}", flush=True)
if not info.get("authenticated"):
raise SystemExit("Obsidian REST API reachable but the API key was rejected")
_BAD = re.compile(r'[\\/:*?"<>|#^\[\]]+')
def note_path(cfg: dict, kind: str, meta: dict, title: str) -> str:
when = (meta.get("first") or dt.datetime.now(dt.timezone.utc).isoformat())[:10]
slug = " ".join(_BAD.sub(" ", title).split())[:60].strip() or "untitled"
return f"{cfg['VAULT_FOLDER']}/{kind}/{when[:4]}/{when} {slug} ({meta['session_id'][:8]}).md"
def render(meta: dict, s: dict, source: str, model: str, src_file: str | None) -> str:
project = pathlib.Path(meta["cwd"]).name if meta.get("cwd") else None
tags = ["claude", source] + [t for t in s["tags"] if t not in ("claude", source)]
fm = {
"type": "claude-conversation", "source": source, "session_id": meta["session_id"],
"title": s["title"], "project": project, "cwd": meta.get("cwd"), "branch": meta.get("branch"),
"started": meta.get("first"), "ended": meta.get("last"), "prompts": meta.get("prompts"),
"summarized_by": model, "synced": dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
"tags": tags,
}
lines = ["---"] + [f"{k}: {json.dumps(v)}" for k, v in fm.items() if v not in (None, "")] + ["---", "",
f"# {s['title']}", "", "> [!summary]"] + [f"> {l}" for l in s["summary"].splitlines() or [""]]
for heading, key in (("Outcomes", "outcomes"), ("Decisions", "decisions"),
("Facts & gotchas", "facts"), ("Open threads", "open_threads")):
if s[key]:
lines += ["", f"## {heading}", ""] + [f"- {x}" for x in s[key]]
lines += ["", "---"]
if source == "claude-code":
lines.append(f"Resume: `claude --resume {meta['session_id']}`" + (f" · transcript `{src_file}`" if src_file else ""))
else:
lines.append(f"claude.ai conversation `{meta['session_id']}`")
return "\n".join(lines) + "\n"
# --------------------------------------------------------------------------- state
class State:
def __init__(self, path: str):
self.path = pathlib.Path(os.path.expanduser(path))
self.lock = threading.Lock()
self.data = json.loads(self.path.read_text()) if self.path.exists() else {}
def save(self) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True)
tmp = self.path.with_suffix(".tmp")
tmp.write_text(json.dumps(self.data, indent=1, sort_keys=True))
tmp.replace(self.path)
def record(self, key: str, value: dict) -> None:
with self.lock:
self.data[key] = value
self.save()
def write_index(cfg: dict, vault: Vault, state: State) -> None:
rows = sorted((v for v in state.data.values() if v.get("note")), key=lambda v: v.get("started") or "", reverse=True)
out = ["---", 'type: "claude-index"', "---", "", "# Claude conversations", "",
f"{len(rows)} conversations. Regenerated by claude-obsidian-sync.", "",
"| Date | Title | Project | Source |", "|---|---|---|---|"]
for v in rows:
link = v["note"][:-3]
title = v.get("title", "").replace("|", "-")
out.append(f"| {(v.get('started') or '')[:10]} | [[{link}\\|{title}]] | {v.get('project') or ''} | {v.get('source')} |")
vault.put(f"{cfg['VAULT_FOLDER']}/Index.md", "\n".join(out) + "\n")
# --------------------------------------------------------------------------- commands
def top_level_sessions(cfg: dict) -> list[pathlib.Path]:
root = pathlib.Path(os.path.expanduser(cfg["CLAUDE_PROJECTS_DIR"]))
return sorted(root.glob("*/*.jsonl"), key=lambda p: p.stat().st_mtime) # nested = subagents
def process(cfg, llm, vault, state, key, meta, source, kind, fingerprint, src_file, dry_run) -> str:
# Cache the summary until the note is written, so a vault failure never pays for it twice.
cache = pathlib.Path(os.path.expanduser(cfg["STATE_FILE"])).parent / "cache" / \
f"{hashlib.sha1(key.encode()).hexdigest()[:16]}-{hashlib.sha1(fingerprint.encode()).hexdigest()[:8]}.json"
llm.label.v = key
if cache.exists():
s = json.loads(cache.read_text())
else:
s = llm.summarize(meta)
cache.parent.mkdir(parents=True, exist_ok=True)
cache.write_text(json.dumps(s))
prev = state.data.get(key, {})
path = prev.get("note") or note_path(cfg, kind, meta, s["title"]) # stable across re-syncs
body = render(meta, s, source, llm.resolved, src_file)
if dry_run == "prefetch":
return f"summary cached ({path})"
if dry_run:
print(f"\n===== {path} =====\n{body}")
return path
vault.put(path, body)
cache.unlink(missing_ok=True)
state.record(key, {"note": path, "title": s["title"], "fingerprint": fingerprint, "source": source,
"started": meta.get("first"),
"project": pathlib.Path(meta["cwd"]).name if meta.get("cwd") else None})
return path
SKIPPED = "(empty, skipped)"
EX_TEMPFAIL = 75 # "try again later" -- systemd logs it, the next timer run retries
def run_jobs(cfg, work, fn) -> tuple[int, int, str | None, int]:
ok = failed = skipped = 0
quota = None
stop = threading.Event() # Future.cancel() cannot stop a job a worker already picked up
def guarded(*w):
if stop.is_set():
raise cf.CancelledError()
try:
return fn(*w)
except (QuotaExhausted, BudgetExhausted):
stop.set()
raise
with cf.ThreadPoolExecutor(max_workers=int(cfg["JOBS"])) as ex:
futs = {ex.submit(guarded, *w): w for w in work}
for fut in cf.as_completed(futs):
label = futs[fut][0]
try:
result = fut.result()
if result == SKIPPED:
print(f"skip {label} (no conversation content)", flush=True); skipped += 1
else:
print(f"ok {label} -> {result}", flush=True); ok += 1
except (QuotaExhausted, BudgetExhausted) as e:
if quota is None:
quota = str(e)
for f in futs: # don't launch the rest
f.cancel()
except cf.CancelledError:
pass
except Exception as e: # keep going; next run retries
print(f"FAIL {label}: {e}", file=sys.stderr, flush=True); failed += 1
return ok, failed, quota, skipped
def finish(ok: int, failed: int, quota: str | None, dry_run, remaining: int, ledger: "Ledger | None" = None) -> None:
verb = {"prefetch": "summarized into cache (vault untouched)"}.get(dry_run) or \
("rendered (dry run, nothing written)" if dry_run else "written")
print(f"done: {ok} {verb}, {failed} failed, {remaining} still queued")
if ledger:
print(ledger.report())
if quota:
print(f"stopped: {quota}", file=sys.stderr)
sys.exit(EX_TEMPFAIL)
sys.exit(1 if failed else 0)
def cmd_usage(cfg, args) -> None:
led = Ledger(cfg)
rows = led.entries()
if not rows:
print("no model calls recorded yet"); return
for label, hours in (("last 5h", 5), ("last 24h", 24), ("last 7d", 168)):
since = dt.datetime.now(dt.timezone.utc) - dt.timedelta(hours=hours)
es = [e for e in rows if dt.datetime.fromisoformat(e["ts"]) >= since]
print(f"{label:>9}: {len(es):4d} calls {sum(e['input'] for e in es):>10,} in "
f"{sum(e['output'] for e in es):>8,} out ${sum(e.get('cost_usd') or 0 for e in es):.3f}")
print(f"budgets: ${led.run_budget:.2f}/run, ${led.day_budget:.2f}/24h (list-price equiv; plan meter: /usage in Claude Code)")
def cmd_stats(cfg, args) -> None:
total = 0
rows = []
for p in top_level_sessions(cfg):
m = parse_claude_code(p)
n = len(redact(m["transcript"]))
total += n
rows.append((n, p.stat().st_size, m["prompts"], m["title"] or p.stem))
rows.sort(reverse=True)
for n, raw, msgs, title in rows[: args.top]:
print(f"{raw / 1e6:7.2f} MB raw -> {n / 1e3:7.1f} KB condensed {msgs:4d} prompts {title[:60]}")
print(f"\n{len(rows)} sessions, {total / 1e6:.1f} MB condensed (~{total / 4 / 1e6:.2f}M tokens in), "
f"{sum(1 for r in rows if r[0] > CHUNK_CHARS)} need map-reduce")
def run_lock(cfg: dict):
"""One run at a time: a timer run overlapping a manual one would race on state and ledger files."""
path = pathlib.Path(os.path.expanduser(cfg["STATE_FILE"])).parent / "run.lock"
path.parent.mkdir(parents=True, exist_ok=True)
fh = path.open("w")
try:
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
print("another claude-obsidian-sync run is in progress -- exiting")
sys.exit(0)
return fh # held until the process exits
def cmd_sync(cfg, args) -> None:
_lock = run_lock(cfg)
state = State(cfg["STATE_FILE"])
settle = dt.timedelta(minutes=int(cfg["SETTLE_MINUTES"]))
now = time.time()
work = []
for p in top_level_sessions(cfg):
if args.session and p.stem != args.session:
continue
st = p.stat()
fp = f"{st.st_size}:{int(st.st_mtime)}"
key = f"claude-code:{p.stem}"
if not args.force and state.data.get(key, {}).get("fingerprint") == fp:
continue
if not (args.force or args.session) and now - st.st_mtime < settle.total_seconds():
continue # still being written
work.append((p, fp, key))
total = len(work)
cap = args.limit or int(cfg["MAX_PER_RUN"])
work = work[-cap:] if not args.oldest else work[:cap] # newest first by default
print(f"{total} session(s) pending, summarizing {len(work)} this run", flush=True)
if not work:
return
prefetch = getattr(args, "prefetch", False)
llm, vault = LLM(cfg), (None if args.dry_run or prefetch else Vault(cfg))
if vault:
vault.ping()
mode = "prefetch" if prefetch else args.dry_run
def one(p, fp, key):
meta = parse_claude_code(p)
if not meta["transcript"].strip():
if not args.dry_run and not prefetch:
state.record(key, {"note": None, "fingerprint": fp, "source": "claude-code"})
return SKIPPED
src = str(p).replace(str(HOME), "~")
return process(cfg, llm, vault, state, key, meta, "claude-code", "Code", fp, src, mode)
ok, failed, quota, skipped = run_jobs(cfg, [(w[0].name, *w) for w in work], lambda _l, p, fp, key: one(p, fp, key))
if vault and ok:
write_index(cfg, vault, state)
finish(ok, failed, quota, mode, total - ok - skipped, llm.ledger)
def cmd_import(cfg, args) -> None:
_lock = run_lock(cfg)
state = State(cfg["STATE_FILE"])
convs = parse_claude_ai_export(pathlib.Path(args.export))
work = []
for m in convs:
key = f"claude-ai:{m['session_id']}"
fp = m.get("last") or str(len(m["transcript"]))
if args.force or state.data.get(key, {}).get("fingerprint") != fp:
work.append((m, fp, key))
total = len(work)
work = work[: args.limit or int(cfg["MAX_PER_RUN"])]
print(f"{len(convs)} conversations in export, {total} new or changed, summarizing {len(work)} this run", flush=True)
if not work:
return
llm, vault = LLM(cfg), (None if args.dry_run else Vault(cfg))
if vault:
vault.ping()
ok, failed, quota, skipped = run_jobs(cfg, [((m.get("title") or m["session_id"])[:50], m, fp, key) for m, fp, key in work],
lambda _l, m, fp, key: process(cfg, llm, vault, state, key, m, "claude-ai", "Web",
fp, None, args.dry_run))
if vault and ok:
write_index(cfg, vault, state)
finish(ok, failed, quota, args.dry_run, total - ok - skipped, llm.ledger)
def main() -> None:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = ap.add_subparsers(dest="cmd", required=True)
sub.add_parser("usage", help="model calls, tokens and spend from the local ledger")
s = sub.add_parser("stats", help="condensed sizes, no network")
s.add_argument("--top", type=int, default=15)
s = sub.add_parser("sync", help="summarize Claude Code sessions")
s.add_argument("--dry-run", action="store_true", help="print notes instead of writing them")
s.add_argument("--prefetch", action="store_true",
help="summarize into the local cache without touching the vault; a later sync writes them at no extra cost")
s.add_argument("--force", action="store_true", help="ignore state and the settle window")
s.add_argument("--session", help="only this session id")
s.add_argument("--limit", type=int, help="max sessions this run (default MAX_PER_RUN)")
s.add_argument("--oldest", action="store_true", help="take the oldest pending sessions instead of the newest")
s = sub.add_parser("import-claude-ai", help="import a claude.ai data export")
s.add_argument("export", help="the export .zip or its conversations.json")
s.add_argument("--dry-run", action="store_true")
s.add_argument("--force", action="store_true")
s.add_argument("--limit", type=int, help="max conversations this run (default MAX_PER_RUN)")
args = ap.parse_args()
cfg = load_config()
{"stats": cmd_stats, "usage": cmd_usage, "sync": cmd_sync, "import-claude-ai": cmd_import}[args.cmd](cfg, args)
if __name__ == "__main__":
main()
+28
View File
@@ -0,0 +1,28 @@
# ~/.config/claude-obsidian/config.env (chmod 600; never commit the real one)
# Every key can also be set as an environment variable, which wins.
# Obsidian Local REST API plugin -- the plain-HTTP server (Settings -> Local REST API ->
# Enable HTTP server). Point this at wherever that port is reachable from this machine.
# Several URLs, comma-separated, are tried in order -- e.g. a permanent address, then a port-forward.
OBSIDIAN_API_URL=http://obsidian.example.internal:27123,http://127.0.0.1:27123
OBSIDIAN_API_KEY_FILE=~/.config/claude-obsidian/api-key
# Summarizer. claude-cli = headless `claude -p` on your own Claude plan (no API key needed).
LLM_BACKEND=claude-cli
LLM_MODEL=haiku # sonnet writes noticeably sharper notes for ~1-3x the usage; set prices to match below
# LLM_PRICE_IN_PER_M=1.0 # used only for the pre-call budget estimate (haiku list; sonnet 3.0 / 15.0)
# LLM_PRICE_OUT_PER_M=5.0
# CLAUDE_BIN=claude
# Or any OpenAI-compatible endpoint (LiteLLM, a local model):
# LLM_BACKEND=openai
# LLM_BASE_URL=https://litellm.example/v1
# LLM_MODEL=some-model
# LLM_API_KEY_FILE=~/.config/claude-obsidian/llm-key # or LLM_API_KEY_ENV=LITELLM_KEY
VAULT_FOLDER=Claude
SETTLE_MINUTES=30 # leave sessions alone until they have been idle this long
JOBS=1
MAX_PER_RUN=10 # summaries share your plan's usage limits; the timer drips a backlog
BUDGET_USD_PER_RUN=0.50 # list-price USD, refused pre-flight; see `claude-obsidian-sync usage`
BUDGET_USD_PER_DAY=2.00
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# Install claude-obsidian-sync for the current user: binary into ~/.local/bin, systemd user
# timer enabled. Idempotent. Does not create config or secrets -- see README.md.
set -eu
HERE="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
mkdir -p "$HOME/.local/bin" "$HOME/.config/systemd/user" "$HOME/.config/claude-obsidian"
chmod 700 "$HOME/.config/claude-obsidian"
ln -sf "$HERE/claude-obsidian-sync" "$HOME/.local/bin/claude-obsidian-sync"
cp "$HERE/systemd/claude-obsidian-sync.service" "$HERE/systemd/claude-obsidian-sync.timer" "$HOME/.config/systemd/user/"
[ -f "$HOME/.config/claude-obsidian/config.env" ] || { cp "$HERE/config.env.example" "$HOME/.config/claude-obsidian/config.env"; chmod 600 "$HOME/.config/claude-obsidian/config.env"; echo "created config.env from the example -- edit it"; }
systemctl --user daemon-reload
systemctl --user enable --now claude-obsidian-sync.timer
echo "installed. check: systemctl --user list-timers claude-obsidian-sync.timer"
echo "timers only run while logged in unless lingering is on: loginctl enable-linger \"\$USER\""
@@ -0,0 +1,11 @@
[Unit]
Description=Summarize Claude conversations into Obsidian
After=network-online.target
[Service]
Type=oneshot
ExecStart=%h/.local/bin/claude-obsidian-sync sync
Environment=PATH=%h/.local/bin:/usr/local/bin:/usr/bin:/bin
# 75 = usage limit reached. The run stopped cleanly and the next timer run resumes -- not a failure.
SuccessExitStatus=75
Nice=10
@@ -0,0 +1,10 @@
[Unit]
Description=Hourly Claude -> Obsidian conversation sync
[Timer]
OnCalendar=hourly
RandomizedDelaySec=10m
Persistent=true
[Install]
WantedBy=timers.target
-114
View File
@@ -1,114 +0,0 @@
#!/usr/bin/env sh
# docker/install.sh — install full (rootful) Docker Engine and let the
# invoking user talk to it without sudo.
#
# What it does:
# 1. Installs Docker Engine from Docker's upstream package repo. Skipped when
# `docker` is already installed. Debian-family hosts go through Docker's
# convenience script (get.docker.com); RHEL-family hosts wire up the
# docker-ce repo directly — see install_rhel() for why.
# 2. Enables and starts the service (systemd hosts only).
# 3. Adds the invoking user to the `docker` group so the daemon socket is
# usable without sudo. Takes effect at next login.
#
# Deliberately no-ops when a rootless Docker daemon is detected (e.g. Kasm
# ships its own rootless setup) — installing rootful Docker there would fight
# the existing runtime. See docker-rootless-cgroup/ for the Kasm-side config.
#
# Idempotent and safe to re-run. Uses sudo.
set -eu
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
# shellcheck source=../lib/os-detect.sh
. "$REPO_ROOT/lib/os-detect.sh"
say() { printf '%s\n' "$*" >&2; }
DOCKER_PKGS="docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin"
# install_rhel — install docker-ce from Docker's EL repo.
#
# WHY NOT get.docker.com: the convenience script allowlists distros by
# /etc/os-release ID and aborts on anything it does not recognize —
# "ERROR: Unsupported distribution 'ol'" on Oracle Linux, and the same for
# other EL rebuilds it has not enumerated. On a match it does nothing more
# than what this function does: add the repo, then install the packages.
# Wiring the repo up directly drops the allowlist, so every EL rebuild
# (Oracle, Rocky, Alma, CentOS, RHEL) installs from one code path.
#
# The repo URL is keyed on $releasever/$basearch, which dnf expands per host,
# so EL9 and EL10 both resolve without a version table here.
install_rhel() {
say "docker: adding Docker CE repo (EL \$releasever/\$basearch)..."
sudo tee /etc/yum.repos.d/docker-ce.repo >/dev/null <<'REPO'
[docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://download.docker.com/linux/centos/$releasever/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
REPO
# --allowerasing: containerd.io conflicts with the distro's own runc/containerd
# on EL rebuilds that ship them (Oracle Linux does). Let dnf swap them out
# rather than dead-end on the conflict.
if command -v dnf >/dev/null 2>&1; then
# shellcheck disable=SC2086
sudo dnf install -y --allowerasing $DOCKER_PKGS
else
# shellcheck disable=SC2086
sudo yum install -y --allowerasing $DOCKER_PKGS
fi
}
# install_debian — Docker's convenience script; it supports every Debian-family
# distro this repo targets and sets up the apt repo and keyring.
install_debian() {
say "docker: installing Docker Engine via get.docker.com..."
tmp="$(mktemp)"
if command -v curl >/dev/null 2>&1; then
curl -fsSL https://get.docker.com -o "$tmp"
elif command -v wget >/dev/null 2>&1; then
wget -qO "$tmp" https://get.docker.com
else
say "docker: need curl or wget"; rm -f "$tmp"; return 1
fi
sudo sh "$tmp"
rm -f "$tmp"
}
# --- skip on rootless-Docker hosts -------------------------------------------
if command -v docker >/dev/null 2>&1 &&
docker info -f '{{println .SecurityOptions}}' 2>/dev/null | grep -q 'name=rootless'; then
say "docker: rootless Docker daemon detected; leaving it alone."
exit 0
fi
# --- 1. install engine -------------------------------------------------------
if command -v docker >/dev/null 2>&1; then
say "docker: engine already installed ($(docker --version 2>/dev/null || echo 'version unknown'))."
else
case "$(detect_pkg_family)" in
rhel) install_rhel ;;
debian) install_debian ;;
*) say "docker: unsupported/undetected OS family; install Docker manually."; exit 1 ;;
esac
fi
# --- 2. enable + start the service (systemd only) ----------------------------
if [ -d /run/systemd/system ] && command -v systemctl >/dev/null 2>&1; then
sudo systemctl enable --now docker
else
say "docker: no systemd; not managing the service (start dockerd yourself)."
fi
# --- 3. group membership for the invoking user -------------------------------
user="$(id -un)"
if id -nG "$user" | tr ' ' '\n' | grep -qx docker; then
say "docker: $user already in the docker group."
else
sudo usermod -aG docker "$user"
say "docker: added $user to the docker group."
say " Log out and back in (or run 'newgrp docker') for it to take effect."
fi
+22 -1
View File
@@ -30,4 +30,25 @@ say "fetch.prune = true (stale remote-tracking refs cleaned on every fetch)"
git config --global credential.helper store
say "credential.helper = store (~/.git-credentials, created on first auth)"
say "Done. On the next HTTPS push/pull to a new host, git prompts once and caches the credential in ~/.git-credentials."
# Our own Gitea authenticates from $GITEA_KEY (exported by secrets/ from the
# vault), not from ~/.git-credentials.
#
# WHY: git never reads GITEA_KEY by itself. With only `store`, the token was
# whatever got typed at the first prompt, copied into ~/.git-credentials, and
# it silently went stale when the vault's key was rotated — or, on a box that
# never pushed, there was no credential at all and the push just failed. Now
# `opc GITEA_KEY` is the only step when the token changes.
#
# The empty helper first clears the inherited `store` for this host, so a
# rejected token is never cached and a stale cached one can never win. Gitea
# accepts the token as the password; the username is informational (GITEA_USER
# overrides it). With GITEA_KEY unset the helper prints nothing and git falls
# back to prompting, as before.
GITEA_URL="https://git.nic-oconnor.com"
git config --global --unset-all "credential.$GITEA_URL.helper" 2>/dev/null || true
git config --global --add "credential.$GITEA_URL.helper" ''
git config --global --add "credential.$GITEA_URL.helper" \
'!f() { test "$1" = get && test -n "${GITEA_KEY:-}" && printf "username=%s\npassword=%s\n" "${GITEA_USER:-$(id -un)}" "$GITEA_KEY"; }; f'
say "credential helper for $GITEA_URL = \$GITEA_KEY (from secrets.env)"
say "Done. $GITEA_URL uses \$GITEA_KEY; other HTTPS hosts prompt once and cache in ~/.git-credentials."
+9 -1
View File
@@ -26,6 +26,12 @@ REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
say() { printf '%s\n' "$*" >&2; }
# The installer links the CLI into ~/.local/bin, which is only on PATH in
# shells that sourced the managed ~/.zshrc. bootstrap run by a non-shell
# parent (the Coder agent's dotfiles script) has no such PATH, so the
# post-install `command -v hermes` failed on a successful install.
PATH="$HOME/.local/bin:$PATH"
# Hermes Agent version (synced with PyPI/npm).
HERMES_VERSION="v0.18.2"
@@ -74,7 +80,9 @@ if [ "$KEY_CHANGED" -eq 1 ] && [ -f "$HOME/.config/mcp-sync/key" ]; then
fi
say "hermes: Updated LITELLM_KEY in ~/.hermes/.env"
else
# Create .env if it doesn't exist
# Create .env if it doesn't exist. On a fresh home this runs BEFORE the
# installer below has created ~/.hermes, so make the directory first.
mkdir -p "$HOME/.hermes"
printf 'LITELLM_KEY=%s\n' "$NEW_KEY" >"$HOME/.hermes/.env"
say "hermes: Created ~/.hermes/.env with LITELLM_KEY"
fi
+62 -2
View File
@@ -8,8 +8,8 @@
# reset.
#
# Source this (callers already run `set -eu`), then use fetch_arch /
# fetch_arch_deb, have_version + mark_version, install_tarball_bin, and
# install_raw_bin.
# fetch_arch_deb, have_version + mark_version, install_tarball_bin,
# install_tarball_tree, and install_raw_bin.
BIN_DIR="${BIN_DIR:-$HOME/.local/bin}"
VERSION_DIR="$BIN_DIR/.versions"
@@ -55,6 +55,38 @@ have_version() {
[ "$(cat "$VERSION_DIR/$1")" = "$2" ]
}
# shadowed <name> — true (printing the winner's path) when another <name> sits
# EARLIER on PATH than $BIN_DIR, so a copy installed there would never run.
#
# WHY: on a desktop that already manages these tools (Omarchy: pacman for VS
# Code and jq, mise for kubectl/helm/node/...), ~/.local/bin is at the END of
# PATH. Installing there anyway left a second, older copy of every tool that
# nothing ran but `type -a` and the version stamps still reported. On the VMs
# this repo targets, zshrc puts ~/.local/bin FIRST, so nothing shadows it and
# this is always false -- installs there behave exactly as before. If $BIN_DIR
# is not on PATH at all (a fresh box, before zshrc), it is false too.
#
# A mise shim with no version configured is not a real provider (it errors when
# run), so shims only count when `mise which` resolves them.
shadowed() {
_sh_name="$1"
_sh_ifs="$IFS"; IFS=:
# shellcheck disable=SC2086 # split PATH on ':' deliberately
set -- $PATH
IFS="$_sh_ifs"
case ":$PATH:" in *":$BIN_DIR:"*) ;; *) return 1 ;; esac
for _sh_dir; do
[ "$_sh_dir" = "$BIN_DIR" ] && return 1
[ -n "$_sh_dir" ] && [ -x "$_sh_dir/$_sh_name" ] && [ ! -d "$_sh_dir/$_sh_name" ] || continue
case "$_sh_dir" in
*/mise/shims) mise which "$_sh_name" >/dev/null 2>&1 || continue ;;
esac
printf '%s\n' "$_sh_dir/$_sh_name"
return 0
done
return 1
}
# mark_version <name> <version> — record the installed version.
mark_version() {
mkdir -p "$VERSION_DIR"
@@ -93,6 +125,34 @@ install_zip_bin() {
fsay "installed $BIN_DIR/$3"
}
# install_tarball_tree <url> <root-within-archive> <dest-dir> — download a
# .tar.gz that holds a whole application directory rather than one binary, and
# swap it into <dest-dir>. install_tarball_bin cannot be reused here: VS Code
# is an Electron bundle of thousands of files that only runs from an intact
# tree, not a single binary that can be picked out of the archive.
#
# The new tree is staged beside the target and moved into place only once the
# download and extraction have both succeeded, so an interrupted run leaves the
# previously installed version intact instead of a half-extracted directory.
install_tarball_tree() {
_dest="$3"
_tmp="$(mktemp -d)"
_fetch_dl "$1" "$_tmp/archive.tar.gz"
tar -xzf "$_tmp/archive.tar.gz" -C "$_tmp"
if [ ! -d "$_tmp/$2" ]; then
fsay "fetch: '$2' not found in archive"; rm -rf "$_tmp"; return 1
fi
mkdir -p "$(dirname "$_dest")"
rm -rf "$_dest.new" "$_dest.old"
mv "$_tmp/$2" "$_dest.new"
if [ -e "$_dest" ]; then
mv "$_dest" "$_dest.old"
fi
mv "$_dest.new" "$_dest"
rm -rf "$_dest.old" "$_tmp"
fsay "installed $_dest"
}
# install_raw_bin <url> <dest-name> — download a single raw binary.
install_raw_bin() {
mkdir -p "$BIN_DIR"
+13 -1
View File
@@ -120,11 +120,23 @@ say "Key stored at $KEYFILE (0600)."
# bootstrap run — and then models and skills silently vanish, because sync.sh
# never runs and OPENCODE_CONFIG is never exported. ~/.zshrc.local is the
# unmanaged file the managed zshrc sources last, so the hook persists.
case "${SHELL:-}" in
#
# Do NOT pick the file from $SHELL alone. On a fresh box bootstrap runs this
# from the user's ORIGINAL shell: zsh/ has just switched the login shell to zsh,
# but $SHELL in this process still says bash. The hook then went into ~/.bashrc,
# below zsh/'s auto-exec hook, so every new terminal exec'd into zsh before
# reaching it, zsh never sourced sync.sh, and opencode came up unconfigured.
# The dotfiles-managed ~/.zshrc (or zsh as the account's login shell) wins.
_login_shell="$(getent passwd "$(id -un)" 2>/dev/null | cut -d: -f7)"
if grep -qs 'Managed by dotfiles/zsh/install.sh' "$HOME/.zshrc"; then
RC="$HOME/.zshrc.local"
else
case "${_login_shell:-${SHELL:-}}" in
*zsh) RC="$HOME/.zshrc.local" ;;
*bash) RC="$HOME/.bashrc" ;;
*) [ -f "$HOME/.zshrc" ] && RC="$HOME/.zshrc.local" || RC="$HOME/.bashrc" ;;
esac
fi
touch "$RC"
# It sits alongside the user's secrets; keep it non-readable by others.
[ "$RC" = "$HOME/.zshrc.local" ] && chmod 0600 "$RC" 2>/dev/null || true
+7
View File
@@ -29,6 +29,13 @@ PREFIX="$HOME/.local/opt/node"
BIN_DIR="$HOME/.local/bin"
NODE_CLIS="node npm npx"
# Another node (mise, nvm, a distro package) already wins on PATH, so ours
# would never run -- see `shadowed` in lib/fetch.sh. npx comes with it.
if _winner="$(shadowed node)"; then
say "node: $_winner comes first on PATH; skipping (a copy in $BIN_DIR would never run)."
exit 0
fi
case "$(fetch_arch)" in
x86_64) NODE_ARCH=x64 ;;
aarch64) NODE_ARCH=arm64 ;;
+86 -17
View File
@@ -6,19 +6,19 @@
# field on EVERY new shell. That made each terminal wait on 1Password, broke
# every shell when the network or the service account was down, and meant each
# box carried its own hand-written copy of the loop. Now the vault is read once
# here (at bootstrap, or on demand with `envs`), and shells read a local file.
# here (at bootstrap, or on demand with `ops`), and shells read a local file.
#
# Layout: one vault per user (default: the login name), holding one item
# (default: also the login name) whose field LABELS are the variable names —
# GITEA_KEY, LITELLM_KEY, PROXMOX_TOKEN_SECRET, ... Every such field is
# exported. Add a secret by adding a field (`envc NAME`), not by editing
# exported. Add a secret by adding a field (`opc NAME`), not by editing
# this repo.
#
# Files, all under ~/.config/dotfiles (0700), none in the repo:
# op.env OP_SERVICE_ACCOUNT_TOKEN, OP_VAULT, OP_ITEM (0600). Kept so a
# refresh needs no prompt. Deliberately NOT sourced by shells:
# the token stays out of every process's environment, and only
# the env* helpers in zsh/zshrc load it, in a subshell.
# the op* helpers in zsh/zshrc load it, in a subshell.
# secrets.env the exported fields (0600), replaced atomically, so a failed
# fetch leaves the previous copy in place.
#
@@ -48,8 +48,17 @@ for _tool in op jq; do
done
# --- 1. resolve the service account, vault and item -------------------------
# Precedence: the environment, then op.env, then a prompt (token) / defaults.
# Precedence: the environment, then op.env, then a prompt / auto-detection.
#
# WHY each piece is checked on its own: the vault holds every key later
# modules need (LITELLM_KEY, GITEA_KEY, ...). A bad token paste used to fall
# through as a generic "could not read item", bootstrap carried on, and the
# first visible symptom was mcp-config/ asking for a LiteLLM key by hand —
# which hid the real problem. Now a rejected token, a wrong vault and a wrong
# item each get their own message, and on a TTY a re-prompt.
_env_token="${OP_SERVICE_ACCOUNT_TOKEN:-}"
_token_src="from OP_SERVICE_ACCOUNT_TOKEN"
[ -n "$_env_token" ] || _token_src="stored in $OP_ENV"
_env_vault="${OP_VAULT:-}"
_env_item="${OP_ITEM:-}"
if [ -f "$OP_ENV" ]; then
@@ -61,6 +70,10 @@ OP_VAULT="${_env_vault:-${OP_VAULT:-}}"
OP_ITEM="${_env_item:-${OP_ITEM:-}}"
_user="$(id -un)"
# Probe in a subshell: in POSIX sh a failed redirect on a special builtin
# like `:` exits the whole script.
have_tty() { ( : < /dev/tty ) 2>/dev/null; }
# ask <prompt> <default> — read one line from the TTY; echoes the default on empty.
ask() {
printf '%s [%s]: ' "$1" "$2" >&2
@@ -68,31 +81,85 @@ ask() {
printf '%s' "${_ans:-$2}"
}
if [ -z "$OP_SERVICE_ACCOUNT_TOKEN" ]; then
if [ -f "$SECRETS_ENV" ]; then
# mask <secret> — first 10 and last 10 characters plus the length, so a
# truncated or doubled paste is visible without printing the whole token.
mask() {
printf '%s' "$1" | awk '{ n = length($0)
if (n <= 20) printf "%s (%d chars)", "(too short)", n
else printf "%s**********%s (first 10 + last 10 of %d chars)", substr($0, 1, 10), substr($0, n - 9), n }'
}
if [ -z "$OP_SERVICE_ACCOUNT_TOKEN" ] && [ -f "$SECRETS_ENV" ]; then
say "secrets: no service-account token; keeping existing $SECRETS_ENV."
exit 0
fi
# Probe in a subshell: in POSIX sh a failed redirect on a special builtin
# like `:` exits the whole script, which would skip the message below.
if ! ( : < /dev/tty ) 2>/dev/null; then
say "secrets: no OP_SERVICE_ACCOUNT_TOKEN and no TTY to prompt."
# The token: `op whoami` is the cheapest call that fails on a bad token and
# nothing else. Up to three pastes before giving up.
_tries=0
while :; do
if [ -n "$OP_SERVICE_ACCOUNT_TOKEN" ]; then
export OP_SERVICE_ACCOUNT_TOKEN
op whoami >/dev/null 2>&1 && break
if [ "$_tries" -eq 0 ]; then
say "secrets: 1Password REJECTED the service-account token $_token_src (bad paste, revoked or expired?)."
say " token: $(mask "$OP_SERVICE_ACCOUNT_TOKEN")"
else
say "secrets: 1Password REJECTED that token — most likely a truncated or bad paste."
say " token: $(mask "$OP_SERVICE_ACCOUNT_TOKEN")"
fi
fi
if ! have_tty; then
say "secrets: no valid OP_SERVICE_ACCOUNT_TOKEN and no TTY to prompt."
say " re-run as: OP_SERVICE_ACCOUNT_TOKEN=ops_... $0"
exit 1
fi
if [ "$_tries" -ge 3 ]; then
say "secrets: giving up after $_tries attempts. Every key in the vault (LITELLM_KEY,"
say " GITEA_KEY, ...) is missing until this works: re-run $0"
exit 1
fi
_tries=$((_tries + 1))
printf '1Password service-account token (ops_..., input hidden): ' >&2
stty -echo < /dev/tty 2>/dev/null || true
IFS= read -r OP_SERVICE_ACCOUNT_TOKEN < /dev/tty || true
stty echo < /dev/tty 2>/dev/null || true
printf '\n' >&2
OP_SERVICE_ACCOUNT_TOKEN="$(printf '%s' "$OP_SERVICE_ACCOUNT_TOKEN" | tr -d ' \t\r\n')"
[ -n "$OP_SERVICE_ACCOUNT_TOKEN" ] || { say "secrets: no token entered."; exit 1; }
[ -n "$OP_VAULT" ] || OP_VAULT="$(ask 'Vault' "$_user")"
[ -n "$OP_ITEM" ] || OP_ITEM="$(ask 'Item' "$_user")"
[ -n "$OP_SERVICE_ACCOUNT_TOKEN" ] || say "secrets: no token entered."
done
say "secrets: service-account token OK: $(mask "$OP_SERVICE_ACCOUNT_TOKEN")"
# pick <kind> <current> <default> <check-fn> <list-cmd...>
# Resolve a vault or item name: keep <current> if it checks out; otherwise use
# the only one the token can see, or ask (TTY) showing what it can see.
pick() {
_kind="$1" _cur="$2" _def="$3"; shift 3
_check="$1"; shift
_names="$("$@" --format json 2>/dev/null | jq -r '.[] | .name // .title')"
if [ -n "$_cur" ]; then
$_check "$_cur" && { printf '%s' "$_cur"; return 0; }
say "secrets: $_kind '$_cur' not found. This token can see:"
printf '%s\n' "$_names" | sed 's/^/ - /' >&2
fi
OP_VAULT="${OP_VAULT:-$_user}"
OP_ITEM="${OP_ITEM:-$_user}"
export OP_SERVICE_ACCOUNT_TOKEN
if [ "$(printf '%s\n' "$_names" | grep -c .)" -eq 1 ]; then
say "secrets: using the only $_kind this token can see: $_names"
printf '%s' "$_names"; return 0
fi
have_tty || { say "secrets: set OP_$(printf '%s' "$_kind" | tr a-z A-Z) and re-run."; return 1; }
printf '%s\n' "$_names" | grep -Fxq -- "$_def" || _def="$(printf '%s\n' "$_names" | head -n 1)"
while :; do
_cur="$(ask "$_kind" "$_def")"
$_check "$_cur" && { printf '%s' "$_cur"; return 0; }
say "secrets: $_kind '$_cur' not found. This token can see:"
printf '%s\n' "$_names" | sed 's/^/ - /' >&2
done
}
vault_ok() { op vault get "$1" >/dev/null 2>&1; }
item_ok() { op item get "$1" --vault "$OP_VAULT" >/dev/null 2>&1; }
OP_VAULT="$(pick vault "$OP_VAULT" "$_user" vault_ok op vault list)" || exit 1
OP_ITEM="$(pick item "$OP_ITEM" "$_user" item_ok op item list --vault "$OP_VAULT")" || exit 1
# --- 2. fetch the item once and render it -----------------------------------
# One `op item get` instead of one `op read` per field. Labels that are not
@@ -115,7 +182,7 @@ _tmp="$(mktemp "$CONF_DIR/secrets.env.XXXXXX")"
trap 'rm -f "$_tmp"' EXIT
{
printf '# Generated by dotfiles secrets/install.sh from op://%s/%s — do not edit.\n' "$OP_VAULT" "$OP_ITEM"
printf '# Refresh with: envs (change the vault with: envc NAME / envd NAME)\n'
printf '# Refresh with: ops (change the vault with: opc NAME / opd NAME)\n'
printf '%s' "$_json" | jq -r '
.fields[]
| select(.purpose != "NOTES")
@@ -132,6 +199,8 @@ fi
mv -f "$_tmp" "$SECRETS_ENV"
trap - EXIT
say "secrets: wrote $_count variables to $SECRETS_ENV (0600)."
grep -q '^export LITELLM_KEY=' "$SECRETS_ENV" \
|| say "secrets: WARNING: op://$OP_VAULT/$OP_ITEM has no LITELLM_KEY field; mcp-config/ and hermes/ need it."
# Remember the account only after it has actually worked.
{
+236
View File
@@ -0,0 +1,236 @@
#!/usr/bin/env sh
# vscode/install.sh — install Visual Studio Code into ~/.local/opt/vscode.
#
# WHY: VS Code is the one GUI app this repo installs, and the obvious way to get
# it — Microsoft's rpm/deb repo — needs root, writes outside $HOME, and is wiped
# by a Kasm redeploy, so the editor quietly disappears on every rebuild. The box
# this was written for had exactly that half-state: /etc/yum.repos.d/vscode.repo
# present, `code` never installed, and nothing in this repo recording that VS
# Code was meant to be there at all. Upstream's portable tarball is the same
# build with none of that: everything lands under $HOME, no sudo, and it
# survives a redeploy. The only thing given up is dnf/apt-driven updates, which
# is what the pinned VSCODE_VERSION below replaces.
#
# It is not in bin/install.sh because that script installs single self-contained
# binaries; VS Code is an Electron tree of thousands of files plus a menu entry
# and an icon. It follows node/ and ansible/ instead: unpack under ~/.local/opt
# and symlink the CLI into ~/.local/bin.
#
# The Chromium sandbox still works from a userland install: the bundled
# chrome-sandbox helper is not setuid here, so Electron falls back to the
# unprivileged user-namespace sandbox. That needs user namespaces enabled
# (user.max_user_namespaces > 0, the default on every distro this repo targets).
# Verified by launching without --no-sandbox. Nothing is given up for dropping
# root; if a host ever has userns disabled, VS Code says so on startup.
#
# Idempotent and safe to re-run. To upgrade, bump VSCODE_VERSION.
set -eu
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
REPO_ROOT="$(dirname -- "$SCRIPT_DIR")"
# shellcheck source=../lib/fetch.sh
. "$REPO_ROOT/lib/fetch.sh"
say() { printf '%s\n' "$*" >&2; }
# Pinned deliberately. VS Code auto-updates itself on other platforms; a tarball
# install does not, which is the point — the editor runs your code and holds
# your Git and extension credentials, so the version is a supply-chain decision
# rather than whatever upstream served today. `code --version` prints this
# string verbatim, so it must match with no leading "v".
VSCODE_VERSION="1.139.0"
PREFIX="$HOME/.local/opt/vscode"
DESKTOP_DIR="$HOME/.local/share/applications"
ARGV_JSON="$HOME/.vscode/argv.json"
# has_desktop — true when this host has a graphical session to run a GUI editor
# in. VS Code is ~400MB unpacked and $HOME is the thing that persists across
# redeploys, so it is not worth carrying on a box that will never display it.
# Desktop images ship session files under /usr/share/{x,wayland-}sessions.
has_desktop() {
[ -n "${XDG_CURRENT_DESKTOP:-}" ] && return 0
for d in /usr/share/xsessions /usr/share/wayland-sessions; do
[ -d "$d" ] && [ -n "$(ls -A "$d" 2>/dev/null)" ] && return 0
done
return 1
}
# --- skip on headless hosts --------------------------------------------------
# DOTFILES_VSCODE=1 forces the install anyway (e.g. for `code tunnel` on a
# server); DOTFILES_VSCODE=0 skips it even on a desktop.
case "${DOTFILES_VSCODE:-auto}" in
0) say "vscode: DOTFILES_VSCODE=0; skipping."; exit 0 ;;
1) ;;
*) if ! has_desktop; then
say "vscode: no graphical session detected; skipping (DOTFILES_VSCODE=1 to force)."
exit 0
fi ;;
esac
# --- defer to a system-wide VS Code -------------------------------------------
# WHY: a distro package (Arch's visual-studio-code-bin on Omarchy, Microsoft's
# rpm/deb elsewhere) and this tarball both add a "Visual Studio Code" menu entry
# and both open the same ~/.config/Code profile. Two builds on one profile is
# the real hazard: the older one opens state the newer one already migrated,
# and extensions break in ways that look like extension bugs. The system copy
# also gets updates this pinned one does not, so it wins.
#
# PATH order is irrelevant here (unlike `shadowed` in bin/): the menu entries
# collide wherever the binaries sit. Anything we installed earlier is removed --
# only ours, recognised by the .desktop Exec pointing into $PREFIX.
_sys_code=""
_sh_ifs="$IFS"; IFS=:
for _d in $PATH; do
[ "$_d" != "$BIN_DIR" ] && [ -x "$_d/code" ] && { _sys_code="$_d/code"; break; }
done
IFS="$_sh_ifs"
if [ -n "$_sys_code" ]; then
say "vscode: system VS Code at $_sys_code; skipping the tarball install."
if grep -qs "^Exec=$PREFIX/" "$DESKTOP_DIR/code.desktop"; then
rm -f "$DESKTOP_DIR/code.desktop"
[ "$(readlink "$BIN_DIR/code" 2>/dev/null)" = "$PREFIX/bin/code" ] && rm -f "$BIN_DIR/code"
rm -f "$VERSION_DIR/code"
rm -rf "$PREFIX"
command -v update-desktop-database >/dev/null 2>&1 &&
update-desktop-database "$DESKTOP_DIR" 2>/dev/null || true
say "vscode: removed the earlier tarball copy ($PREFIX, its menu entry and $BIN_DIR/code)."
fi
exit 0
fi
# Upstream keys its download URLs on x64/arm64 rather than amd64/arm64.
case "$(fetch_arch)" in
x86_64) VSCODE_ARCH=x64 ;;
aarch64) VSCODE_ARCH=arm64 ;;
*) say "vscode: unsupported arch $(fetch_arch)"; exit 1 ;;
esac
# --- install the app tree ----------------------------------------------------
# Verify it RUNS, not just that the tree exists: a Kasm redeploy can leave a
# present-but-broken directory behind (the same failure node/ and ansible/
# guard against).
#
# The root fallback is not optional. Run as root, `code --version` refuses --
# it prints "You are trying to start Visual Studio Code as a super user"
# instead of the version, and still exits 0 -- so a bare run-check never
# matches and would re-download 325MB on every single bootstrap, silently.
# That is the forced-install path (DOTFILES_VSCODE=1 in a container or on a
# server), so for root only, fall back to the recorded version stamp.
installed_version="$("$PREFIX/bin/code" --version 2>/dev/null | head -1 || true)"
if [ "$installed_version" = "$VSCODE_VERSION" ]; then
say "vscode: $VSCODE_VERSION already present at $PREFIX."
elif [ "$(id -u)" = "0" ] && have_version code "$VSCODE_VERSION"; then
say "vscode: $VSCODE_VERSION already present at $PREFIX (stamped; root cannot query --version)."
else
say "vscode: installing $VSCODE_VERSION into $PREFIX (this is a ~325MB download)..."
install_tarball_tree \
"https://update.code.visualstudio.com/$VSCODE_VERSION/linux-$VSCODE_ARCH/stable" \
"VSCode-linux-$VSCODE_ARCH" "$PREFIX"
# bin/code is upstream's CLI wrapper (`code .`, `code --install-extension`);
# the Electron binary at the tree root is what the menu entry launches.
mkdir -p "$BIN_DIR"
ln -sf "$PREFIX/bin/code" "$BIN_DIR/code"
mark_version code "$VSCODE_VERSION"
say "vscode: installed $VSCODE_VERSION and linked $BIN_DIR/code."
fi
# --- menu entry --------------------------------------------------------------
# Icon= is an absolute path rather than a theme name: the bundled icon is a
# single 1024x1024 PNG and hicolor tops out at 512x512, so there is no correct
# themed directory to drop it in. The spec allows an absolute path, and it needs
# no icon-cache rebuild to take effect.
mkdir -p "$DESKTOP_DIR"
cat >"$DESKTOP_DIR/code.desktop" <<DESKTOP
[Desktop Entry]
Type=Application
Name=Visual Studio Code
GenericName=Text Editor
Comment=Code Editing. Redefined.
Exec=$PREFIX/code %F
Icon=$PREFIX/resources/app/resources/linux/code.png
Terminal=false
StartupNotify=true
StartupWMClass=Code
Categories=TextEditor;Development;IDE;
MimeType=text/plain;inode/directory;
Keywords=vscode;editor;
Actions=new-window;
[Desktop Action new-window]
Name=New Empty Window
Exec=$PREFIX/code --new-window %F
Icon=$PREFIX/resources/app/resources/linux/code.png
DESKTOP
say "vscode: wrote $DESKTOP_DIR/code.desktop"
# Refresh the per-user menu caches so the entry appears without a re-login.
if command -v update-desktop-database >/dev/null 2>&1; then
update-desktop-database "$DESKTOP_DIR" 2>/dev/null || true
fi
if command -v kbuildsycoca5 >/dev/null 2>&1; then
kbuildsycoca5 --noincremental >/dev/null 2>&1 || true
fi
# --- keyring prompt suppression ----------------------------------------------
# WHY: on first launch Electron's safeStorage reaches for the OS keyring through
# libsecret. On a KDE session that wakes kwalletd, which pops an unlock/create
# dialog before the editor is usable -- and since kwallet-pam is not installed
# on these hosts, nothing unlocks the wallet at login, so it prompts every
# session. "basic" keeps VS Code's secrets in its own store and never reaches
# for the wallet at all. Verified with dbus-monitor: no org.kde.KWallet traffic
# on launch.
#
# TRADE-OFF: basic is weaker than a real keyring. Secrets VS Code stores (Git
# and GitHub auth tokens, extension credentials) end up obfuscated on disk
# rather than sealed behind a wallet password. To move to the wallet later,
# install kwallet-pam so it unlocks at login, then change the value to
# "kwallet5" and restart VS Code.
#
# The path comes from product.json's dataFolderName (".vscode"), NOT the
# user-data-dir: argv.json lives at ~/.vscode/argv.json and a copy written
# anywhere else is read by nothing and silently ignored.
if [ ! -e "$ARGV_JSON" ]; then
mkdir -p "$(dirname "$ARGV_JSON")"
cat >"$ARGV_JSON" <<'ARGV'
// Runtime arguments for VS Code. See dotfiles vscode/install.sh.
{
// Keep secrets in VS Code's own store rather than the OS keyring, so KDE's
// kwallet does not prompt for an unlock on every launch.
"password-store": "basic"
}
ARGV
say "vscode: wrote $ARGV_JSON (password-store=basic)"
elif grep -q '"password-store"' "$ARGV_JSON"; then
say "vscode: $ARGV_JSON already sets password-store; leaving it alone."
else
# Merge rather than overwrite: this file is JSON-with-comments that VS Code
# and the user both own, and it carries a crash-reporter-id that should
# survive. Insert the key after the opening brace and keep the rest byte for
# byte.
cp -p "$ARGV_JSON" "$ARGV_JSON.bak"
if awk '
BEGIN { done = 0 }
{
print
if (!done && $0 ~ /^[[:space:]]*\{[[:space:]]*$/) {
print "\t// Added by dotfiles vscode/install.sh: keep secrets in VS Code'"'"'s"
print "\t// own store so kwallet does not prompt for an unlock on every launch."
print "\t\"password-store\": \"basic\","
done = 1
}
}
END { exit(done ? 0 : 1) }
' "$ARGV_JSON.bak" >"$ARGV_JSON.new"; then
mv "$ARGV_JSON.new" "$ARGV_JSON"
say "vscode: set password-store=basic in $ARGV_JSON (backup: $ARGV_JSON.bak)"
else
rm -f "$ARGV_JSON.new"
say "vscode: WARNING: could not find the object opening in $ARGV_JSON."
say " Add '\"password-store\": \"basic\"' by hand to stop the keyring prompt."
fi
fi
say "vscode: ready."
+58 -39
View File
@@ -2,19 +2,6 @@
# overwritten on the next bootstrap run. Edit zsh/zshrc in the dotfiles repo
# instead.
# If this account was added to the docker group (dotfiles/docker/install.sh)
# but this shell's session predates that, re-exec via newgrp so docker works
# without a full logout. Guarded: fires only when the group is on the account
# yet missing from the running shell's groups, so the newgrp'd shell (which
# has the group) cannot loop.
# Only for real terminals ([ -t 0 ]): newgrp replaces the shell and would
# swallow the command in scripted `zsh -i -c ...` invocations.
if [ -t 0 ] && [ -S /var/run/docker.sock ] \
&& id -nG "${USER:-$(id -un)}" 2>/dev/null | grep -qw docker \
&& ! id -Gn 2>/dev/null | grep -qw docker; then
exec newgrp docker
fi
# Userland tools installed by dotfiles/bin/install.sh live here.
case ":$PATH:" in
*":$HOME/.local/bin:"*) ;;
@@ -34,6 +21,9 @@ SAVEHIST=10000
setopt SHARE_HISTORY
setopt HIST_IGNORE_DUPS
# grmb and the op* helpers below have a bash port in bash/dotfiles.bash for
# hosts that keep bash; change both together.
# Delete remote branches without fighting tab-completion.
#
# `git push origin --delete <TAB>` completes to `origin/foo`, but `origin/foo`
@@ -76,57 +66,86 @@ grmb() {
# Secrets from this user's 1Password vault, written by dotfiles/secrets/install.sh.
# Reading a local 0600 file keeps shell startup off the network: nothing here
# calls op. envc/envd refresh the file themselves; after editing the vault
# anywhere else (the 1Password app, another box), run `envs`.
# calls op. opc/opd refresh the file themselves; after editing the vault
# anywhere else (the 1Password app, another box), run `ops`.
_DOTFILES_CONF="$HOME/.config/dotfiles"
[ -f "$_DOTFILES_CONF/secrets.env" ] && source "$_DOTFILES_CONF/secrets.env"
# envr NAME read one field straight from the vault
# envc NAME [VALUE] create or update a field (prompts, hidden, when VALUE is
# omitted, which keeps the secret out of shell history),
# then refresh and reload this shell
# envd NAME delete a field, refresh, and unset it in this shell
# envl [VAULT [ITEM]] show every field and its current value (default: your
# opr NAME read one field straight from the vault
# opc NAME=VALUE create or update a field, then refresh and reload this
# opc NAME shell; with no VALUE (NAME or NAME=) it prompts, hidden,
# which keeps the secret out of shell history.
# `opc NAME VALUE` also still works.
# opd NAME delete a field, refresh, and unset it in this shell
# opl [VAULT [ITEM]] show every field and its current value (default: your
# vault and item from op.env)
# envs re-fetch the vault into secrets.env and reload this shell
# ops re-fetch the vault into secrets.env and reload this shell
#
# The service-account token lives in op.env and is loaded only inside these
# subshells, so it never lands in the environment of everything you run.
_dotfiles_op() {
[ -f "$_DOTFILES_CONF/op.env" ] || { print -u2 "no $_DOTFILES_CONF/op.env; run: envs"; return 1; }
[ -f "$_DOTFILES_CONF/op.env" ] || { print -u2 "no $_DOTFILES_CONF/op.env; run: ops"; return 1; }
( set -a; source "$_DOTFILES_CONF/op.env"; set +a; "$@" )
}
_dotfiles_env_name() {
[[ "$1" =~ '^[A-Za-z_][A-Za-z0-9_]*$' ]] && return 0
print -u2 "$2: '$1' is not a valid variable name"; return 2
}
envr() {
_dotfiles_env_name "$1" envr || return
_dotfiles_op sh -c 'op read "op://$OP_VAULT/$OP_ITEM/$1"' envr "$1"
opr() {
_dotfiles_env_name "$1" opr || return
_dotfiles_op sh -c 'op read "op://$OP_VAULT/$OP_ITEM/$1"' opr "$1"
}
envc() {
opc() {
emulate -L zsh
local name=$1 value
_dotfiles_env_name "$name" envc || return
if (( $# >= 2 )); then
if [[ $1 == *=* ]]; then
# NAME=VALUE: split on the first '=' so values may contain '='
name=${1%%=*} value=${1#*=}
elif (( $# >= 2 )); then
value=$2
else
fi
_dotfiles_env_name "$name" opc || return
if [[ -z $value ]]; then
read -rs "value?$name: " && print
[[ -n $value ]] || { print -u2 "opc: empty value for $name; nothing changed"; return 1; }
fi
# [concealed] makes it a password field; an existing field of that name is updated.
_dotfiles_op sh -c 'op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[concealed]=$2" >/dev/null' \
envc "$name" "$value" && envs
# op can exit non-zero with "(404) Not Found" after an edit it did save (seen with
# op 2.34.1 and 2.39.0), so its exit code is ignored: the field is read back instead.
_dotfiles_op sh -c '
clean() { printf "%s" "${1:-(no output)}" | sed "s/^\[ERROR\] [0-9/]* [0-9:]* //"; }
ref="op://$OP_VAULT/$OP_ITEM/$1"
err=$(op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[concealed]=$2" 2>&1 >/dev/null)
if [ "$(op read "$ref" 2>/dev/null)" = "$2" ]; then
printf "opc: saved %s\n" "$ref" >&2
else
printf "opc: could not save %s\n op said: %s\n check vault, item and service-account write access with: opl\n" \
"$ref" "$(clean "$err")" >&2
exit 1
fi' opc "$name" "$value" && ops
}
envd() {
_dotfiles_env_name "$1" envd || return
_dotfiles_op sh -c 'op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[delete]" >/dev/null' envd "$1" \
&& unset "$1" && envs
opd() {
_dotfiles_env_name "$1" opd || return
# As in opc, op's exit code is unreliable here: check the item's field labels instead.
_dotfiles_op sh -c '
clean() { printf "%s" "${1:-(no output)}" | sed "s/^\[ERROR\] [0-9/]* [0-9:]* //"; }
ref="op://$OP_VAULT/$OP_ITEM/$1"
err=$(op item edit "$OP_ITEM" --vault "$OP_VAULT" "$1[delete]" 2>&1 >/dev/null)
if ! item=$(op item get "$OP_ITEM" --vault "$OP_VAULT" --format json 2>&1); then
printf "opd: could not re-read %s to confirm the delete\n op said: %s\n" "$ref" "$(clean "$item")" >&2
exit 1
elif printf "%s" "$item" | jq -r ".fields[].label" | grep -qxF "$1"; then
printf "opd: could not delete %s\n op said: %s\n" "$ref" "$(clean "$err")" >&2
exit 1
fi
printf "opd: deleted %s\n" "$ref" >&2' opd "$1" \
&& unset "$1" && ops
}
envl() {
opl() {
_dotfiles_op sh -c '
vault=${1:-$OP_VAULT} item=${2:-$OP_ITEM}
printf "op://%s/%s\n" "$vault" "$item" >&2
op item get "$item" --vault "$vault" --reveal --format json' envl "$@" \
op item get "$item" --vault "$vault" --reveal --format json' opl "$@" \
| jq -r '
[.fields[] | select(.purpose != "NOTES")] as $f
| ([$f[] | .label // "" | length] | max // 0) as $w
@@ -136,7 +155,7 @@ envl() {
| "\($name) \($lines[0] // "")",
($lines[1:][] | (" " * ($w + 2)) + .)'
}
envs() {
ops() {
"${DOTFILES_DIR:-$HOME/.dotfiles}/secrets/install.sh" \
&& source "$_DOTFILES_CONF/secrets.env"
}
+1 -1
View File
@@ -36,7 +36,7 @@
# Nothing to set here. Do NOT loop over `op read` in this file: that ran on
# every shell. secrets/install.sh stores the service-account token in
# ~/.config/dotfiles/op.env and writes the vault's fields to secrets.env, which
# the managed ~/.zshrc loads. Use envr / envc / envd (see zsh/zshrc).
# the managed ~/.zshrc loads. Use opr / opc / opd (see zsh/zshrc).
# --- Proxmox -------------------------------------------------------------
# One pair of credentials, several consumers: the proxmox MCP server