Files
dotfiles/claude-obsidian/README.md
T
noconnorandClaude Opus 5 376ec03428 claude-obsidian: unattended overnight runs -- lock, URL fallback, config fix
Needed to leave a full backlog drain running unattended:

- Run lock (flock): enabling the timer fired an immediate catch-up run
  (Persistent=true) while a manual run was about to start; without a
  lock the two would race on state.json and usage.jsonl.
- OBSIDIAN_API_URL accepts a comma-separated list, first reachable wins,
  so the permanent address and an interim port-forward share one config.
- Fix: inline "# comments" in config.env were parsed into values, so
  BUDGET_USD_PER_RUN=0.50  # ... crashed float(). The shipped
  config.env.example had exactly that shape on several lines -- anyone
  copying it would have hit the crash. A '#' inside a value (URL fragment)
  still survives. Tested against both the example and a real config.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KF322NzozeaFjpgWvh4F4g
2026-09-17 16:14:45 -04:00

6.7 KiB
Raw Blame History

claude-obsidian — Claude conversations → Obsidian notes

Summarizes every Claude conversation into one searchable Obsidian note: what happened, what was decided and why, the facts and gotchas worth remembering, and what was left open.

  • Claude Code sessions sync automatically and incrementally (hourly timer).
  • claude.ai chats import from a data export (claude.ai has no API, so each export is manual).

Notes are written through the Obsidian Local REST API plugin, so the vault can live anywhere the plugin is reachable. The same plugin exposes a built-in MCP server (/mcp/), which is how Claude reads the notes back — no separate MCP process.

~/.claude/projects/*/*.jsonl ─┐                                          ┌─ Claude/Code/2026/…md
claude.ai export .zip ────────┼─ condense → redact → summarize → PUT ────┼─ Claude/Web/2026/…md
                              └─ (headless claude -p, no transcript)     └─ Claude/Index.md

Install

  1. In Obsidian, install Local REST API, enable its HTTP server, and make it reachable from this machine. Put the API key in ~/.config/claude-obsidian/api-key (chmod 600).

  2. ./install.sh — links the binary into ~/.local/bin, enables the hourly user timer, and creates ~/.config/claude-obsidian/config.env from config.env.example if absent.

  3. Edit config.env (at minimum OBSIDIAN_API_URL), then preview one note:

    claude-obsidian-sync stats                       # sizes, no network
    claude-obsidian-sync sync --dry-run --limit 1    # prints the note, writes nothing
    

Use

claude-obsidian-sync sync                          # what the timer runs
claude-obsidian-sync sync --session <session-id>   # one session, now, ignoring the settle window
claude-obsidian-sync import-claude-ai ~/Downloads/data-2026-09-13.zip
claude-obsidian-sync sync --prefetch --limit 3     # summarize into the local cache, vault untouched
claude-obsidian-sync usage                         # calls, billed tokens, spend: last 5h / 24h / 7d

Claude reads the vault through the plugin's MCP server:

claude mcp add --scope user --transport http obsidian http://<host>:27123/mcp/ \
  --header "Authorization: Bearer $(cat ~/.config/claude-obsidian/api-key)"

How it behaves — and why

  • Summaries run on your own Claude plan via headless claude -p (LLM_BACKEND=claude-cli). No API key to manage. An OpenAI-compatible backend (LLM_BACKEND=openai) exists for local models. Note a LiteLLM gateway that fronts Claude by passing through the caller's subscription has no key of its own, so a background job cannot use its claude-* models.
  • --no-session-persistence is load-bearing. Without it every summary writes a new transcript into ~/.claude/projects, and the next run summarizes its own summarizing.
  • Usage is metered and capped. Every model call is appended to ~/.local/state/claude-obsidian/usage.jsonl with billed tokens (the CLI's modelUsage, not the visible turn) and total_cost_usd — API list-price equivalent, a faithful proxy for how much was consumed; the plan's own meter is only visible in Claude Code's /usage. Before each call its cost is estimated (+20% margin) and the call is refused if it would cross BUDGET_USD_PER_RUN or the rolling BUDGET_USD_PER_DAY — so budgets are never overshot. Not claude -p --max-budget-usd: that aborts after the tokens are spent and discards the result (measured: $0.26 for nothing). Every run ends with a one-line usage report.
  • Three cost levers, measured on one session — $0.057 → $0.010, same summary content: alwaysThinkingEnabled=false (extended thinking was ~85% of output tokens), DISABLE_PROMPT_CACHING=1 (the CLI writes each prompt to the 1-hour cache at 2× input price, and a one-shot call never reads it back), and CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 (otherwise a hidden side request re-reads the entire prompt, doubling billed input). A whole ~110-session backlog is about $2 at list price with all three.
  • Summaries are cached until their note is written, so a vault outage never pays for the same summary twice, and --prefetch can do the summarizing while the vault is unreachable.
  • It shares your plan's usage limits. A first backfill of ~100 sessions in parallel hit a session limit within minutes — which also limits interactive use. So: one job at a time, MAX_PER_RUN sessions per run (newest first), and the run stops at the first usage-limit error (exit 75, which the systemd unit treats as success) instead of failing the rest of the queue. The next hourly run resumes. A 100-session backlog drains in about ten hours.
  • Secrets are redacted before anything leaves the machine, and again in the note: private keys, bearer tokens, password=-style assignments, JWTs, provider tokens, ≥48-char hex, and random mixed-case strings. It over-redacts on purpose; git SHAs, md5s and paths survive. The summarizer is also told never to reproduce secrets.
  • One run at a time (a flock on run.lock): an hourly timer firing during a long manual run exits immediately instead of racing it on the state and ledger files.
  • Several vault URLs can be listed; the first that answers wins, so one config covers both a permanent address and a temporary kubectl port-forward.
  • Incremental and idempotent. State (~/.local/state/claude-obsidian/state.json) records each session's size+mtime. A resumed session is re-summarized into the same note path. A session is left alone until idle for SETTLE_MINUTES, so a live conversation isn't summarized mid-flight. Empty sessions are recorded and skipped.
  • Condensed, not truncated. Tool output is dropped and each tool call becomes one line; prose is kept. Typical compression is ~60× (271 MB of transcripts → 4.4 MB). Sessions over 300 KB condensed are summarized in parts, then combined.
  • Subagent transcripts (<session>/subagents/**) are part of their parent session and are not synced separately.

Note format

type: claude-conversation
source: claude-code            # or claude-ai
session_id: 3f9c2a1e-…
title: Fix flaky CI cache after runner upgrade
project: my-service            # basename of the working directory
branch: …
started / ended: …
prompts: 1                     # user prompts, not tool-loop records
summarized_by: claude-haiku-4-5-20251001
tags: [claude, claude-code, ci, caching, …]

followed by a summary callout and Outcomes / Decisions / Facts & gotchas / Open threads, and a claude --resume <id> line. Claude/Index.md lists every note, newest first.