diff --git a/.gitignore b/.gitignore index 0bca839..37b3f73 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,9 @@ sessions/ journal/ knowledge/ .env +.api-key +.api-key.* +.oauth-optin-warned bin/jq bin/jq.exe *.larry-prerollback.* diff --git a/CHANGELOG.md b/CHANGELOG.md index 9626441..b1df7da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,109 @@ All notable changes to `cloverleaf-larry` / `larry-anywhere` are recorded here. Versioning is loose-semver; bumps trigger the in-process self-update on every running client via `LARRY_BASE_URL` + `MANIFEST`. +## v0.8.11 — 2026-05-27 + +Two headline features land together (Clover): + +1. **API key is now the default auth rail** — sanctioned, not edge-throttled; + OAuth-impersonation disabled by default to protect the user's Max account; + secure per-client key via `/set-api-key`, stored 0600, CR-safe, masked in + diagnostics. +2. **Manifest-hashing auto-update** — relaunch skips unchanged files by + comparing each MANIFEST sha256 to a local hash, so only changed/missing + files download. Relaunch drops from minutes to seconds. + +--- + +### 1. API key as the default auth rail + +Bryan's decision (2026-05-27): the durable fix for the multi-hour OAuth +`rate_limit_error` is to STOP impersonating the official Claude Code client and +move to the sanctioned API-key rail. Anthropic actively fingerprints and blocks +Claude-Code OAuth impersonation (server-side enforcement since ~2026-01-09, ToS +change 2026-02-19/20); every impersonated OAuth request flags the user's Max +account. The API key (`sk-ant-api03-`) is the intended programmatic rail — a +plain `x-api-key` request, billed pay-as-you-go, not edge-throttled. (Research: +`Deliverables/2026-05-27-claude-code-oauth-request-requirements-research.md`.) + +- **API key is the default.** `LARRY_AUTH_MODE` defaults to `apikey`. The + request shape is the clean sanctioned form: `x-api-key` + `anthropic-version: + 2023-06-01` + `content-type: application/json` — **no `Authorization: Bearer`, + no `anthropic-beta: claude-code-*`, no `claude-cli (external, cli)` UA, no + `x-app: cli`, no `?beta=true`, no "You are Claude Code" system block.** All of + the prior impersonation scaffolding was removed. +- **OAuth is OFF by default (opt-in only).** larry fires OAuth ONLY when + `LARRY_AUTH_MODE=oauth` is set explicitly, and prints a one-time + account-risk warning when it does. There is **no silent OAuth fallback** — + larry never auto-pokes the impersonation tripwire. The opt-in OAuth request is + minimal and honest (`Bearer` + `anthropic-beta: oauth-2025-04-20` only). +- **Secure per-client API-key provisioning + storage** (Bryan's core ask): + - `/set-api-key` (and `larry-auth.sh --api-key`) prompts with `read -s` + (silent, never echoed, never in argv / process table / shell history), + optionally validates with one cheap `/v1/messages` ping (`max_tokens:1`), + then stores the key at `$LARRY_HOME/.api-key` (mode **0600**, owner-only), + **CR-stripped** (MobaXterm/Cygwin CRLF-safe). `--clear` removes it; + `--status` shows it masked. Each client holds its OWN key (mint one per + machine at console.anthropic.com — independently revocable, leak-contained). + - The key is fed to curl via `--config -` on **stdin**, so it never appears in + curl's argv / the process table (`ps`-clean), in all request and validation + paths. +- **Secret hygiene:** the key is never logged, never committed (added to + `.gitignore` and to larry's PHI/secret `read_file` path-block), and is + **masked** as `sk-ant-api03-XXXX…last4` in `/auth`, `/auth-debug` (alias + `/api-debug`), and `/set-api-key --status`. The audit-trail secret-guard's + `sk-ant-[A-Za-z0-9_-]{20,}` pattern catches `sk-ant-api03-` specifically. +- **429-discrimination (reused from the prior thread's good work):** a real + rate-limit 429 ALWAYS carries `anthropic-ratelimit-*` headers → legitimate + backoff; a 429 with NO such headers on the API-key rail → clear "edge/ + transient bounce, not your quota" message (no futile backoff). On the opt-in + OAuth rail, that same edge-reject signature triggers an automatic flip to the + sanctioned API-key rail if a key is configured (`LARRY_NO_EDGE_FALLBACK=1` to + opt out). +- **Migration UX:** first launch with the apikey default and no key → friendly + prompt to run `/set-api-key` (not a bounce into the risky OAuth path). + +### 2. Manifest-hashing auto-update speedup + +`sync_from_manifest` used to re-download **all 48 manifest entries** every +relaunch over authenticated HTTPS (Gitea via proxy + Cloudflare) and `cmp` +locally to find the 0–3 that changed — ~3 min on the work-box for a 3-file +update, because the MANIFEST was paths-only and the client could not tell what +changed without fetching everything. + +- **MANIFEST now ships each file's expected sha256** (`pathsha256`, + generated at release by `scripts/make-manifest.sh`). The client fetches + MANIFEST once, hashes its LOCAL copy of each path, and downloads ONLY entries + whose hash differs or are missing. 48 round-trips → **1 (MANIFEST) + a local + hash pass + N real downloads** (N = actually-changed files, usually 0–3). + Relaunch drops from minutes to seconds. +- **Fail-SAFE: a doubt NEVER skips an update.** A download is skipped only when + ALL hold — a working sha256 tool, a valid 64-hex manifest hash, the local file + exists, and its hash matches exactly. No tool / empty / malformed / non-hex + hash, missing local file, hash-tool error, or any mismatch (including a stale + or wrong published hash) all fall through to **download**. Worst case is a + needless re-download, never a missed update. +- **sha256 tool fallback chain:** `sha256sum` → `shasum -a 256` → + `openssl dgst -sha256` → (none → full-download fallback, updater never breaks). + Detected once, cached. Tool output normalized to bare lowercase 64-hex. +- **CR-safe:** the MANIFEST is fetched from Gitea (CRLF risk); the whole line is + CR-stripped before splitting and the hash-tool output is `tr -d '\r'`'d, so a + CRLF-tainted hash never forces a needless re-download. +- **Download path unchanged** from v0.8.9 — still routes through `fetch_validate` + (HTML-sign-in-trap detection), keeps the post-download `cmp` guard + (idempotent), `chmod +x` for `*.sh`, and per-file `--max-time`. The v0.8.9 + live progress indicator now renders over the new "verifying (local)" phase and + the (fewer) "downloading" frames. New summary line: `manifest sync: N updated, + M unchanged (local hash), F failed, T total`. +- **Release tooling (committed, not synced to clients):** + `scripts/make-manifest.sh` regenerates/checks the MANIFEST hashes; + `scripts/hooks/pre-commit` blocks a commit whose MANIFEST hashes drifted. These + are release-side only — the work-box consumes manifests, never generates them — + so they are deliberately NOT listed in MANIFEST. + +Deliverables: `Deliverables/2026-05-27-cloverleaf-larry-api-key-default-rail.md`, +`Deliverables/2026-05-27-cloverleaf-larry-manifest-hashing-speedup.md`. + ## v0.8.9 — 2026-05-27 Manifest-sync live progress indicator (Clover). Symptom: Bryan's auto-update diff --git a/MANIFEST b/MANIFEST index 643fe3f..3f26243 100644 --- a/MANIFEST +++ b/MANIFEST @@ -1,60 +1,76 @@ # larry-anywhere update manifest -# Format: one path per line, relative to the bundle root. +# Format (v0.8.11+): one entry per line as PATH{TAB}SHA256 relative to the +# bundle root (a literal tab separates the path from its hash). The sha256 is +# the file's expected content hash AT RELEASE. # Lines starting with '#' and blank lines are ignored. +# # Every file listed here is auto-synced by larry.sh's self_update() each time # the running larry.sh version changes (and on first launch of a new version). +# The client (sync_from_manifest) fetches THIS file once, hashes its local copy +# of each path, and downloads ONLY the entries whose hash differs or are missing +# — turning ~48 network round-trips into 1 + a local hash pass + N real fetches. # -# To add a new file to the auto-sync set: list it here and bump VERSION. +# DO NOT HAND-EDIT THE HASHES. They are generated. Every release MUST run: +# scripts/make-manifest.sh (rewrites this file with correct hashes) +# A pre-commit hook (scripts/make-manifest.sh --install-hook) blocks commits +# whose hashes have drifted from the working tree. +# +# Backward/fail-safe: a path line with NO hash (old paths-only format, or a +# malformed/un-hashable entry) is treated by the client as "can't verify -> +# download" — a stale or missing hash can never SKIP a real update. +# +# To add a new file to the auto-sync set: list its path here, then run +# scripts/make-manifest.sh and bump VERSION. # Top-level scripts -larry.sh -larry-tunnel.sh -larry-auth.sh -larry-rollback.sh -install-larry.sh +larry.sh 3ebb6334b6411edd9bb58f4781f8a2db4e5cc470ad4df1d4496dce32084dcf94 +larry-tunnel.sh 6b050e4eeab15669f4858eaf3b807f168f211ced07815db9521bc40a093f6aaa +larry-auth.sh a220cdf7878569dc3028951ee57fc8d5e706a8ca5c6aa45347b58facb386f831 +larry-rollback.sh 91b5e9aa6c79266bf306dcfba4ca791c07971bd6924d67a779037531648aa6d0 +install-larry.sh e97da4e12a0d8863ca18d79b12f6c4294c72fa6d4b11dffeab66504236bb4eb1 # Metadata -VERSION -MANUAL.md -CHANGELOG.md +VERSION f9769094d309a393d86c25196f58f5e47ba88ce3b0e8921458610e7120ed992e +MANUAL.md 755d98b802cb16a5d2d207d423b12c6ca632f118ee372cb5093fe2320a6515ce +CHANGELOG.md eb66afac74c1992ff0812e755c921d425d842bd0b0616ff8eaa293186cadf224 # Agent personas (system-prompt overlays) -agents/larry.md -agents/clover.md -agents/cloverleaf-cheatsheet.md -agents/regress.md +agents/larry.md ace30b97a166c9f244df66ac5f5944e9251dda375a45340d443bccb34bc5ec94 +agents/clover.md d1bbfd6cc4642c2bff6e15dcbdf051d71b063b3fe29e0be97d17b3180d3c7ac5 +agents/cloverleaf-cheatsheet.md c0a2aab91f1ddf092bce312def02cc6f3f62a1f653ca5af67a9430c3fcef4c3f +agents/regress.md bb05ed1439b1e35d6e9799e32d683bfab166472c72115c1f02757e227c74e42f # Cygwin/MobaXterm CR-taint defense primitives (sourced by every tool) -lib/cygwin-safe.sh +lib/cygwin-safe.sh efa83387f03e213a2b200b78cf5468dc930d71b4e3dbb98477187057fa8f4857 # v0.8.4: content-validating fetch (HTML-sign-in-page trap detection + per- # file-type shape checks) for the installer/auto-updater. Canonical home of the # validators that install-larry.sh and larry.sh also carry inline (pre-source). -lib/fetch-safe.sh +lib/fetch-safe.sh abecf0045b9856f63ffa346119443c11de56547344be32bddaed9fbae6b021f4 # Auth implementation -lib/oauth.sh +lib/oauth.sh 04a93376f88fe53cc1c86a5dbe577735c60375dadd4f2fda55b921ef3cddf22b # Secure SSH with ControlMaster (password hidden from Larry-the-LLM) -lib/ssh-helper.sh +lib/ssh-helper.sh d73924a18b0c0c5856c68fb538af706316e23679f72cd94b694be73325231c9b # v0.8.6: work-box → Mac headers.log sync (tsk-2026-05-27-023). Incremental, # offset-tracked push of $LARRY_HOME/log/headers.log to a daemon-watched path # on Bryan's Mac, riding the existing ssh-helper ControlMaster. Drives the # /headers-sync slash command and the on-exit auto-sync hook. Graceful on every # failure mode (no target / closed master / transport error → warn + continue). -lib/headers-sync.sh +lib/headers-sync.sh 47b1946f807b213a2e77cec71128a84a35f103e12fea13ae88d24610d8ee817a # Logging / capture -lib/lessons.sh -lib/journal.sh +lib/lessons.sh 45ea4fdadb843701cd3e87f6a0011ba4097978661851ebc9098ad22ea219efb1 +lib/journal.sh 11c62a2d47b6b67a2f423fd8b86c454126df18d2dc3e150233bbd08293e39fe7 # HL7 utilities -lib/hl7-sanitize.sh -lib/hl7-desanitize.sh -lib/hl7-diff.sh -lib/hl7-field.sh -lib/hl7-schema.sh +lib/hl7-sanitize.sh 6c7d068e0f8538683074c11cf3350868021e9c0f1823f26bf83afdc285d5dc75 +lib/hl7-desanitize.sh d43e29eefde170cdee64b31383d32ccc995773eec9ccad26a18d4cf2270e58f5 +lib/hl7-diff.sh 162ad0e2ed2cd0e57f395ed53c4b3aa0d8f094ee08fa648f4724e0bda176f464 +lib/hl7-field.sh e70b032b6f3d7056fe77a564dafb1025c0feae4eaf596fb7cf315893442c1d42 +lib/hl7-schema.sh 2ba4057a214867ff4950f10057ee4ffd7149e1a82ba94b07b6857d77bf10d75f # v0.8.2: Microsoft Presidio sidecar (optional, opt-in install). # Closes V1 free-text PHI gap from Vera's audit. Requires Python 3.9+ and @@ -62,31 +78,31 @@ lib/hl7-schema.sh # + spaCy en_core_web_sm. install-larry.sh offers to install on first run. # Larry's tier-5 silently skips when sidecar is unreachable, so syncing # these files is safe even on hosts where Python deps aren't installed. -lib/phi-presidio-sidecar.py -lib/phi-sidecar.sh -lib/phi-client.sh +lib/phi-presidio-sidecar.py 8b2662a932d0090bcad97633f12883b1ada6685f349959077a8f8d8760353673 +lib/phi-sidecar.sh e57c6a03f1e6432f9a0f96df44508e6fd656238ac17043bed9b1231f20a9b83a +lib/phi-client.sh defe69b92cfedc6ca01aced20bdbc40a4c3d1002d63686ccadbd16b598028e81 # Generic helpers -lib/each.sh -lib/each-site.sh -lib/len2nl.sh -lib/csv-to-table.sh -lib/table-to-csv.sh +lib/each.sh 14afb974dc27ee8d94e55189323950175fef443106394e20b36b550f46504c84 +lib/each-site.sh 00a31166c2cfcb610e16d4fd8bef439554b10b9e07f93f6917e5732c4ba0957d +lib/len2nl.sh e84196f965ff77c49d3c8b3a776be30342f78c8b2e1c165907089d6ace908746 +lib/csv-to-table.sh ca9cfbcb9f6f42bf7925bf8b8d1e25f0f7209be138d7b971e3adef3ff6b5b3dd +lib/table-to-csv.sh ad98e73687bc9e9f6ae0cd79ed5ba26c856076902865230f822dec1a1beae4b1 # NetConfig tooling -lib/nc-engine.sh -lib/nc-status.sh -lib/nc-table.sh -lib/nc-xlate.sh -lib/nc-smat-diff.sh -lib/nc-create-thread.sh -lib/nc-tclgen.sh -lib/nc-parse.sh -lib/nc-inbound.sh -lib/nc-make-jump.sh -lib/nc-msgs.sh -lib/nc-document.sh -lib/nc-diff-interface.sh -lib/nc-find.sh -lib/nc-insert-protocol.sh -lib/nc-regression.sh +lib/nc-engine.sh fbb87aa704a1517f4fa713ccc57301e8744672a69a3e83589a444ff915b7ec24 +lib/nc-status.sh a300efdaaef8e2764256ca6d8288a5fd4c1cf5097d8c5b7495135ac0ebf0f5a2 +lib/nc-table.sh a6d5c11dd460cfb100ea50c74d57c1a46ef49112632037534a32cd28600abe7f +lib/nc-xlate.sh ea02693c3dff5db271771d4bb2927b23465b07798df2f9912bc2d2b58a134d54 +lib/nc-smat-diff.sh ac003954701ea6b7f4aa1f6941f8536af5b5cdfbb75e306789753d453f06800e +lib/nc-create-thread.sh 5a9d5407c117183cad831d6b95f0e785b1b806f5ccc67f803c12b3695882b5b7 +lib/nc-tclgen.sh dc95f523d543192fc7b3ae204107ce67ebb9b7e5184fa0642a1af2e2454d3241 +lib/nc-parse.sh 834c294b156f4b10776db27203a8cc0ede1e98c753ef0d9d087c8619ca710d73 +lib/nc-inbound.sh 52d28c5f8d97bdf96f0fc7b5300d35b106b8e1226578f4cda430deb2a8b4a91b +lib/nc-make-jump.sh 08a0bc58a299c95c60a59a5202792daf0ada3a8a0be7dc1b4cccc5724f5c9c79 +lib/nc-msgs.sh 729e2d6c9159e83fa177fc6b982e48ed8453a9743477cc90afdd3cd4ec7e620c +lib/nc-document.sh 1f95082df3a88086868e5c159dddd4fd4019b706dbe1e48f0d7500eb9cd6c063 +lib/nc-diff-interface.sh 6b64ec3070a3d75d1d79632c9aeb357177fdbcc77c474aa78e6f6929fda1a324 +lib/nc-find.sh 8c79e0acad7de56e4e1f12d61e071a4b98c4e2310a1f7fb183697df521215e3f +lib/nc-insert-protocol.sh ad1fa0bafbf4fdfb12bad20f9c22c3eed519f8846774331e26aa9becd6f8898a +lib/nc-regression.sh b3583fb07cbf46518312613401acb1e5b07bd2d81a4d259a297b47342182b403 diff --git a/VERSION b/VERSION index 55485e1..83ce05d 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.8.9 +0.8.11 diff --git a/install-larry.sh b/install-larry.sh index c218232..023b3c0 100755 --- a/install-larry.sh +++ b/install-larry.sh @@ -105,7 +105,9 @@ fetch_validate() { if printf '%s' "$first" | grep -q '<'; then rm -f "$tmp"; printf 'error: %s — MANIFEST contains HTML markup ("<").\n' "$url" >&2; return 1 fi - if ! grep -Eq '^[A-Za-z0-9_][A-Za-z0-9_./-]*$' "$tmp"; then + # v0.8.11: accept "pathsha256" (hash group optional => legacy paths- + # only still matches). The '<' guard above is the HTML-trap defense. + if ! grep -Eq '^[A-Za-z0-9_][A-Za-z0-9_./-]*([[:space:]]+[0-9a-fA-F]{64})?[[:space:]]*$' "$tmp"; then rm -f "$tmp"; printf 'error: %s — MANIFEST has no plausible path line.\n' "$url" >&2; return 1 fi ;; script) diff --git a/larry-auth.sh b/larry-auth.sh index 74a0028..26612f8 100755 --- a/larry-auth.sh +++ b/larry-auth.sh @@ -1,16 +1,107 @@ #!/usr/bin/env bash -# larry-auth.sh — top-level wrapper for OAuth subscription auth. -# Forwards to lib/oauth.sh, which contains the actual implementation. +# larry-auth.sh — top-level auth wrapper. +# larry-auth.sh --api-key [--no-validate] set the per-client API key (default rail) +# larry-auth.sh --api-key --clear remove the stored key +# larry-auth.sh --api-key --status show the key masked +# larry-auth.sh forward to lib/oauth.sh (opt-in OAuth) +# +# API key is the DEFAULT / sanctioned rail (v0.8.10). OAuth is opt-in and risks +# the user's Max account (Anthropic blocks Claude-Code impersonation). See +# Deliverables/2026-05-27-cloverleaf-larry-api-key-default-rail.md. set -e SELF_DIR="$(cd "$(dirname "$0")" && pwd)" LARRY_HOME="${LARRY_HOME:-$HOME/.larry}" +LARRY_API_KEY_FILE="${LARRY_API_KEY_FILE:-$LARRY_HOME/.api-key}" +LARRY_API_URL="${LARRY_API_URL:-https://api.anthropic.com/v1/messages}" +LARRY_MODEL="${LARRY_MODEL:-claude-sonnet-4-6}" -# Locate oauth.sh: prefer sibling-of-this-script, then $LARRY_HOME/lib -OAUTH="" -for c in "$SELF_DIR/lib/oauth.sh" "$LARRY_HOME/lib/oauth.sh"; do - [ -x "$c" ] && { OAUTH="$c"; break; } -done -[ -n "$OAUTH" ] || { echo "larry-auth: cannot find lib/oauth.sh — reinstall larry-anywhere" >&2; exit 1; } +# strip_cr — prefer the shared primitive; fall back to inline parameter expansion. +if [ -r "$SELF_DIR/lib/cygwin-safe.sh" ]; then + # shellcheck disable=SC1091 + . "$SELF_DIR/lib/cygwin-safe.sh" +elif [ -r "$LARRY_HOME/lib/cygwin-safe.sh" ]; then + # shellcheck disable=SC1091 + . "$LARRY_HOME/lib/cygwin-safe.sh" +fi +command -v strip_cr >/dev/null 2>&1 || strip_cr() { local v="${1:-}"; printf '%s' "${v//$'\r'/}"; } -exec "$OAUTH" "$@" +_mask_api_key() { + local k; k=$(strip_cr "${1:-}") + [ -z "$k" ] && { printf '(none)'; return 0; } + local len=${#k} + [ "$len" -le 12 ] && { printf '(set, len=%d)' "$len"; return 0; } + printf '%s…%s (len=%d)' "$(printf '%s' "$k" | cut -c1-13)" "$(printf '%s' "$k" | tail -c 4)" "$len" +} + +# One cheap test call to confirm the key authenticates. 0=valid, 1=invalid +# (prints HTTP code), 2=could-not-test (no curl / network). +_validate_api_key() { + local key; key=$(strip_cr "${1:-}") + command -v curl >/dev/null 2>&1 || return 2 + local body code + body='{"model":"'"$LARRY_MODEL"'","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}' + # Key via --config on stdin (off argv / process table). + code=$(printf 'header = "x-api-key: %s"\n' "$key" | curl -sS -o /dev/null -w '%{http_code}' --config - \ + -H "anthropic-version: 2023-06-01" -H "content-type: application/json" \ + --max-time 20 -d "$body" "$LARRY_API_URL" 2>/dev/null) || return 2 + [ "$code" = "200" ] && return 0 + printf '%s' "$code"; return 1 +} + +set_api_key_cli() { + local do_validate=1 + [ "${1:-}" = "--no-validate" ] && do_validate=0 + mkdir -p "$LARRY_HOME" 2>/dev/null || true + echo "Set Anthropic API key (per-client). Mint one for THIS machine at" + echo "https://console.anthropic.com — one dedicated, independently-revocable key." + echo "Stored at $LARRY_API_KEY_FILE (mode 0600); never leaves this machine." + printf 'Paste key (input hidden): ' + local key="" + read -rs key 2>/dev/null || read -r key + echo "" + key=$(strip_cr "$key"); key="${key//$'\n'/}" + key="${key#"${key%%[![:space:]]*}"}"; key="${key%"${key##*[![:space:]]}"}" + [ -z "$key" ] && { echo "larry-auth: no key entered — nothing changed" >&2; return 1; } + case "$key" in sk-ant-*) : ;; *) echo "warning: doesn't look like sk-ant-... — storing anyway" >&2 ;; esac + if [ "$do_validate" = "1" ]; then + printf 'Validating… ' + local vout vrc; vout=$(_validate_api_key "$key"); vrc=$? + if [ "$vrc" = "0" ]; then echo "valid" + elif [ "$vrc" = "2" ]; then echo "skipped (curl/network unavailable)" + else echo "FAILED (HTTP ${vout:-?})"; echo "larry-auth: key did not authenticate — not stored." >&2; key=""; return 1; fi + fi + umask 077 + printf '%s\n' "$key" > "$LARRY_API_KEY_FILE" + chmod 600 "$LARRY_API_KEY_FILE" 2>/dev/null || true + echo "stored at $LARRY_API_KEY_FILE (0600) — $(_mask_api_key "$key")" + key="" +} + +case "${1:-status}" in + --api-key|--apikey|api-key|apikey) + shift + case "${1:-}" in + --clear) + if [ -f "$LARRY_API_KEY_FILE" ]; then rm -f "$LARRY_API_KEY_FILE"; echo "cleared $LARRY_API_KEY_FILE" + else echo "no key file to remove ($LARRY_API_KEY_FILE)"; fi ;; + --status) + if [ -f "$LARRY_API_KEY_FILE" ]; then + k=$(strip_cr "$(cat "$LARRY_API_KEY_FILE" 2>/dev/null)"); k="${k//$'\n'/}" + echo "API key: $(_mask_api_key "$k") [$LARRY_API_KEY_FILE]"; k="" + else echo "API key: (none set) — run: larry-auth.sh --api-key"; fi ;; + --no-validate) set_api_key_cli --no-validate ;; + ""|set) set_api_key_cli ;; + *) echo "usage: larry-auth.sh --api-key [--clear|--status|--no-validate]" >&2; exit 2 ;; + esac + ;; + *) + # OAuth (opt-in) path — forward to lib/oauth.sh. + OAUTH="" + for c in "$SELF_DIR/lib/oauth.sh" "$LARRY_HOME/lib/oauth.sh"; do + [ -x "$c" ] && { OAUTH="$c"; break; } + done + [ -n "$OAUTH" ] || { echo "larry-auth: cannot find lib/oauth.sh — reinstall larry-anywhere" >&2; exit 1; } + exec "$OAUTH" "$@" + ;; +esac diff --git a/larry.sh b/larry.sh index 16b714a..874fc95 100755 --- a/larry.sh +++ b/larry.sh @@ -34,7 +34,14 @@ # without flipping it public. If a fetch returns the # Gitea HTML sign-in page (HTTP 200), the updater now # FAILS LOUD instead of parsing HTML as file content. -# ANTHROPIC_API_KEY overrides $LARRY_HOME/.env if set +# ANTHROPIC_API_KEY overrides $LARRY_HOME/.api-key and $LARRY_HOME/.env if set +# LARRY_AUTH_MODE auth rail. DEFAULT = apikey (sanctioned x-api-key rail). +# Set to "oauth" to opt INTO subscription OAuth — OFF by +# default because it requires impersonating the official +# Claude Code client, which Anthropic blocks and which +# flags your Max account. No silent OAuth fallback. +# LARRY_API_KEY_FILE per-client key store (default $LARRY_HOME/.api-key, 0600). +# Set via /set-api-key or larry-auth.sh --api-key. # # Slash commands during chat: # /quit /exit /q exit @@ -65,7 +72,7 @@ set -o pipefail # ───────────────────────────────────────────────────────────────────────────── # Config # ───────────────────────────────────────────────────────────────────────────── -LARRY_VERSION="0.8.9" +LARRY_VERSION="0.8.11" LARRY_HOME="${LARRY_HOME:-$HOME/.larry}" # ───────────────────────────────────────────────────────────────────────────── @@ -131,6 +138,39 @@ LARRY_MAX_TOKENS="${LARRY_MAX_TOKENS:-8192}" LARRY_API_URL="${LARRY_API_URL:-https://api.anthropic.com/v1/messages}" LARRY_NO_UPDATE="${LARRY_NO_UPDATE:-0}" +# ───────────────────────────────────────────────────────────────────────────── +# v0.8.10: API key is the DEFAULT / primary auth rail (Bryan's decision, +# 2026-05-27) +# ───────────────────────────────────────────────────────────────────────────── +# WHY API KEY IS DEFAULT (and OAuth-impersonation is OFF): +# The OAuth (`sk-ant-oat01-`) rail bills a Claude Max/Pro subscription, but to +# reach it from a non-Claude-Code client you must SPOOF the official client's +# request fingerprint (the `anthropic-beta: claude-code-*` flag, the +# `claude-cli/ (external, cli)` UA, the `x-app: cli` header, and a +# "You are Claude Code, ..." system block). Anthropic is ACTIVELY enforcing +# against exactly that impersonation: server-side fingerprint blocking live +# since ~2026-01-09, formal ToS change 2026-02-19/20 (the "OpenClaw ban"). +# Every spoofed OAuth request flags the user's account. To protect Bryan's Max +# account we do NOT impersonate Claude Code and we do NOT fire OAuth by default. +# +# The API key (`sk-ant-api03-`) is the SANCTIONED programmatic rail: a plain +# `x-api-key` request, billed pay-as-you-go, NOT subject to the impersonation +# block and NOT edge-throttled as anomalous traffic. That is why it is the +# durable default. See: +# Deliverables/2026-05-27-claude-code-oauth-request-requirements-research.md +# Deliverables/2026-05-27-cloverleaf-larry-api-key-default-rail.md +# +# OPT-IN OAUTH (discouraged): set LARRY_AUTH_MODE=oauth explicitly. larry prints +# a one-time account-risk warning and fires the OAuth rail. There is NO silent +# OAuth fallback — larry never auto-pokes the impersonation tripwire. +LARRY_AUTH_MODE="${LARRY_AUTH_MODE:-}" # "", "apikey", or "oauth"; resolved below + +# Per-client API-key file (the secure provisioning store; see /set-api-key). +# Mode 0600, owner-only, CR-stripped on read. Each client machine holds its OWN +# key — Bryan mints a separate, independently-revocable key per client at +# console.anthropic.com. The key never leaves the machine it is entered on. +LARRY_API_KEY_FILE="${LARRY_API_KEY_FILE:-$LARRY_HOME/.api-key}" + # ───────────────────────────────────────────────────────────────────────────── # Colors (only if stdout is a tty) # ───────────────────────────────────────────────────────────────────────────── @@ -227,7 +267,12 @@ fetch_validate() { if printf '%s' "$first" | grep -q '<'; then rm -f "$tmp"; printf 'error: %s — MANIFEST contains HTML markup ("<").\n' "$url" >&2; return 1 fi - if ! grep -Eq '^[A-Za-z0-9_][A-Za-z0-9_./-]*$' "$tmp"; then + # v0.8.11: accept BOTH the new "pathsha256" form AND the legacy + # paths-only form. A plausible line starts with a path token, optionally + # followed by whitespace + a 64-hex hash (the hash group is optional, so + # legacy paths-only manifests still validate). The blanket grep -q '<' + # HTML-trap guard above is unchanged and remains the real defense. + if ! grep -Eq '^[A-Za-z0-9_][A-Za-z0-9_./-]*([[:space:]]+[0-9a-fA-F]{64})?[[:space:]]*$' "$tmp"; then rm -f "$tmp"; printf 'error: %s — MANIFEST has no plausible path line.\n' "$url" >&2; return 1 fi ;; script) @@ -300,84 +345,211 @@ mkdir -p "$LARRY_HOME/agents" "$LARRY_HOME/sessions" "$LARRY_HOME/bin" 2>/dev/nu chmod 700 "$LARRY_HOME" 2>/dev/null || true # ───────────────────────────────────────────────────────────────────────────── -# Authentication — two modes, OAuth preferred when available: -# 1. OAuth subscription auth (bills against your Claude Max/Pro subscription). -# Token file at $LARRY_HOME/.oauth.json — managed by larry-auth.sh. -# 2. API key (separate pay-as-you-go API billing). Stored in $LARRY_HOME/.env. +# Authentication — API key is the DEFAULT / primary rail (v0.8.10). +# 1. API key (`sk-ant-api03-`) — SANCTIONED programmatic billing, the default. +# Stored per-client at $LARRY_HOME/.api-key (0600, CR-safe). Provisioned +# via /set-api-key (or larry-auth.sh --api-key). Legacy: $LARRY_HOME/.env +# with ANTHROPIC_API_KEY=... is still honored. +# 2. OAuth subscription auth — OPT-IN ONLY (LARRY_AUTH_MODE=oauth). Bills a +# Claude Max/Pro subscription but requires impersonating the official +# Claude Code client, which Anthropic actively blocks/flags. OFF by default +# to protect the user's account. There is NO silent OAuth fallback. # ───────────────────────────────────────────────────────────────────────────── -LARRY_AUTH_MODE="" # set later: "oauth" or "apikey" -if [ -f "$LARRY_HOME/.oauth.json" ]; then - LARRY_AUTH_MODE="oauth" -elif [ -z "${ANTHROPIC_API_KEY:-}" ]; then +# _load_api_key_into_env — populate $ANTHROPIC_API_KEY (if not already set in the +# environment) from the per-client key file first, then legacy .env. CR-stripped +# inline (strip_cr from cygwin-safe.sh isn't sourced this early). An env-supplied +# ANTHROPIC_API_KEY always wins and is never overwritten. +_load_api_key_into_env() { + [ -n "${ANTHROPIC_API_KEY:-}" ] && return 0 + if [ -f "$LARRY_API_KEY_FILE" ]; then + local _k; _k=$(cat "$LARRY_API_KEY_FILE" 2>/dev/null) + _k="${_k//$'\r'/}"; _k="${_k//$'\n'/}" # strip CR and any stray newline + if [ -n "$_k" ]; then ANTHROPIC_API_KEY="$_k"; export ANTHROPIC_API_KEY; return 0; fi + fi if [ -f "$LARRY_HOME/.env" ]; then # shellcheck disable=SC1091 set -a; . "$LARRY_HOME/.env"; set +a + # A CR-tainted .env (CRLF) can leave a trailing \r on the key; scrub it. + [ -n "${ANTHROPIC_API_KEY:-}" ] && ANTHROPIC_API_KEY="${ANTHROPIC_API_KEY//$'\r'/}" fi - [ -n "${ANTHROPIC_API_KEY:-}" ] && LARRY_AUTH_MODE="apikey" + return 0 +} + +if [ "$LARRY_AUTH_MODE" = "oauth" ]; then + # Explicit opt-in to the OAuth-impersonation rail. We honor it but DO load the + # API key too (so /logout or a manual flip lands on a working rail) and warn + # once (see _warn_oauth_optin_once, fired at first use). No spoofing happens + # here — call_api's OAuth branch is the only place the OAuth token is sent. + _load_api_key_into_env else - LARRY_AUTH_MODE="apikey" + # DEFAULT: API key. Resolve a key from the per-client file or legacy .env. + _load_api_key_into_env + if [ -n "${ANTHROPIC_API_KEY:-}" ]; then + LARRY_AUTH_MODE="apikey" + else + LARRY_AUTH_MODE="" # no key yet → first-run prompt guides to /set-api-key + fi fi +# Snapshot the operator's CHOSEN primary auth mode for diagnostics/status. +LARRY_PRIMARY_AUTH_MODE="$LARRY_AUTH_MODE" + +# _warn_oauth_optin_once — one-time account-risk warning printed the first time +# the OAuth rail is actually used. OAuth-impersonation flags the user's Max +# account (Anthropic's anti-impersonation enforcement); the API-key rail is the +# safe default. Guarded by a flag file so it prints at most once per LARRY_HOME. +_warn_oauth_optin_once() { + local flag="$LARRY_HOME/.oauth-optin-warned" + [ -f "$flag" ] && return 0 + warn "OAuth mode is OPT-IN and risks your Claude Max account." + warn " Reaching the OAuth rail requires impersonating the official Claude Code" + warn " client, which Anthropic actively fingerprints and blocks (ToS change" + warn " 2026-02-19; enforcement live since ~2026-01-09). Each request can flag" + warn " your account. The API key (sk-ant-api03-) is the sanctioned default —" + warn " run /set-api-key and unset LARRY_AUTH_MODE to use it. (This warning" + warn " shows once.)" + : > "$flag" 2>/dev/null || true +} + +# _mask_api_key KEY — render a key for human display as the sk-ant-api03- +# prefix + "…" + last 4 chars. NEVER prints the middle. Used by every +# diagnostic so the full key is never echoed. +_mask_api_key() { + local k="${1:-}" + k="${k//$'\r'/}" + [ -z "$k" ] && { printf '(none)'; return 0; } + local len=${#k} + if [ "$len" -le 12 ]; then + # Too short to be a real key; show only a length hint, never the bytes. + printf '(set, len=%d)' "$len" + return 0 + fi + local prefix last4 + prefix=$(printf '%s' "$k" | cut -c1-13) # "sk-ant-api03-" + last4=$(printf '%s' "$k" | tail -c 4) + printf '%s…%s (len=%d)' "$prefix" "$last4" "$len" +} + +# _validate_api_key KEY — one cheap test call to /v1/messages (max_tokens:1) to +# confirm the key authenticates before we store it. Returns 0 on HTTP 200, 1 +# otherwise. The key travels ONLY in the x-api-key header of this curl (never in +# argv, never logged). Best-effort: if curl/jq are missing or the network is +# down we SKIP validation (return 2) rather than block provisioning. +_validate_api_key() { + local key="${1:-}" + command -v curl >/dev/null 2>&1 || return 2 + key="${key//$'\r'/}" + local body code + body='{"model":"'"${LARRY_MODEL:-claude-sonnet-4-6}"'","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}' + # Key via --config on stdin so it never lands in curl's argv / the process table. + code=$(printf 'header = "x-api-key: %s"\n' "$key" | curl -sS -o /dev/null -w '%{http_code}' --config - \ + -H "anthropic-version: 2023-06-01" \ + -H "content-type: application/json" \ + --max-time 20 \ + -d "$body" \ + "${LARRY_API_URL:-https://api.anthropic.com/v1/messages}" 2>/dev/null) || return 2 + [ "$code" = "200" ] && return 0 + # A 200 OR a 400 "max_tokens too small"-class response both prove the key + # authenticated; a 401/403 means a bad key. Treat 200 as the clean pass and + # surface the code to the caller via stdout for messaging. + printf '%s' "$code" + return 1 +} + +# set_api_key [--no-validate] — the canonical secure provisioning path. Prompts +# for the key with read -s (silent, NEVER echoed, NEVER in argv so it can't hit +# the process table or shell history), optionally validates it, then stores it +# CR-stripped at $LARRY_API_KEY_FILE with mode 0600. Used by the first-run +# prompt AND the /set-api-key slash command AND larry-auth.sh --api-key. +set_api_key() { + local do_validate=1 + [ "${1:-}" = "--no-validate" ] && do_validate=0 + printf '%sSet Anthropic API key (per-client)%s\n' "$C_BOLD" "$C_RESET" + echo " Mint a key for THIS machine at https://console.anthropic.com (one" + echo " dedicated, independently-revocable key per client). It is stored at" + echo " $LARRY_API_KEY_FILE (mode 0600) and never leaves this machine." + echo "" + printf ' Paste key (input hidden): ' + # read -s: silent, no echo. The key lands in $key only — never in argv, never + # in the process table, never in shell history. + local key="" + read -rs key 2>/dev/null || read -r key # -s unsupported on some shells → fallback + echo "" + # CR-strip (MobaXterm/Cygwin paste taints with \r; a \r in the key breaks the + # x-api-key header). Also drop any stray surrounding whitespace/newline. + key="${key//$'\r'/}"; key="${key//$'\n'/}" + key="${key#"${key%%[![:space:]]*}"}"; key="${key%"${key##*[![:space:]]}"}" + if [ -z "$key" ]; then err "no key entered — nothing changed"; return 1; fi + case "$key" in + sk-ant-*) : ;; + *) warn "that doesn't look like an Anthropic key (expected sk-ant-...). Storing anyway." ;; + esac + + if [ "$do_validate" = "1" ]; then + printf ' Validating key (one cheap test call)… ' + local vrc vout + vout=$(_validate_api_key "$key"); vrc=$? + if [ "$vrc" = "0" ]; then + printf '%svalid%s\n' "$C_GREEN" "$C_RESET" + elif [ "$vrc" = "2" ]; then + printf '%sskipped (curl/network unavailable)%s\n' "$C_YELLOW" "$C_RESET" + else + printf '%sFAILED (HTTP %s)%s\n' "$C_RED" "${vout:-?}" "$C_RESET" + err "key did not authenticate — not stored. Check the key and retry, or use --no-validate to force." + key="" # scrub + return 1 + fi + fi + + umask 077 + printf '%s\n' "$key" > "$LARRY_API_KEY_FILE" + chmod 600 "$LARRY_API_KEY_FILE" 2>/dev/null || true + ANTHROPIC_API_KEY="$key"; export ANTHROPIC_API_KEY + LARRY_AUTH_MODE="apikey" + LARRY_PRIMARY_AUTH_MODE="apikey" + log "API key stored at $LARRY_API_KEY_FILE (0600) — $(_mask_api_key "$key")" + key="" # scrub local before return + return 0 +} + +# clear_api_key — remove the stored per-client key and unset it from the env. +clear_api_key() { + if [ -f "$LARRY_API_KEY_FILE" ]; then + rm -f "$LARRY_API_KEY_FILE" + unset ANTHROPIC_API_KEY + log "API key cleared (removed $LARRY_API_KEY_FILE)." + else + echo "no API key file to remove ($LARRY_API_KEY_FILE)" + fi +} + +# show_api_key_status — masked status only. NEVER prints the full key. +show_api_key_status() { + if [ -n "${ANTHROPIC_API_KEY:-}" ]; then + printf ' API key: %s [source: %s]\n' \ + "$(_mask_api_key "$ANTHROPIC_API_KEY")" \ + "$([ -f "$LARRY_API_KEY_FILE" ] && echo "$LARRY_API_KEY_FILE" || echo 'env/.env')" + elif [ -f "$LARRY_API_KEY_FILE" ]; then + local k; k=$(cat "$LARRY_API_KEY_FILE" 2>/dev/null); k="${k//$'\r'/}"; k="${k//$'\n'/}" + printf ' API key: %s [source: %s]\n' "$(_mask_api_key "$k")" "$LARRY_API_KEY_FILE" + k="" + else + printf ' API key: (none set) — run /set-api-key\n' + fi +} prompt_first_run_auth() { printf '%sFirst-run authentication setup%s\n\n' "$C_BOLD" "$C_RESET" cat </dev/null - read -r key - stty echo 2>/dev/null - echo "" - # v0.7.5: strip CR so the API key written to .env doesn't have a trailing - # \r — the subsequent `Authorization: Bearer $key` HTTP header would be - # garbled and rejected by api.anthropic.com. - key="${key//$'\r'/}" - if [ -z "$key" ]; then err "no key entered"; exit 1; fi - umask 077 - printf 'ANTHROPIC_API_KEY=%s\n' "$key" > "$LARRY_HOME/.env" - chmod 600 "$LARRY_HOME/.env" - ANTHROPIC_API_KEY="$key" - log "API key saved." + set_api_key || { err "no API key set — run /set-api-key, or relaunch with LARRY_AUTH_MODE=oauth"; exit 1; } } # NOTE: the auth-prompt CALL (prompt_first_run_auth) is deliberately deferred @@ -474,6 +646,57 @@ _record_origin() { _LARRY_LAST_ORIGIN_URL="$2" } +# ───────────────────────────────────────────────────────────────────────────── +# v0.8.11: local sha256 for manifest-hash skip-unchanged. +# +# WHY: sync_from_manifest used to re-download EVERY manifest entry over an +# authenticated HTTPS round-trip (Gitea via proxy + Cloudflare) and `cmp` +# locally to find the few that changed — ~3 min on Bryan's work-box for a +# 3-file update. The MANIFEST now ships each file's expected sha256 (generated +# by scripts/make-manifest.sh at release). The client fetches MANIFEST once, +# hashes its LOCAL copy of each path, and downloads ONLY entries whose hash +# differs or are missing. +# +# These run BEFORE lib/cygwin-safe.sh is sourced (self_update is early), so this +# block is self-contained — no strip_cr / coerce_int dependency. +# +# sha256 TOOL FALLBACK CHAIN (priority): sha256sum, shasum -a 256, +# openssl dgst -sha256. Detected ONCE and cached in _LARRY_SHA_TOOL. If NONE is +# available, _LARRY_SHA_TOOL stays "none" and sync_from_manifest falls back to +# the old full-download behaviour entirely (never breaks the updater). +_LARRY_SHA_TOOL="" # "", then one of: sha256sum|shasum|openssl|none +_detect_sha_tool() { + [ -n "$_LARRY_SHA_TOOL" ] && return 0 + if command -v sha256sum >/dev/null 2>&1; then _LARRY_SHA_TOOL="sha256sum" + elif command -v shasum >/dev/null 2>&1; then _LARRY_SHA_TOOL="shasum" + elif command -v openssl >/dev/null 2>&1; then _LARRY_SHA_TOOL="openssl" + else _LARRY_SHA_TOOL="none" + fi + return 0 +} + +# _local_sha256 FILE — print FILE's bare lowercase 64-hex sha256, or empty on +# any failure (missing file, tool error, unparseable output). Empty result is +# the caller's signal to DOWNLOAD (fail toward correctness — never skip). +# Normalizes each tool's differing output shape to a bare 64-hex string: +# sha256sum / shasum -> " " +# openssl -> "SHA2-256()= " +_local_sha256() { + local f="$1" out="" + [ -f "$f" ] || return 1 + case "$_LARRY_SHA_TOOL" in + sha256sum) out="$(sha256sum "$f" 2>/dev/null)" ;; + shasum) out="$(shasum -a 256 "$f" 2>/dev/null)" ;; + openssl) out="$(openssl dgst -sha256 "$f" 2>/dev/null)" ;; + *) return 1 ;; + esac + # tr -d '\r' first: a Cygwin-built sha tool can emit a CR on stdout; the hex + # grep would still match but belt-and-suspenders. Lowercase A-F for compare. + out="$(printf '%s' "$out" | tr -d '\r' | grep -oE '[0-9a-fA-F]{64}' | head -1 | tr 'A-F' 'a-f')" + [ -n "$out" ] || return 1 + printf '%s' "$out" +} + # ───────────────────────────────────────────────────────────────────────────── # v0.8.9: manifest-sync progress indicator. # @@ -550,20 +773,45 @@ sync_from_manifest() { local self="$0" case "$self" in /*) ;; *) self="$PWD/$self" ;; esac + # v0.8.11: detect the sha256 tool ONCE. If none is available we set + # _have_sha=0 and the per-file loop falls back to the OLD full-download + # behaviour for every entry (download + cmp) — never skips on a missing tool. + _detect_sha_tool + local _have_sha=1 + [ "$_LARRY_SHA_TOOL" = "none" ] && _have_sha=0 + [ "$_have_sha" -eq 0 ] && warn "no sha256 tool (sha256sum/shasum/openssl) — manifest-hash skip disabled, full sync this launch" + # v0.8.9: pre-count manifest entries so the progress indicator has a # denominator. Cheap local pass (no network) over the just-fetched manifest. local total=0 _l while IFS= read -r _l; do + _l="${_l//$'\r'/}" case "$_l" in ''|'#'*) continue ;; esac _l="${_l%%[[:space:]]*}" [ -z "$_l" ] && continue total=$((total + 1)) done < "$manifest" - local count=0 updated=0 failed=0 path tmp dest - while IFS= read -r path; do - case "$path" in ''|'#'*) continue ;; esac - path="${path%%[[:space:]]*}" # strip trailing whitespace/comments + # v0.8.11: skipped = files whose LOCAL sha256 matched the manifest hash + # (verified locally, no download). The new "verifying N/total (local)" phase + # covers this fast local pass; the v0.8.9 "downloading" phase still covers the + # few real fetches. + local count=0 updated=0 failed=0 skipped=0 path mhash lhash tmp dest _rest + while IFS= read -r _l; do + # CR-safety: strip CR from the whole line FIRST (the MANIFEST is fetched + # from Gitea; a CRLF tail would taint the hash field — a CR-tainted hash + # would never match a clean local hash and force needless re-downloads, or + # mismatch). This is the v0.7.5/v0.8.5 CR-taint class. We are pre-source so + # strip via parameter expansion (strip_cr not yet defined here). + _l="${_l//$'\r'/}" + case "$_l" in ''|'#'*) continue ;; esac + # Split "pathhash". path = first token; mhash = next token (if + # any). A line with no second token (old paths-only format) -> mhash empty + # -> treated as "can't verify -> download" below (fail-safe). + path="${_l%%[[:space:]]*}" + _rest="${_l#"$path"}" + _rest="${_rest#"${_rest%%[![:space:]]*}"}" # ltrim the separator whitespace + mhash="${_rest%%[[:space:]]*}" # first token of remainder [ -z "$path" ] && continue count=$((count + 1)) @@ -571,19 +819,49 @@ sync_from_manifest() { # the running script mid-execution. [ "$path" = "larry.sh" ] && continue - # v0.8.9: live progress BEFORE the network round-trip, so a fetch that - # hangs (slow file / proxy stall) shows exactly which file it is stuck on - # rather than freezing silently. fetch_validate's --max-time bounds each - # hang to ${_kind} fetch timeout (15s); on timeout it fails loud, counts as - # a fail, and the loop advances to the next file — never an infinite stall. - _sync_progress checking "$count" "$total" "$path" - dest="$LARRY_HOME/$path" tmp="$dest.new" mkdir -p "$(dirname "$dest")" 2>/dev/null + + # ── v0.8.11 LOCAL-SKIP DECISION (fail toward correctness) ──────────────── + # Skip the download ONLY when ALL of these hold: + # (a) we have a working sha256 tool, + # (b) the manifest line carried a syntactically valid 64-hex hash, + # (c) the local file exists and hashes to exactly that value. + # ANY doubt — no tool, no/short/non-hex manifest hash, missing local file, + # local-hash failure, or hash MISMATCH — falls through to download. A stale + # or wrong hash can therefore never SKIP a real update; worst case is a + # needless re-download. + local _can_skip=0 + if [ "$_have_sha" -eq 1 ] && printf '%s' "$mhash" | grep -qiE '^[0-9a-f]{64}$'; then + if [ -f "$dest" ]; then + lhash="$(_local_sha256 "$dest")" + if [ -n "$lhash" ] && [ "$lhash" = "$(printf '%s' "$mhash" | tr 'A-F' 'a-f')" ]; then + _can_skip=1 + fi + fi + fi + + if [ "$_can_skip" -eq 1 ]; then + # Verified unchanged locally — no network round-trip. Show a fast local + # progress frame so the operator sees forward motion through the verify. + _sync_progress "verifying (local)" "$count" "$total" "$path" + skipped=$((skipped + 1)) + continue + fi + + # ── DOWNLOAD PATH (changed, missing, unverifiable, or no-tool fallback) ── + # v0.8.9: live progress BEFORE the network round-trip, so a fetch that + # hangs (slow file / proxy stall) shows exactly which file it is stuck on + # rather than freezing silently. fetch_validate's --max-time bounds each + # hang to the per-kind fetch timeout (15s); on timeout it fails loud, counts + # as a fail, and the loop advances — never an infinite stall. + _sync_progress downloading "$count" "$total" "$path" + # v0.8.4: per-file content validation. Infer the shape contract from the # path so a sign-in-page (or any HTML) response can never be written over a - # real lib/agent/metadata file. fetch_validate writes $tmp only on success. + # real lib/agent/metadata file. fetch_validate (HTML-sign-in-trap detection, + # v0.8.4) writes $tmp only on success. UNCHANGED on this path. local _kind case "$path" in VERSION) _kind=version ;; @@ -592,10 +870,10 @@ sync_from_manifest() { *) _kind=text ;; esac if fetch_validate "$base/$path" "$tmp" "$_kind" 15 && [ -s "$tmp" ]; then + # cmp guard retained: even after a download, only count + write if the + # bytes actually differ (e.g. the manifest hash was stale-but-the-file-is- + # actually-current, or we fell back here without a tool). Idempotent. if [ ! -f "$dest" ] || ! cmp -s "$dest" "$tmp"; then - # Distinguish a real change (a download that lands) from the common - # case of an unchanged file, so the operator sees actual writes. - _sync_progress downloading "$count" "$total" "$path" mv "$tmp" "$dest" case "$path" in *.sh) chmod +x "$dest" 2>/dev/null || true ;; esac updated=$((updated + 1)) @@ -612,11 +890,12 @@ sync_from_manifest() { # v0.8.9: clear the in-place progress line so the summary lands clean. _sync_progress_done - if [ "$updated" -gt 0 ] || [ "$failed" -gt 0 ]; then - log "manifest sync: $updated updated, $failed failed, $count total (from $base)" + if [ "$updated" -gt 0 ] || [ "$failed" -gt 0 ] || [ "$skipped" -gt 0 ]; then + log "manifest sync: $updated updated, $skipped unchanged (local hash), $failed failed, $count total (from $base)" fi LARRY_SYNC_UPDATED_COUNT="$updated" LARRY_SYNC_FAILED_COUNT="$failed" + LARRY_SYNC_SKIPPED_COUNT="$skipped" return 0 } @@ -908,6 +1187,7 @@ tool_read_file() { # $LARRY_HOME/sessions/ — prior transcript history # $LARRY_HOME/.oauth.json — OAuth subscription tokens # $LARRY_HOME/.env — env-var secrets (if present) +# $LARRY_HOME/.api-key — per-client Anthropic API key (v0.8.10, 0600) # # Portability: # - GNU `realpath -m` resolves nonexistent paths; macOS `realpath` requires @@ -939,6 +1219,16 @@ _read_file_path_blocked() { local canon hcanon canon=$(_read_file_canon "$p" 2>/dev/null || true) hcanon=$(_read_file_canon "$home" 2>/dev/null || true) + # Always block the configured per-client API-key file, even if LARRY_API_KEY_FILE + # points outside $LARRY_HOME. Compare both literal and canonicalized forms so + # a symlinked or relative request can't slip past. + if [ -n "${LARRY_API_KEY_FILE:-}" ]; then + local akf akf_canon + akf="${LARRY_API_KEY_FILE}" + akf_canon=$(_read_file_canon "$akf" 2>/dev/null || true) + case "$p" in "$akf"|"$akf_canon") return 0 ;; esac + [ -n "$canon" ] && case "$canon" in "$akf"|"$akf_canon") return 0 ;; esac + fi local h hp for h in "$home" "$hcanon"; do [ -z "$h" ] && continue @@ -950,6 +1240,7 @@ _read_file_path_blocked() { "$h"/sessions|"$h"/sessions/*) return 0 ;; "$h"/.oauth.json|"$h"/.oauth.json.*) return 0 ;; "$h"/.env|"$h"/.env.*) return 0 ;; + "$h"/.api-key|"$h"/.api-key.*) return 0 ;; esac done done @@ -2459,6 +2750,9 @@ _LARRY_OUTPUT_TOKENS=0 _LARRY_CACHE_READ_TOKENS=0 _LARRY_CACHE_WRITE_TOKENS=0 _LARRY_TURNS=0 +# v0.8.10: one-shot guard — set to 1 once the edge-429→API-key fallback has +# flipped LARRY_AUTH_MODE to apikey, so we never flip-flop within a session. +_LARRY_EDGE_FALLBACK_DONE=0 # ───────────────────────────────────────────────────────────────────────────── # v0.6.9: Persistent status line — ctx + rate-limit visibility @@ -2496,6 +2790,13 @@ STATUS_api_reset_epoch="" # earliest of the *-reset RFC3339 timestamps, a STATUS_retry_after_secs="" # raw `retry-after` header value (seconds), if present STATUS_rl_tripped_rail="" # which bucket is at/over limit: requests|input-tokens|output-tokens|tokens|unified-5h|unified-7d STATUS_rl_reset_epoch="" # epoch when the tripped rail resets (best-effort) +# v0.8.10: edge-rejection discrimination. Set to 1 by _parse_response_headers +# when the response is the EDGE-429 signature: HTTP 429 + x-should-retry:true + +# ZERO anthropic-ratelimit-* headers. That is Anthropic's gateway bouncing the +# request BEFORE rate-limit accounting (an auth/fingerprint reject), NOT a quota +# limit. A real quota 429 carries anthropic-ratelimit-* headers → this stays 0 +# and the normal backoff path runs. Reset per response so it never goes stale. +STATUS_rl_edge_reject=0 # session_cost is reused from _LARRY_INPUT/OUTPUT/CACHE_*_TOKENS via # _render_session_cost_dollars (no new state needed). # Session turns counter == _LARRY_TURNS (no new state needed). @@ -2688,6 +2989,38 @@ _parse_response_headers() { _is_429=1 fi + # ── v0.8.10: edge-rejection discrimination ─────────────────────────────── + # Distinguish an EDGE-429 (Anthropic's gateway bouncing the request before + # rate-limit accounting — an auth/fingerprint reject) from a REAL quota 429. + # The signature, proved by v0.8.8's header capture on Bryan's box: + # HTTP 429 + x-should-retry:true + ZERO anthropic-ratelimit-* headers + # A genuine quota 429 ALWAYS carries at least one anthropic-ratelimit-* header + # (the burst rail's -remaining:0, or the unified-* family). Its ABSENCE on a + # 429 means the request never reached the rate-limit accountant → edge bounce. + # We reset to 0 every parse so the flag never goes stale across calls; a 200 + # or a real-quota 429 leaves it 0 (→ normal backoff). Only the edge signature + # sets it 1 (→ agent_turn diverts to the API-key fallback instead of futile + # backoff against an edge that will never accept the OAuth token). + STATUS_rl_edge_reject=0 + if [ "$_is_429" = "1" ]; then + local _has_rl_hdr=0 + if grep -iqE '^anthropic-ratelimit-' "$f" 2>/dev/null; then + _has_rl_hdr=1 + fi + local _should_retry; _should_retry=$(strip_cr "$(_header_value "$f" "x-should-retry")") + # Edge reject = 429 with NO ratelimit headers. x-should-retry:true is the + # corroborating tell from Bryan's capture but we don't HARD-require it — the + # load-bearing discriminator is the absence of anthropic-ratelimit-* (a real + # quota 429 cannot omit them). If ratelimit headers ARE present it's a + # legitimate quota/burst 429 → leave the flag 0 regardless of x-should-retry. + if [ "$_has_rl_hdr" = "0" ]; then + STATUS_rl_edge_reject=1 + # Annotate the rail so _humanize_api_error / the 429-log banner name it. + [ -z "$STATUS_rl_tripped_rail" ] && STATUS_rl_tripped_rail="edge-reject" + fi + : "${_should_retry:=}" # referenced for clarity; absence is tolerated + fi + local log_dir="$LARRY_HOME/log" # ── ALWAYS-ON 429 CAPTURE (the whole point of headers.log) ─────────────── @@ -3463,6 +3796,20 @@ TOOLS_END # ───────────────────────────────────────────────────────────────────────────── # API call # ───────────────────────────────────────────────────────────────────────────── + +# _curl_config_apikey — emit a curl config snippet carrying the x-api-key +# header, to be piped to `curl --config -` on STDIN. This keeps the API key OUT +# of curl's argv (and therefore out of the process table / `ps` output) — a +# hardening over passing it as `-H "x-api-key: ..."`. curl config syntax: +# header = "x-api-key: " +# The value is CR-stripped (defense-in-depth; the stored key is already clean). +# Nothing here is logged; the snippet exists only on the pipe. +_curl_config_apikey() { + local k="${ANTHROPIC_API_KEY:-}" + k="${k//$'\r'/}" + printf 'header = "x-api-key: %s"\n' "$k" +} + call_api() { local payload_file="$1" local auth_args=() @@ -3497,9 +3844,23 @@ call_api() { return 1 fi [ -n "$oauth_stderr_file" ] && rm -f "$oauth_stderr_file" - auth_args=(-H "Authorization: Bearer $token" -H "anthropic-beta: oauth-2025-04-20") + # OAuth is OPT-IN only. We send the MINIMAL honest OAuth header set — a + # Bearer token + the oauth beta flag. We DO NOT impersonate the official + # Claude Code client (no claude-code-* beta flag, no claude-cli UA, no + # x-app:cli, no "You are Claude Code" system block). That impersonation is + # exactly what Anthropic fingerprints and blocks, and what flags the user's + # Max account. The one-time risk warning fires here. + _warn_oauth_optin_once + auth_args=( + -H "Authorization: Bearer $token" + -H "anthropic-beta: oauth-2025-04-20" + ) else - auth_args=(-H "x-api-key: $ANTHROPIC_API_KEY") + # DEFAULT / sanctioned rail: a plain programmatic API-key request. No + # Bearer, no impersonation headers, no Claude-Code system spoof. The + # x-api-key header is fed to curl via --config on STDIN (see _curl_config_*) + # so the key never appears in curl's argv / the process table. + auth_args=() fi # v0.6.9: dump response headers to a tempfile via -D so the status-line # tracker can parse anthropic-ratelimit-* fields after the call returns. @@ -3509,12 +3870,22 @@ call_api() { local _hdrs_file; _hdrs_file=$(mktemp 2>/dev/null || echo "") local _curl_args=( -sS --max-time 180 ) [ -n "$_hdrs_file" ] && _curl_args+=( -D "$_hdrs_file" ) - curl "${_curl_args[@]}" \ - "${auth_args[@]}" \ - -H "anthropic-version: 2023-06-01" \ - -H "content-type: application/json" \ - --data-binary "@$payload_file" \ - "$LARRY_API_URL" + if [ "$LARRY_AUTH_MODE" = "apikey" ]; then + # Key travels in the curl config on stdin, NOT in argv. + _curl_config_apikey | curl "${_curl_args[@]}" --config - \ + "${auth_args[@]}" \ + -H "anthropic-version: 2023-06-01" \ + -H "content-type: application/json" \ + --data-binary "@$payload_file" \ + "$LARRY_API_URL" + else + curl "${_curl_args[@]}" \ + "${auth_args[@]}" \ + -H "anthropic-version: 2023-06-01" \ + -H "content-type: application/json" \ + --data-binary "@$payload_file" \ + "$LARRY_API_URL" + fi local _curl_rc=$? # Parse headers regardless of whether the body parse will succeed; headers # carry rate-limit info even on 429s. @@ -3546,9 +3917,16 @@ call_api_stream() { err "OAuth token unavailable (streaming); run /login to re-authenticate" return 1 fi - auth_args=(-H "Authorization: Bearer $token" -H "anthropic-beta: oauth-2025-04-20") + # OAuth opt-in: minimal honest header set, kept in lockstep with call_api. + # NO Claude Code impersonation (see call_api for the rationale). + _warn_oauth_optin_once + auth_args=( + -H "Authorization: Bearer $token" + -H "anthropic-beta: oauth-2025-04-20" + ) else - auth_args=(-H "x-api-key: $ANTHROPIC_API_KEY") + # API-key header travels via --config on stdin (off argv); see call_api. + auth_args=() fi # v0.6.9: dump response headers via -D for status-line tracking. -D writes # the header block immediately when the server emits it, BEFORE the SSE body @@ -3565,13 +3943,23 @@ call_api_stream() { : > "$_hdrs_file" 2>/dev/null || _hdrs_file="" local _curl_args=( -sN --max-time 300 ) [ -n "$_hdrs_file" ] && _curl_args+=( -D "$_hdrs_file" ) - curl "${_curl_args[@]}" \ - "${auth_args[@]}" \ - -H "anthropic-version: 2023-06-01" \ - -H "content-type: application/json" \ - -H "accept: text/event-stream" \ - --data-binary "@$payload_file" \ - "$LARRY_API_URL" + if [ "$LARRY_AUTH_MODE" = "apikey" ]; then + _curl_config_apikey | curl "${_curl_args[@]}" --config - \ + "${auth_args[@]}" \ + -H "anthropic-version: 2023-06-01" \ + -H "content-type: application/json" \ + -H "accept: text/event-stream" \ + --data-binary "@$payload_file" \ + "$LARRY_API_URL" + else + curl "${_curl_args[@]}" \ + "${auth_args[@]}" \ + -H "anthropic-version: 2023-06-01" \ + -H "content-type: application/json" \ + -H "accept: text/event-stream" \ + --data-binary "@$payload_file" \ + "$LARRY_API_URL" + fi } # _drain_pending_stream_headers — called by the parent shell after a streaming @@ -4029,6 +4417,11 @@ agent_turn() { local payload_file; payload_file=$(mktemp) local stream_flag="false" [ "$LARRY_NO_STREAM" != "1" ] && stream_flag="true" + # The `system` field is a plain string in BOTH auth modes — larry's own + # persona prompt, nothing more. We do NOT prepend a "You are Claude Code, + # ..." identity block: that Claude-Code system spoof is part of the + # impersonation Anthropic blocks, and the API-key rail (the default) is a + # sanctioned programmatic request that needs no such block. jq -n \ --arg model "$LARRY_MODEL" \ --argjson max_tokens "$LARRY_MAX_TOKENS" \ @@ -4091,6 +4484,40 @@ agent_turn() { # to the burst the way the old immediate stream→non-stream re-send did. case "$err_type" in rate_limit_error|overloaded_error) + # ── 429-discrimination (reused from #13's good work) ───────────── + # A REAL rate-limit 429 ALWAYS carries anthropic-ratelimit-* headers + # → STATUS_rl_edge_reject stays 0 → legitimate backoff below. A 429 + # with NO such headers is an edge/auth bounce, not a quota limit. We + # branch on that distinction for clear, accurate messaging. + # + # API-KEY RAIL (the default): an edge-reject 429 here is NOT your + # quota — backing off won't help. Surface that plainly and stop. + if [ "$err_type" = "rate_limit_error" ] \ + && [ "$STATUS_rl_edge_reject" = "1" ] \ + && [ "$LARRY_AUTH_MODE" = "apikey" ]; then + err "429 with NO rate-limit headers on the API-key rail — an edge/transient bounce, not your quota." + err "If this persists, check console.anthropic.com (key active? billing enabled?) or retry shortly. (full headers captured in \$LARRY_HOME/log/headers.log)" + rm -f "$tools_file" "$system_file" + return 1 + fi + # OAUTH RAIL (opt-in only): an edge-reject 429 means Anthropic's edge + # bounced the OAuth token (the impersonation block / acceleration + # throttle). Backing off is futile — the edge will not accept it. If + # an API key is configured, flip to the sanctioned API-key rail (which + # works) for the rest of the session and retry immediately. One-shot + # per session (_LARRY_EDGE_FALLBACK_DONE). Honors LARRY_NO_EDGE_FALLBACK=1. + if [ "$err_type" = "rate_limit_error" ] \ + && [ "$STATUS_rl_edge_reject" = "1" ] \ + && [ "$LARRY_AUTH_MODE" = "oauth" ] \ + && [ "${LARRY_NO_EDGE_FALLBACK:-0}" != "1" ] \ + && [ "${_LARRY_EDGE_FALLBACK_DONE:-0}" != "1" ] \ + && [ -n "${ANTHROPIC_API_KEY:-}" ]; then + _LARRY_EDGE_FALLBACK_DONE=1 + LARRY_AUTH_MODE="apikey" + warn "edge rejected the OAuth token (429 with no rate-limit headers — an edge bounce, NOT your quota)." + warn "falling back to the sanctioned API-key rail for the rest of this session (set LARRY_NO_EDGE_FALLBACK=1 to disable)." + continue # rebuild payload (now API-key shaped) and retry on the key + fi _rl_attempts=$(( _rl_attempts + 1 )) local _max; _max=$(coerce_int "$LARRY_RL_MAX_RETRIES" 3) if [ "$_rl_attempts" -le "$_max" ]; then @@ -4101,6 +4528,14 @@ agent_turn() { sleep "$_wait" 2>/dev/null || true continue # rebuild payload and re-attempt (the ONLY retry path) fi + # v0.8.10: if we exhausted retries on an edge-reject and could NOT flip + # (no API key), say so explicitly — backoff was never going to help. + if [ "$STATUS_rl_edge_reject" = "1" ] && [ "$LARRY_AUTH_MODE" = "oauth" ]; then + err "OAuth edge-reject persisted after $_max retries and no API key is set to fall back to." + err "This 429 carries NO rate-limit headers — it is an edge bounce, not your quota. Set ANTHROPIC_API_KEY for an automatic fallback, or re-run /login." + rm -f "$tools_file" "$system_file" + return 1 + fi err "API error: $(_humanize_api_error "$resp") (gave up after $_max retries)" rm -f "$tools_file" "$system_file" return 1 @@ -4331,12 +4766,18 @@ Slash commands: /load load file contents as your next user message /sys print the active system prompt /env print detected Cloverleaf env (HCIROOT, HCISITE, tools) - /auth show OAuth status (or "not authenticated") - /login run OAuth login flow (switch from API-key to subscription auth) - /logout delete OAuth tokens (revert to API-key auth) - /oauth-debug dump full OAuth diagnostic (file state, parsed expiry, - jq path/flavor, cygpath translation, truncated tokens, - live ensure trace). Safe to copy-paste; secrets truncated. + /auth show the active auth rail + masked API-key status + /set-api-key set the per-client API key (silent input, validated, + stored 0600, CR-safe). --clear removes it; --status shows + it masked (sk-ant-api03-XXXX…last4). API key is the default, + sanctioned rail. Mint one per machine at console.anthropic.com. + /auth-debug masked auth diagnostic across both rails (NEVER prints a + full key/token). Alias: /api-debug. Safe to copy-paste. + /login OPT INTO OAuth (discouraged — Anthropic blocks Claude-Code + impersonation; risks your Max account). Prefer /set-api-key. + /logout delete OAuth tokens; revert to the default API-key rail + /oauth-debug dump OAuth diagnostic (tokens truncated). Use /auth-debug + for the masked, both-rails view. /lesson capture a lesson to local file (paste back to home-Larry later) /lessons list all captured lessons (newest first) /export dump the lesson bundle for paste-back to home-Larry @@ -4579,6 +5020,9 @@ _LARRY_SLASH_CMDS=( /pwd /env /auth + /set-api-key + /auth-debug + /api-debug /login /logout /oauth-debug @@ -4632,10 +5076,13 @@ _LARRY_SLASH_CMDS_DESC=( [/sys]="print the active system prompt" [/pwd]="show current working directory" [/env]="print detected Cloverleaf env (HCIROOT, HCISITE, tools)" - [/auth]="show OAuth status (or not authenticated)" - [/login]="run OAuth login flow (switch to subscription auth)" - [/logout]="delete OAuth tokens (revert to API-key auth)" - [/oauth-debug]="dump full OAuth diagnostic" + [/auth]="show auth rail + masked API-key status" + [/set-api-key]="set/clear/show the per-client API key (silent, 0600, validated)" + [/auth-debug]="masked auth diagnostic across both rails (never prints secrets)" + [/api-debug]="alias of /auth-debug" + [/login]="opt into OAuth (discouraged — risks Max account)" + [/logout]="delete OAuth tokens (revert to the default API-key rail)" + [/oauth-debug]="dump OAuth diagnostic (tokens truncated)" [/lesson]=" capture a lesson for paste-back to home-Larry" [/lessons]="list all captured lessons (newest first)" [/export]="dump the lesson bundle for paste-back" @@ -5670,10 +6117,61 @@ main_loop() { /sys) printf '%s\n' "$system_prompt"; continue ;; /pwd) echo "$(pwd)"; continue ;; /env) printf '%s\n' "$CLOVERLEAF_CTX"; continue ;; - /auth) if [ -x "$LARRY_LIB_DIR/oauth.sh" ]; then "$LARRY_LIB_DIR/oauth.sh" status; else echo "(oauth.sh not installed)"; fi; continue ;; - /login) if [ -x "$LARRY_LIB_DIR/oauth.sh" ]; then "$LARRY_LIB_DIR/oauth.sh" login && LARRY_AUTH_MODE="oauth" && larry_say "switched to OAuth subscription auth"; else err "oauth.sh not installed"; fi; continue ;; - /logout) if [ -x "$LARRY_LIB_DIR/oauth.sh" ]; then "$LARRY_LIB_DIR/oauth.sh" logout; LARRY_AUTH_MODE="apikey"; fi; continue ;; + /auth) printf '%sauth rail: %s (primary: %s)%s\n' "$C_BOLD" "$LARRY_AUTH_MODE" "$LARRY_PRIMARY_AUTH_MODE" "$C_RESET" + show_api_key_status + if [ "$LARRY_AUTH_MODE" = "oauth" ] && [ -x "$LARRY_LIB_DIR/oauth.sh" ]; then + "$LARRY_LIB_DIR/oauth.sh" status + fi + continue ;; + # /set-api-key [--clear|--status] — the secure per-client key provisioning + # entry point. Default action prompts (silent, validated, 0600, CR-safe). + /set-api-key*) + local _ska; _ska=$(_slash_args "/set-api-key" "$input") + _ska="${_ska//$'\r'/}"; _ska="${_ska%"${_ska##*[![:space:]]}"}" + case "$_ska" in + --clear) clear_api_key ;; + --status|"") + if [ "$_ska" = "--status" ]; then show_api_key_status + else set_api_key; fi ;; + --no-validate) set_api_key --no-validate ;; + *) err "usage: /set-api-key [--clear|--status|--no-validate]" ;; + esac + continue ;; + /login) if [ -x "$LARRY_LIB_DIR/oauth.sh" ]; then + warn "OAuth is opt-in and risks your Max account (Anthropic blocks Claude-Code impersonation). API key is the default rail — prefer /set-api-key." + "$LARRY_LIB_DIR/oauth.sh" login && LARRY_AUTH_MODE="oauth" && larry_say "switched to OAuth subscription auth (opt-in)" + else err "oauth.sh not installed"; fi; continue ;; + /logout) if [ -x "$LARRY_LIB_DIR/oauth.sh" ]; then "$LARRY_LIB_DIR/oauth.sh" logout; fi + # Revert to the default API-key rail if a key is available. + if [ -n "${ANTHROPIC_API_KEY:-}" ] || [ -f "$LARRY_API_KEY_FILE" ]; then + _load_api_key_into_env; LARRY_AUTH_MODE="apikey"; larry_say "reverted to the API-key rail" + else + LARRY_AUTH_MODE=""; warn "no API key set — run /set-api-key" + fi + continue ;; + # /auth-debug — masked auth diagnostic across BOTH rails. NEVER prints a + # full key or token. The API-key portion shows sk-ant-api03-XXXX…last4. + /auth-debug|/api-debug) + printf '%s=== auth diagnostic (secrets masked) ===%s\n' "$C_BOLD" "$C_RESET" + printf ' primary auth mode: %s\n' "$LARRY_PRIMARY_AUTH_MODE" + printf ' active auth mode: %s\n' "$LARRY_AUTH_MODE" + show_api_key_status + if [ -f "$LARRY_API_KEY_FILE" ]; then + local _akmode; _akmode=$(stat -f '%Lp' "$LARRY_API_KEY_FILE" 2>/dev/null || stat -c '%a' "$LARRY_API_KEY_FILE" 2>/dev/null || echo '?') + printf ' api-key file: %s (present, mode %s)\n' "$LARRY_API_KEY_FILE" "$_akmode" + else + printf ' api-key file: %s (absent)\n' "$LARRY_API_KEY_FILE" + fi + if [ -x "$LARRY_LIB_DIR/oauth.sh" ] && [ -f "$LARRY_HOME/.oauth.json" ]; then + printf '\n [opt-in OAuth state — tokens truncated by oauth.sh]\n' + "$LARRY_LIB_DIR/oauth.sh" debug 2>&1 | sed 's/^/ /' + else + printf ' oauth: (no .oauth.json; OAuth is opt-in/off)\n' + fi + printf '%s=== end auth diagnostic ===%s\n' "$C_BOLD" "$C_RESET" + continue ;; /oauth-debug) + # Retained for muscle memory; routes to the masked /auth-debug. if [ -x "$LARRY_LIB_DIR/oauth.sh" ]; then "$LARRY_LIB_DIR/oauth.sh" debug else diff --git a/lib/fetch-safe.sh b/lib/fetch-safe.sh index 5e4be0e..1dd6431 100644 --- a/lib/fetch-safe.sh +++ b/lib/fetch-safe.sh @@ -141,7 +141,12 @@ fetch_validate() { printf 'error: %s — MANIFEST contains HTML markup ("<"), not a path list.\n' "$url" >&2 return 1 fi - if ! grep -Eq '^[A-Za-z0-9_][A-Za-z0-9_./-]*$' "$tmp"; then + # v0.8.11: a plausible line is a path token, OPTIONALLY followed by + # whitespace + a 64-hex sha256 (the new "pathsha256" format). The + # legacy paths-only form still matches (the hash group is optional). The + # '<' guard above remains the real HTML-trap defense; this just confirms + # the body looks like a manifest, not random text. + if ! grep -Eq '^[A-Za-z0-9_][A-Za-z0-9_./-]*([[:space:]]+[0-9a-fA-F]{64})?[[:space:]]*$' "$tmp"; then rm -f "$tmp" printf 'error: %s — MANIFEST has no plausible path line.\n' "$url" >&2 return 1 diff --git a/scripts/hooks/pre-commit b/scripts/hooks/pre-commit new file mode 100755 index 0000000..bf74208 --- /dev/null +++ b/scripts/hooks/pre-commit @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# pre-commit hook — block a commit whose MANIFEST sha256 hashes have drifted +# from the working tree. Ensures every release ships a MANIFEST whose published +# hashes match the bytes being pushed, so the auto-updater's local-skip logic +# (larry.sh sync_from_manifest) never compares against a stale hash. +# +# Install: scripts/make-manifest.sh --install-hook +# +# The hook is intentionally NON-FATAL when sha256 tooling is unavailable on the +# committer's box (it can't verify, so it warns and allows) — the same fail-safe +# philosophy the client uses. It only BLOCKS when it can prove drift. + +root="$(git rev-parse --show-toplevel 2>/dev/null)" || exit 0 +gen="$root/scripts/make-manifest.sh" +[ -x "$gen" ] || exit 0 # generator absent (older checkout) — don't block + +# Only enforce when MANIFEST or a manifested file is part of this commit, to +# avoid hashing on every unrelated commit. Cheap heuristic: always check; the +# generator is fast. +out="$("$gen" --check 2>&1)"; rc=$? +case "$rc" in + 0) exit 0 ;; + 1) + echo "" >&2 + echo "pre-commit: MANIFEST hashes are out of date." >&2 + echo "$out" >&2 + echo "" >&2 + echo "Fix: run scripts/make-manifest.sh then git add MANIFEST and re-commit." >&2 + exit 1 + ;; + 2|*) + # Generation error (e.g. no sha256 tool on this box). Can't verify -> warn, + # but don't block the commit. CI / the release box still enforces. + echo "pre-commit: WARN — could not verify MANIFEST hashes ($out). Allowing commit." >&2 + exit 0 + ;; +esac diff --git a/scripts/make-manifest.sh b/scripts/make-manifest.sh new file mode 100755 index 0000000..791a954 --- /dev/null +++ b/scripts/make-manifest.sh @@ -0,0 +1,184 @@ +#!/usr/bin/env bash +# make-manifest.sh — regenerate larry-anywhere's MANIFEST with authoritative +# sha256 hashes for every shipped file. +# +# WHY (v0.8.11 — manifest-hashing speedup, Clover #14): +# The auto-updater (larry.sh sync_from_manifest) used to re-download ALL ~48 +# manifest files over authenticated HTTPS every relaunch, then `cmp` locally +# to find the few that changed and discard the rest — ~3 min for a 3-file +# update. By publishing each file's expected sha256 IN the MANIFEST, the +# client fetches MANIFEST ONCE, hashes its local copies, and downloads only +# the files whose hash differs (or are missing). N round-trips -> 1 + a local +# hash pass + N real downloads. +# +# For that to be correct, the published hashes MUST be authoritative: this +# script recomputes them at release time. The MANIFEST hash always wins — +# local != manifest => download. A stale/missing/malformed hash can therefore +# never SKIP a real update (the client falls back to downloading on any doubt; +# see sync_from_manifest's fail-safe matrix), but it CAN cause a needless +# re-download — so we keep the hashes correct by regenerating here. +# +# FORMAT it emits (one entry per line, after the comment header): +# +# Comment (#...) and blank lines are preserved verbatim from the source list so +# the human-readable section structure survives. Only path lines gain a hash. +# +# USAGE: +# scripts/make-manifest.sh # rewrite ./MANIFEST in place +# scripts/make-manifest.sh --check # verify MANIFEST is up to date (CI/hook) +# # exit 0 = in sync, 1 = drift, 2 = error +# scripts/make-manifest.sh --stdout # print the regenerated MANIFEST, no write +# +# RELEASE DISCIPLINE — every release MUST regenerate MANIFEST so the hashes +# match the bytes being pushed. This is wired so it can't be forgotten: +# 1. A pre-commit hook (scripts/hooks/pre-commit) runs `--check` and BLOCKS a +# commit whose MANIFEST hashes drift from the working tree. Install with +# `scripts/make-manifest.sh --install-hook`. +# 2. The CHANGELOG release checklist references this script. +# Run from the bundle root (the script cd's there regardless of invocation dir). + +set -o pipefail + +# ── Resolve bundle root (parent of this scripts/ dir) ───────────────────────── +_self="${BASH_SOURCE[0]:-$0}" +_self_dir="$(cd "$(dirname "$_self")" 2>/dev/null && pwd)" +ROOT="$(cd "$_self_dir/.." 2>/dev/null && pwd)" +[ -n "$ROOT" ] || { echo "error: cannot resolve bundle root" >&2; exit 2; } + +MANIFEST="$ROOT/MANIFEST" + +# ── sha256 tool detection — same fallback chain larry.sh uses, kept in sync ── +# Priority: sha256sum, shasum -a 256, openssl dgst -sha256. We normalize each +# tool's differing output to a bare lowercase 64-hex string. +_SHA_TOOL="" +_detect_sha_tool() { + [ -n "$_SHA_TOOL" ] && return 0 + if command -v sha256sum >/dev/null 2>&1; then _SHA_TOOL="sha256sum"; return 0; fi + if command -v shasum >/dev/null 2>&1; then _SHA_TOOL="shasum"; return 0; fi + if command -v openssl >/dev/null 2>&1; then _SHA_TOOL="openssl"; return 0; fi + return 1 +} + +# _sha256_of FILE — print the bare 64-hex sha256 of FILE, or empty on failure. +_sha256_of() { + local f="$1" out="" + case "$_SHA_TOOL" in + sha256sum) out="$(sha256sum "$f" 2>/dev/null)" ;; + shasum) out="$(shasum -a 256 "$f" 2>/dev/null)" ;; + openssl) out="$(openssl dgst -sha256 "$f" 2>/dev/null)" ;; + *) return 1 ;; + esac + # Extract the first 64-hex run from whatever shape the tool emitted: + # sha256sum/shasum -> " " + # openssl -> "SHA2-256()= " + out="$(printf '%s' "$out" | tr -d '\r' | grep -oE '[0-9a-fA-F]{64}' | head -1 | tr 'A-F' 'a-f')" + [ -n "$out" ] || return 1 + printf '%s' "$out" +} + +# ── Generate the hashed MANIFEST text to stdout ────────────────────────────── +# Source of truth for WHICH files ship: the existing MANIFEST's path lines plus +# its comment/section structure. We re-read the current MANIFEST, preserve every +# comment/blank line verbatim, and (re)hash every path line. A path line is the +# first whitespace-delimited token; any pre-existing hash is discarded and +# recomputed (so re-running is idempotent and always authoritative). +_generate() { + _detect_sha_tool || { echo "error: no sha256 tool (sha256sum / shasum / openssl) found" >&2; return 2; } + [ -f "$MANIFEST" ] || { echo "error: $MANIFEST not found (need the path list to hash)" >&2; return 2; } + + local line path hash rc=0 + while IFS= read -r line || [ -n "$line" ]; do + # Strip any CR so a CRLF-tainted source MANIFEST doesn't poison output. + line="${line//$'\r'/}" + case "$line" in + ''|'#'*) + # Preserve comments and blank lines verbatim. + printf '%s\n' "$line" + continue + ;; + esac + # First whitespace-delimited token is the path; drop any existing hash. + path="${line%%[[:space:]]*}" + [ -z "$path" ] && { printf '%s\n' "$line"; continue; } + if [ ! -f "$ROOT/$path" ]; then + echo "error: listed file missing on disk: $path" >&2 + rc=2 + # Emit the path with NO hash. The client treats a hashless line as + # "can't verify -> download" (fail-safe), so a missing file degrades to + # the old behaviour for that one entry rather than publishing a wrong hash. + printf '%s\n' "$path" + continue + fi + hash="$(_sha256_of "$ROOT/$path")" + if [ -z "$hash" ]; then + echo "error: could not hash $path" >&2 + rc=2 + printf '%s\n' "$path" + continue + fi + printf '%s\t%s\n' "$path" "$hash" + done < "$MANIFEST" + return $rc +} + +# ── Entry points ────────────────────────────────────────────────────────────── +_install_hook() { + local hook_src="$ROOT/scripts/hooks/pre-commit" + local git_dir + git_dir="$(cd "$ROOT" && git rev-parse --git-dir 2>/dev/null)" || { + echo "error: not a git repo at $ROOT" >&2; return 2; } + [ -f "$hook_src" ] || { echo "error: $hook_src missing" >&2; return 2; } + local dest="$git_dir/hooks/pre-commit" + cp "$hook_src" "$dest" && chmod +x "$dest" && echo "installed pre-commit hook -> $dest" +} + +case "${1:-}" in + --stdout) + _generate + exit $? + ;; + --check) + # Compare freshly generated text against the on-disk MANIFEST. + tmp="$(mktemp)" || { echo "error: mktemp failed" >&2; exit 2; } + if ! _generate > "$tmp"; then + rc=$? + rm -f "$tmp" + echo "make-manifest --check: generation error (rc=$rc)" >&2 + exit "$rc" + fi + if cmp -s "$tmp" "$MANIFEST"; then + rm -f "$tmp" + echo "MANIFEST is up to date." + exit 0 + fi + echo "MANIFEST is OUT OF DATE — run scripts/make-manifest.sh to regenerate." >&2 + echo "drift (diff: current MANIFEST vs freshly hashed):" >&2 + diff "$MANIFEST" "$tmp" >&2 || true + rm -f "$tmp" + exit 1 + ;; + --install-hook) + _install_hook + exit $? + ;; + --help|-h) + sed -n '2,40p' "$0" + exit 0 + ;; + '') + # Default: rewrite MANIFEST in place (write to temp, then move atomically). + tmp="$(mktemp)" || { echo "error: mktemp failed" >&2; exit 2; } + if ! _generate > "$tmp"; then + rc=$? + rm -f "$tmp" + echo "make-manifest: generation failed (rc=$rc); MANIFEST left unchanged" >&2 + exit "$rc" + fi + mv "$tmp" "$MANIFEST" && echo "MANIFEST regenerated with sha256 hashes ($(grep -cE ' [0-9a-f]{64}$' "$MANIFEST") entries hashed)." + exit 0 + ;; + *) + echo "error: unknown argument: $1 (try --help)" >&2 + exit 2 + ;; +esac