Wires nc_status, nc_engine, nc_xlate, nc_smat_diff, nc_tclgen as first-class
LLM tools (all 4 surfaces). nc_engine unlocks TPS testing (hcitps) + the
route-test driver. Fixes a real nc-engine.sh bug surfaced by the exposure:
the dispatcher treated every --flag as taking a value (--dry-run ate the next
token) and a set -u leak from journal.sh crashed start/stop/bounce on bash 3.2;
fixed with set +u + a multi-case parser (no over-shift on bare trailing flags).
Corrects stale CHANGELOG + nc_engine schema text that misstated the bug as live.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New tool: show how a thread/system/site changed over time by diffing
Cloverleaf NetConfig revision snapshots, annotated with who saved each and
when. Handles the non-zero-padded NetConfig<TS> revision dirs by parsing the
prologue date into a sortable key; scopes diffs to the requested thread/system
via nc-parse. Flags: <thread>[.<site>], --system, --site, --format
timeline|diff, --limit, --since. Wired as nc_revisions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Shared _sanitize_ctl (unconditional, nc-document) and _sanitize_ctl_tty
(strips only when stdout is a terminal) now live in cygwin-safe.sh. nc-msgs,
nc-parse, and the hl7-* tools route stdout through the tty-gated variant, so a
terminal is protected from raw HL7/NetConfig control bytes while pipes and
redirects stay byte-exact (the 0x1c framing route_test needs is preserved).
Exit codes propagate via PIPESTATUS. ssh-helper _read_hidden installs its
restore trap before stty -echo on every path and saves/restores the prior trap.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
nc-document.sh now sanitizes C0 control bytes (except tab/LF/CR) at the single
output sink, so raw ESC sequences embedded in NetConfig/.tcl content can no
longer flip the terminal's mode and break line editing when output isn't
redirected. ssh-helper.sh password prompts save/restore termios via stty -g +
trap so a ^C mid-prompt no longer leaves echo off. UTF-8 preserved; portable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Default render is now plain text (no #/**/pipe-tables/fences) so it pastes
cleanly into OneNote. Tabular sections default to label:value hop blocks;
--onenote-table emits tab-separated rows for paste -> Insert>Table. Raw TCL
moved behind opt-in --raw-tcl (readable UPOC bits stay inline). Removed the
verbose "Filter / translation logic (surfaced deterministically...)" label.
Fixes a markdown leak in the proc-not-found fallback (Vera gate). Folds in
two prior deferred minors (dead counters; local _dest_hit/_d).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Resolves the downstream route chain via nc-paths, grabs N recent messages
from the START inbound's SMAT, walks each ENTRY node (START + post-==> remote
inbounds) running hciroutetest -a -d -f nl, chaining each step's selected
.out.<DEST> across cross-site hops. Generates per-chain commands.sh for the
engine box; --dry-run stubs the engine. Command syntax mined verbatim from
the v1/v2 route_test wrappers. Fixes --help sed range (header ends at 94).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Single-line `local a=… b="…$a…"` referenced the 1st var before it was bound
within the SAME `local` statement, aborting under set -u on Cygwin/MobaXterm
(and modern bash). Split larry.sh:6903 (/ssh-set-hciroot) and the same latent
pattern at larry.sh:6925 (/ssh) into set-u-safe declare-then-assign form.
Codebase-wide bug-class audit (larry.sh + all lib/*.sh + scripts): zero
remaining instances. Closed the v0.8.15 gate gap by driving the ACTUAL REPL
slash-dispatch handler bodies under set -u + BASH_COMPAT=3.2 (not just the
ssh-helper subcommand): /ssh-set-hciroot normal + empty-path-clear, /ssh, and
usage paths all pass; old code aborts under the same harness. No-traffic-bypass
line unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
MAJOR-1: regenerate MANIFEST (larry.sh, lib/ssh-helper.sh, VERSION,
CHANGELOG.md hashes now authoritative for the v0.8.15 bytes).
MINOR-1: print_help /sites line documents the --hciroot <path> pin
convenience and the pinned-vs-login resolution distinction.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
larry tools list / <name> [args] makes all 24 lib/ Cloverleaf+HL7 tools
discoverable and runnable by hand with no API/LLM; dispatches before
bootstrap/self-update/network. _diagnose_api_block recognizes a blocked
API (curl rc/stderr/body/headers, incl. Cisco Umbrella fingerprints) and
guides the operator to manual-tools mode + IT allowlisting instead of a
raw error dump. Graceful degradation + honest guidance only — NO traffic
masking/proxy-hiding/circumvention on a PHI box.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root-cause fix for the live-session friction where "how many sites are on
qa?" stalled on repeated `export $HCIROOT` nags despite a working `qa` SSH
alias:
1. $HCIROOT login-shell fix: ssh-helper.sh `exec` now wraps remote commands in
`bash -lc` so the Cloverleaf login profile sources and $HCIROOT/$HCISITE/PATH
populate as for an interactive operator login. Escape hatch: NOLOGIN prefix
or LARRY_SSH_NO_LOGIN=1. pull-smat find/sample use the same wrapper.
2. Both-mode detection: startup surfaces a MODE= line (LOCAL / REMOTE / UNKNOWN)
and leads with what it found instead of asking for paths.
3. First-class list_sites tool + /sites [alias]: enumerates sites in both modes
(hcisitelist fast-path, NetConfig-walk fallback) via new ssh-helper discover.
4. System-prompt de-nagging: agents/larry.md + env-diff/regression prompts no
longer tell Larry to ask Bryan to export $HCIROOT for a reachable host.
5. Streaming slowness (dominant residual): new pure-bash _json_str_decode
un-escapes the common escape-free delta with zero forks, halving per-turn
jq forks on top of v0.8.12. Round-trip verified.
6. pull-smat path capture hardened (Vera Minor #1): resolved path now emitted
behind a SMATDB_PATH: sentinel and selected by pattern not position, so a
login-shell MOTD/banner on stdout can't be mistaken for the path; falls back
to prior tail -1 when no sentinel present. Selection logic unit-verified.
Vera gate: PASS-WITH-NOTES (v0.8.13). bash -n clean on larry.sh + ssh-helper.sh;
MANIFEST regenerated (48 entries) and --check clean.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Crash/slowness/cost pass on the API-key rail (Vera-QA-passed). Full diagnosis:
Deliverables/2026-05-27-cloverleaf-larry-v0812-crash-slowness-cost.md.
- Crash fix: CR-coerce at 3 response-path accounting sites (non-streaming cost,
streaming cost, _record_ctx_used) via coerce_int promoted to lib/cygwin-safe.sh
— guards CRLF-tainted jq usage counts on Cygwin/MobaXterm (v0.7.5 Anomaly-#4
recurrence on the response path).
- Slowness: collapsed per-delta jq forks in the SSE hot path (3 -> 2 forks/event
on text, 1 on ignored deltas) — dominant fix for the laggy per-turn render on
Windows fork emulation. Plus per-session PHI sidecar /health probe caching.
- Cost: prompt caching wired in — system sent as block array + last tool marked
cache_control: ephemeral, billing the ~12.7K static prefix at the cache-read
rate after turn 1 (~90% prefix cut, lower TTFB).
- PHI tier-5 notice now default-silent (LARRY_PHI_NOTICE=1 to re-enable).
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Root cause: sync_from_manifest fully downloads all 48 manifest entries
sequentially (authenticated HTTPS via proxy + Cloudflare), then cmp-compares
locally to find the few that changed — 48 silent round-trips, ~3 min, no output.
Add _sync_progress/_sync_progress_done: live in-place "checking N/48 <file>"
(switching to "downloading N/48 <file>" on real changes) via \r\033[K only —
MobaXterm-safe (no scroll-region/cursor-save/abs-pos). Gates on [ -t 2 ];
non-TTY emits a plain heartbeat every 10 files (no \r). Current filename shown
so a hang is visible by name; per-file curl --max-time bounds each stall.
Hash-skip speedup deferred: MANIFEST is paths-only (no hashes), so local
skip-unchanged needs a manifest-format + release-tooling change — filed for
v0.9.x. Sync correctness unchanged.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Bryan's MobaXterm work-box 429s never wrote headers.log because the v0.8.5
gate only fired on (OAuth + unified-*) OR retry-after — and his bare burst
429s carry neither. Detect 429 from the HTTP status line in the -D dump and
ALWAYS write the full raw header block, exempt from the OAuth 50-call cap
(own STATUS_429_HEADER_LOG_LIMIT budget), with a live phi/rl> stderr pointer.
Non-stream path already reached the parser (call_api -D dump); the bug was
the write-gate, not the call. Streaming path shares the same function.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Root cause: render_status_line suppressed the OAuth line whenever ctx_used,
5h_util, and 7d_util were ALL empty. On a rate-limited session ctx is never
recorded (the error path returns before _record_ctx_used) and pre-v0.8.5 the
unified-* headers weren't captured on errors — so all three stayed empty turn
after turn and the line never appeared on Bryan's work-box. NOT a positioning
bug: the line is a plain printf'd dim line (no scroll-region/cursor escapes)
and is not coupled to streaming or mouse mode.
Fix: suppress only before the first turn (_LARRY_TURNS==0); thereafter always
render — empty fields show "—" placeholders, reset date fills in once headers
populate. /status now renders on demand even pre-first-turn. CR-taint sweep:
coerce_int the reset-epoch arithmetic comparisons + strip_cr the oauth-status
color case (MobaXterm CRLF would otherwise crash/blank the line).
Verify: bash -n clean; 7/7 unit tests (turn-0 suppressed, turn>=1 placeholders,
reset date when populated, renders with LARRY_NO_STREAM=1 + mouse off, survives
CR-tainted epoch, LARRY_NO_STATUS=1 still disables).
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Closes the last gap in the rate-limit-diagnosis pipeline: anthropic-ratelimit-*
headers captured on the MobaXterm work-box now flow to the Mac memory daemon
(Tier 4 Hindsight + Tier 7 mem0) automatically.
- lib/headers-sync.sh: incremental, offset-tracked, idempotent push of
headers.log to ~/.cloverleaf/headers-<hostname>.jsonl on the Mac, riding the
existing authenticated SSH ControlMaster. No new auth; password never in
argv/env. No-op when nothing new; re-seed on local rotation/shrink. Fully
graceful (no target / closed master / transport error → warn + continue;
never crashes the session).
- /headers-sync on|off|status|target <alias>|now slash command + TAB-completion
+ /help. Config persisted to $LARRY_HOME/.env. Auto-sync fires on REPL exit.
- Security: headers.log carries only anthropic-* headers + status lines — NO
PHI per Vera audit V7; transport reused unchanged (not weakened).
Layered cleanly on top of Clover #8's v0.8.5 (4f1ea86) — edits isolated to new
lib + help/array/trap/dispatch hunks; no overlap with the streaming parser,
retry/backoff, error-display, or phi-notice regions.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Diagnose-don't-assume rate-limit cluster (Clover #8). The rate_limit_error on a
work-box with 90% of the 5h Max quota free was a short-window BURST rail, not 5h
exhaustion — tripped by a stream->non-stream double-send per turn with no backoff.
- Rate-limit backoff honoring retry-after (else exp 2/4/8 cap 30) + actionable
header-parsed message naming the tripped rail; headers.log now captures every
429 (was OAuth+unified-* only), tagged with retry-after + rail.
- parse_stream_to_response detects a non-SSE JSON error body (429/overload) and
returns a distinct code so agent_turn surfaces it WITH backoff instead of
re-sending the whole prompt (single-send invariant). Auto LARRY_NO_STREAM=1 on
MobaXterm/Cygwin/MSYS; explicit LARRY_NO_STREAM=0 still forces streaming on.
- ErrorPI fix: strip_cr on err_type/err_msg in _humanize_api_error (a trailing
CR broke the case match AND carriage-overprinted "API error"); err/warn/log
now strip embedded CRs defensively. (v0.7.5 sweep missed the error-display path.)
- phi tier-5 notice once-per-session via $LARRY_HOME/.phi-notice-shown SESSION_ID
flag (old export flag died in the $(...) subshell -> per-turn nag). Same-pattern
sweep fixed the identical subshell-flag bug in _auto_phi_b64_roundtrip.
Deliverable: Deliverables/2026-05-27-cloverleaf-larry-v085-ratelimit-streaming-fixes.md
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Hardens the installer + auto-updater against the Gitea private-repo trap
(Clover #5 diagnosis): an unauthenticated raw-file read of a sign-in-gated
Gitea returns the HTML Sign-In page at HTTP 200, which `curl -fsSL` treats as
success — so the old code parsed HTML as VERSION/MANIFEST/larry.sh content and
silently aborted (or overwrote real files with HTML). This stranded a work-box
at v0.7.3 until the REQUIRE_SIGNIN_VIEW=false flip.
- New lib/fetch-safe.sh: fetch_validate URL DEST KIND [MAX_TIME]. Detects the
HTML-login trap (DOCTYPE/<html/"Sign In - Gitea"/<title>Sign In markers, or
text/html Content-Type) and validates content shape per file type (semver
VERSION, path-list MANIFEST, shebang larry.sh, non-HTML .sh). On failure:
actionable error + non-zero, target file left untouched.
- install-larry.sh (curl|bash bootstrap) and larry.sh self_update() each carry
a byte-identical inline copy (both run before lib/ can be sourced).
- Every remote-content fetch routed through the validator: install fetch();
agent fetch; sync_from_manifest MANIFEST + per-file; _fetch_with_fallback.
- Optional LARRY_GITEA_TOKEN / GITEA_TOKEN env var adds Authorization: token
<PAT> for authenticated fetch against private repos. Never hardcoded/logged.
Documented in --help + MANUAL.md.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
The slash-command completer (__larry_complete_slash) intentionally appends
a trailing space after a unique match for arg-command ergonomics, but the
main_loop dispatcher matched exact `case` globs — so a completed `/quit `
missed the `/quit)` arm and fell through to "unknown command". Latent since
v0.6.6 (tab completion). Fixed by rtrimming the dispatch key once at the
`case "$input"` boundary, which also transitively protects the sub-command
dispatchers (/origin, /phi-auto, /phi-sidecar, /mouse) that consume the
same $input via _slash_args. Interior `/load FILE` spacing is preserved.
Added a shared rtrim() helper to lib/cygwin-safe.sh next to strip_cr.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
The only path that closes V1 (free-text PHI gap — the dominant real-world
failure mode per Vera). Opt-in install; larry runs in v0.8.1 mode on hosts
without Presidio (MobaXterm/Cygwin per Bryan's accepted tradeoff).
New files:
- lib/phi-presidio-sidecar.py — FastAPI service on 127.0.0.1:$LARRY_PHI_PORT
(default 41189). Presidio AnalyzerEngine + AnonymizerEngine over spaCy
en_core_web_sm + 3 HL7-specific custom recognizers (HL7_MRN, HL7_CARET_NAME,
HL7_PHONE_BARE). POST /redact and GET /health.
- lib/phi-sidecar.sh — lifecycle (start/stop/status/health/ensure). ensure
is idempotent; called backgrounded from main_loop so it never blocks the
first prompt. Honors LARRY_PHI_VENV.
- lib/phi-client.sh — bash client (phi_client_available / phi_redact_text /
phi_redact_entities). CR-safe; 5s timeout bounds tier-5 stall.
larry.sh:
- auto_detect_phi gains tier-5: after tiers 1-4, before status summary,
source phi-client.sh, run Presidio on a token-masked copy of the input,
tokenize each entity through hl7-sanitize.sh tokenize-value (category
presidio_<TYPE>) so token IDs stay stable. Honors confirm + strict modes.
Removed the v0.7.3 early-return that skipped past tier-5 when tiers 1-4
found nothing — pure prose now always reaches tier-5.
- Token-safe substitution: existing [[...]] tokens are pulled to sentinels,
tier-5 value is replaced, sentinels restored — prevents the token-within-
token corruption that naive literal-replace caused on already-tokenized
text. Acronym guard drops HL7/clinical jargon (SSN/MRN/DOB/ADT) Presidio
over-tags as ORGANIZATION.
- Graceful degradation: sidecar unreachable → tier-5 no-ops with a one-time
stderr warning. /phi-sidecar slash command + completion table.
install-larry.sh:
- Probes python3 3.9+; offers to create $LARRY_HOME/phi-venv and install
presidio + fastapi + uvicorn + en_core_web_sm. Skips silently (with a
v0.8.1-mode note) on Cygwin/MobaXterm without python3, and on
non-interactive pipe installs. Sets LARRY_PHI_VENV in the larry shim.
MANIFEST: three new lib files added for auto-sync.
Prototype validation (Bryan's Mac, Apple Silicon, Python 3.14):
cold start (en_core_web_sm): ~9s (vs ~82s if Presidio auto-grabs _lg;
we pin _sm for the REPL budget)
warm analyzer latency: P50 20.6ms / P95 22.7ms
end-to-end HTTP round-trip: ~57ms warm; ~150ms first-post-startup
All comfortably under the 200ms-per-turn budget.
MobaXterm verdict: v0.8.2 is Mac/Linux-only. MobaXterm stays on v0.8.1 +
nudges, per Bryan's explicit acceptance. install-larry.sh enforces this
by platform detection; larry.sh tier-5 silently no-ops when the sidecar
is absent (which IS the MobaXterm path — no code is platform-gated).
Verification: bash -n clean on larry.sh + all 3 new lib scripts; python3
ast.parse clean on the sidecar; end-to-end tier-5 tested live against the
sidecar (pure prose, rule-pack+tier-5 combined with no token corruption,
!nophi bypass); strict-mode fail-closed abort tested; CR-taint, path-block,
and base64 round-trip batteries re-run green.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Three changes expanding PHI safety envelope on the tool-result surface.
Closes V2 + V12 + V2-sub from Vera's audit. No behavior change for users
not interacting with HL7-shaped data.
- Tool-name allow-list dropped. The v0.7.3 tool-result auto-PHI gate ran
only on read_file (.hl7|.txt), nc_msgs, hl7_field, hl7_diff. v0.8.1
runs _auto_phi_looks_like_hl7 on EVERY tool result. On hit → route
through lib/hl7-sanitize.sh. On miss → pass through unchanged.
Closes V2: bash_exec / ssh_exec / grep_files / read_file of any
extension all get scanned when their output is HL7-shaped. False-
positive cost is negligible (extra regex pass on non-HL7 has zero
behavioral impact).
- Base64-wrapped HL7 round-trip. New _auto_phi_b64_roundtrip helper.
Detects candidate base64 runs (length >= 200, [A-Za-z0-9+/=] only,
length divisible by 4 — NOT entropy-based per Pax §V2-sub: HL7's
repetitive prefixes survive base64 with LOW entropy, so entropy is
the wrong signal). Speculatively decodes each candidate; if decoded
bytes look like HL7, routes through hl7-sanitize.sh and re-encodes
back into the result. Catches ssh_pull_smat sampled mode's TSV
format. Requires python3 (installed everywhere larry runs); skipped
with a one-time stderr warning when unavailable. Server-side TSV
encoding kept (binary-safe transport); client-side unwrap handles
the safety concern, no remote refactor needed.
- Operator review gate for bash_exec/ssh_exec/ssh_pull/ssh_pull_smat
results. When the tool produced HL7-shaped output OR the result
exceeds LARRY_TOOL_RESULT_REVIEW_THRESHOLD bytes (default 8192),
Larry prompts [Y/n/i] before passing the result back to the model.
'i' opens the full output in $PAGER then re-prompts. Default Y
(zero friction). N substitutes a refusal JSON so the model surfaces
that something was withheld. Skipped when LARRY_AUTO_PHI=off (opt-out
consistency) OR no TTY (headless scripts unaffected). Override with
LARRY_TOOL_RESULT_REVIEW=always for paranoid mode. Closes V12.
Proactive same-pattern sweep. Searched for other call sites where tool
output bypasses content-shape gating: only the one in agent_turn. The
v0.8.0-c strict-mode tool-result branch was updated in lockstep so it
now triggers on the broader (content-only) eligibility.
Verification: bash -n clean; b64 round-trip unit-tested with three
cases (real-world HL7 base64 → decoded contains tokenized PHI not
clear-text PHI; plain text → passthrough; non-HL7 b64 → passthrough,
no false positive).
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
Three independent zero-risk patches closing V3/V4/V5/V6/V11 gaps from
Vera's static PHI-leak audit. Implemented per Pax's mitigation
recommendations. No new deps, no behavior change for users not handling PHI.
- tool_read_file / tool_grep_files / tool_glob_files / tool_list_dir now
refuse paths under $LARRY_HOME/{log,sanitize,sessions} and
$LARRY_HOME/{.oauth.json,.env} with a structured JSON error the model
must surface. Block-list evaluates at call time; comparison runs against
both the literal and realpath-canonicalized form of both PATH and
$LARRY_HOME. Closes V4 + V6 + V11 (de-sanitization key, OAuth tokens,
PHI clear-text audit log). The proactive same-pattern sweep extended
the block from read_file alone to grep_files/glob_files/list_dir.
- /load <file> pre-routes HL7-shaped content through lib/hl7-sanitize.sh
(segment-aware tokenizer) BEFORE the user_input auto-PHI pass. Closes
V3 — smat dumps loaded via /load no longer rely on the lighter per-word
classifier.
- LARRY_AUTO_PHI=strict (fourth value alongside off/on/confirm) is the
fail-closed mode. Aborts the turn when sanitizer is missing or returns
empty on HL7-shaped content, or when tokenize-value fails. On the
tool-result surface (can't kill an in-flight tool_use), substitutes
the result with a refusal sentinel so raw HL7 NEVER reaches the model.
Existing off/on/confirm semantics unchanged. /phi-auto strict toggle,
/help text, and tests updated. Closes V5.
Refs:
Deliverables/2026-05-27-cloverleaf-larry-phi-leak-audit.md (Vera)
Deliverables/2026-05-27-cloverleaf-larry-phi-mitigation-research.md (Pax)
Verification: bash -n clean; path-block unit-tested with 13 cases including
symlink resolution (file and dir), ../ traversal, nonexistent paths, and
the empty-LARRY_HOME edge case — all pass.
Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
The v0.7.2 GitHub fallback is now functionally broken: the GitHub mirror
is being made private, so anonymous raw fetches return 401/403. Rather
than ship a silent-failure path to a dead URL, remove the fallback
entirely.
Changes:
larry.sh
- LARRY_BASE_URL_FALLBACK / LARRY_ORIGIN_DEFAULT_GITHUB removed
- sync_from_manifest_with_fallback and _fetch_with_fallback retained
by name (call-site compat) but are now single-source wrappers; on
origin unreachable they warn "auto-update skipped this launch" and
proceed with locally cached files (no crash)
- status-line _origin_badge collapses to "" (default) or "custom"
(user-pinned HTTPS URL); legacy github / fallback badges gone
- /origin slash command simplified:
/origin show effective origin + pin file
/origin gitea pin to the default Gitea URL
/origin auto clear the pin
/origin <https-url> pin to an arbitrary HTTPS mirror
/origin github returns a clear error (mirror is private)
- /help text updated to reflect single-source model
- LARRY_VERSION 0.7.3 -> 0.7.4
install-larry.sh
- LARRY_BASE_URL_FALLBACK removed; single-origin install path
- fetch() dies with a clear error when origin unreachable:
"install failed: cannot reach LARRY_BASE_URL=... — verify the URL
or set LARRY_BASE_URL to a reachable mirror"
- post-install fallback-warning block removed
VERSION: 0.7.3 -> 0.7.4
Migration: stale $LARRY_HOME/.origin files containing the legacy keyword
"github" are treated as invalid — Larry warns once at startup and reverts
to the default Gitea origin. We deliberately do NOT auto-rewrite the
file (so the user can choose) and do NOT translate it to the GitHub raw
URL (which would just 401 on the next fetch).
Verification:
- bash -n larry.sh / install-larry.sh: pass
- larry --version: prints 0.7.4
- LARRY_BASE_URL=https://invalid.example.invalid + stale .last-sync-version:
logs "warn: ... unreachable, auto-update skipped this launch", no crash
- /origin auto clears pin file; /origin (no arg) shows current effective
origin and pin file; /origin <https-url> persisted; /origin github
returns clear error; /origin gitea re-pins to default
- stale .origin containing "github" -> startup warn + revert to default
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds automatic PHI tokenization on two surfaces: user input and HL7-shaped
tool results. Supersedes Bryan's reverted af2ffe8 prototype with a tiered
confidence model, explicit blacklist contexts, structured audit log, and
tool-result coverage.
Bryan's directive: "Err on the side of caution and tokenize anything you
think you may need to as long as it doesn't break the tools." Priority
order: (1) don't break tools (constraint), (2) catch all PHI (goal),
(3) minimize false positives (secondary).
Detection — four-tier model (first match wins per token):
Tier 1 DEFINITE SSN (with dashes), email, formatted phone, NPI with
explicit "NPI:" prefix. Always tokenize.
Tier 2 CONTEXTUAL Numeric value preceded by MRN/Patient/DOB/Account/
Visit/Acct/Record/Birth within 20 chars. Always.
Tier 3 HL7-CTX Plausibly-PHI-shaped values when line mentions
PID.3/5/7/11/13/18, NK1.*, GT1.*, IN1.16-20.
Aggressive — prompts in confirm mode.
Tier 4 KNOWN Value already exists in $LARRY_HOME/sanitize/lookup.tsv.
Tier-4 scans the full set of categories actually present
in the table (not a hardcoded shortlist), so any
category Bryan has used before is checked.
Blacklist contexts (NEVER tokenize, even on tier match):
* Path-like (/, ./, ../, ~/, contains /)
* HL7 field references like PID.18 — the digit after the dot is a
field index, not an MRN (spec verification scenario #5)
* Version strings (vN.N.N, semver) and ISO dates (overridden by
explicit DOB/Birth context so "DOB 1980-01-15" still tokenizes)
* Port keywords (:NNNN, port NNNN, tcp/udp NNNN, LISTEN/PORT=)
* Error/status codes (error NNN, code NNN, HTTP NNN, rc=N)
* JSON key position (value followed by ": or :)
* Fenced code blocks (``` ... ``` skipped via awk redactor)
* Timestamps (epoch ms 13+ digits, epoch s 10 digits starting 1)
Tool-result surface — routed through hl7-sanitize.sh:
* Eligible tools: read_file (.hl7/.HL7/.txt/.TXT only), nc_msgs,
hl7_field, hl7_diff
* Eligibility further gated by _auto_phi_looks_like_hl7 shape check
(segment headers MSH/PID/EVN/PV1 with | delimiter)
* Generic outputs (list_dir, grep_files, bash_exec, glob_files, ssh_exec,
web search) NEVER scanned — spec is explicit about this
* For HL7-shaped content we use the canonical field-aware pipeline
rather than the prose detector, since segments are pipe-delimited
and would otherwise be a single whitespace token. Both pipelines
share lookup.tsv so tokens are stable across surfaces.
Behavior controls:
* env LARRY_AUTO_PHI: 1/on (default), 0/off, confirm
* /phi-auto on|off|confirm|status slash command
* "!nophi " per-turn prefix override
* Manual @@VALUE / {{phi:VALUE}} markers always win — preprocessed
FIRST; auto-PHI fills gaps in things Bryan didn't manually mark.
* After each pass, dim status line summarises:
phi> auto-tokenized 3 value(s) [user_input]: MRN×1 EMAIL×1 SSN×1
Audit — JSONL log at $LARRY_HOME/log/auto-phi.log:
{ "ts": "...", "value": "...", "category": "...", "token": "...",
"tier": "definite|contextual|hl7|known|hl7_pipeline",
"surface": "user_input|tool_result", "context": "..." }
Mode 0600, parent dir 0700. Best-effort write; never fails the host call.
Library changes (lib/hl7-sanitize.sh):
* normalize_value: re-add EMAIL + PHONE arms + new NPI arm. EMAIL and
PHONE arms were originally in af2ffe8 (reverted with v0.7.1) — cited
in the source comments.
* normalize-value subcommand: exposes canonical normalization so auto-PHI
can build per-session memory keys. Originally af2ffe8.
* lookup-original subcommand: probes the table for an exact match without
creating new tokens. Used by Tier-4 "already-known" detection.
Implementation notes:
* macOS bash 3.2 compatibility: ${pos: -20} returns empty when len < 20;
use explicit ${pos:$((len-20))} guarded by length check.
* Per-session decision cache (accept/decline) uses bash 4 associative
arrays with a 3.2 fallback to pipe-delimited string membership.
* Confirm-mode prompts only Tier 3-4 — Tier 1-2 hits are high-confidence
and always tokenize even in confirm mode (Bryan: err on caution).
* Detection loop iterates line-by-line so fenced-code redaction works
and so left/right context is meaningful per token.
Verification matrix (18/18 pass):
1 SSN tokenized, 2 Email tokenized, 3 MRN contextual,
4 bare digits skipped, 5 PID.18 skipped, 6 path skipped,
7 version skipped, 8 port skipped, 9 Tier-4 known catches custom
category (EMP), 10 !nophi skips, 11 existing token left alone,
12 read_file .hl7 sanitizes all PHI fields, 13 .py not HL7-shaped,
14 list_dir not HL7-shaped, 15 mode=off skips, 16a /phi-auto off
skips, 16b /phi-auto on tokenizes, 17 audit JSONL parseable.
No regressions to v0.7.2 origin switching, v0.7.1 status-line position,
v0.7.0 HL7 completion + mouse mode, v0.6.9 status state, v0.6.7 streaming,
or any earlier OAuth/SSH/lessons work. MANIFEST unchanged.
Divergence from af2ffe8 (cited in source comments):
* Tiered classifier (vs. flat regex set) — enables reasoning about WHY
a value tokenized; gates confirm-mode behavior.
* Explicit blacklist contexts — addresses spec false-positive cases
that af2ffe8 missed (HL7 field refs, ports, error codes, JSON keys).
* Tool-result surface — af2ffe8 only ran on user input.
* Structured JSONL audit log — af2ffe8 had no per-tokenization log.
* /phi-auto semantics: on|off|confirm|status (spec) vs. af2ffe8's
/auto-phi on|off|aggressive|confirm.
* Dropped the loose "Title Case Title Case" pair detector and its
name-allowlist — too high FP rate against narrative prose
("Larry Anywhere", "Mac Studio") and Bryan's name-allowlist couldn't
keep up with the long tail. Name detection now Tier-3 (HL7-context
only) and Tier-4 (already-known) only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Switches the canonical $LARRY_BASE_URL default from raw.githubusercontent.com
to the self-hosted Gitea mirror at git.bjnoela.com. GitHub stays in the
loop as $LARRY_BASE_URL_FALLBACK and is used automatically when the primary
fails (DNS, timeout, HTTP error, private repo).
What's new
- Origin defaults split into LARRY_BASE_URL (Gitea) +
LARRY_BASE_URL_FALLBACK (GitHub). Env vars still override either side.
- Every network call in self_update tries primary first, then fallback.
Emits "warn: gitea unreachable, falling back to github" on switch and
"warn: self-update skipped (both origins unreachable)" if both fail.
- New /origin slash-command family:
/origin — show current primary/fallback + which served last
/origin gitea — pin to Gitea (default state)
/origin github — swap so GitHub is primary, Gitea fallback
/origin auto — clear pin, revert to defaults
/origin <https://...> — pin to an arbitrary HTTPS base URL
Pin is persisted to $LARRY_HOME/.origin and re-read on next launch.
- Status line picks up a light origin badge when state is non-default
("github" pinned, "custom" pinned, or "gitea→github" on failover).
- install-larry.sh mirrors the same primary→fallback fetch logic so
first-contact installs still work even if Gitea is unreachable.
ACTION REQUIRED — Bryan, before this commit's auto-update path becomes
live you must set git.bjnoela.com/bryan/cloverleaf-larry repo visibility
to PUBLIC. Gitea defaults to private; until you toggle it, every client
will silently fall back to GitHub. Verify by running, from any box:
curl -fsSI https://git.bjnoela.com/bryan/cloverleaf-larry/raw/branch/main/VERSION
A 200 with the published VERSION means clients hit Gitea; a 404/403 means
they still ride the GitHub fallback.
Don't break
- v0.7.1 status-line position (between turns)
- v0.7.0 HL7 completion, mouse mode
- v0.6.9 status line state tracking, header capture
- v0.6.7 streaming, @file, slash completion, persistent history
- v0.6.6 CR-strip + slash TAB
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
render_status_line is no longer called before printf 'you[model]>'. It is
now invoked after read_user_input returns and after @file/PHI preprocessing
complete, immediately before add_user_text/agent_turn. The visual effect is
that the dim status divider sits BETWEEN turns — summarising the cost of
the just-completed turn as the user heads into the next one.
The slash-command and empty-input paths all 'continue' before the new call
site, so no status line renders on /help, /status, /clear, /quit, etc.
First-turn suppression continues to live inside render_status_line (it
returns silently while STATUS_* globals are empty and _LARRY_TURNS=0), so
the very first prompt of a session still has nothing above the response.
/status on-demand command is unchanged; LARRY_NO_STATUS=1 still disables
entirely. Comments updated at render_status_line, the STATUS_* globals
header, the help block, and the LARRY_NO_STATUS env doc.
Supersedes the earlier combined v0.7.1 (af2ffe8). PHI auto-detection and
session-artifact upload are intentionally NOT in this build — this is the
narrow status-line-only v0.7.1 Bryan requested. lib/hl7-sanitize.sh
returns to its v0.7.0 shape (PHONE/EMAIL normalize-value cases + the
normalize-value subcommand are removed because nothing in larry.sh now
calls them).
LARRY_VERSION + VERSION -> 0.7.1.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Feature 1 — Status line BELOW the prompt (was: above).
The dim status line now renders AFTER each completed agent_turn and BEFORE
the next prompt, sitting between turns as a footer to the just-finished
exchange. Shipped Option B from the spec — render_status_line moved to the
tail of the REPL loop, the call before printing the prompt was removed.
Option A (cursor manipulation under an active readline prompt) was rejected
because `read -e` takes exclusive control of the cursor and inserting a
repositioned footer below an active prompt is fragile on MobaXterm / Cygwin
(readline redisplay clobbers manual cursor moves). Visual outcome is
identical to "below the previous prompt cycle", and /status still forces a
re-render mid-conversation if needed.
Feature 2 — Automatic PHI detection.
New auto_detect_phi() runs BEFORE preprocess_phi_markers and tokenizes any
value matching PHI-shaped patterns (email, SSN, phone, DOB, MRN 6-12 digits,
HL7 caret-name, "Last, First", or loose "Title Case Title Case"). Uses the
existing hl7-sanitize.sh tokenize-value pipeline so canonicalization
(sort-unique-lowercase NAME tokens, ISO DOB, digits-only PHONE/SSN,
lowercase EMAIL) collapses different surface forms onto one token across
the session. Skipped: paths, URLs, already-tokenized values, manual @@/{{phi:}}
markers, timestamps (13+ digits or 10 digits starting with '1'), and a
built-in allowlist of common non-PHI two-word phrases ("Home Assistant",
"Mac Studio", etc.).
Modes: confirm (default — prompts Y/n on loose name-like matches once per
session), aggressive (silent always-tokenize), off. Env LARRY_AUTO_PHI;
runtime /auto-phi and /auto-phi-status slash commands. Per-turn override
with "!nophi " prefix. Manual markers always win. New normalize-value
subcommand on hl7-sanitize.sh exposes the canonicalization step so the
per-session memory cache uses canonical keys (so "John Smith" and
"JOHN SMITH" share one confirm decision). EMAIL + PHONE categories added
to normalize_value().
Feature 3 — Session-artifact upload at close.
New upload_session_artifacts() POSTs $LARRY_HOME/log/headers.log,
$LARRY_HOME/sessions/<id>.log.md, and <id>.messages.json to
$LARRY_MEMORY_UPLOAD_URL on session exit. Each request carries
X-Larry-Source (headers-log | session-log | session-messages),
X-Larry-Version, and X-Session-Id headers so the ingest side can route
appropriately. Fires from both the clean main_loop exit and the EXIT/INT/TERM
trap (idempotent via _LARRY_UPLOAD_FIRED guard). Unset URL = silent skip
with a one-line warn. Auth tokens are never logged: headers.log captures
only response headers matching ^anthropic-* or ^retry-after: (per v0.6.9
writer); the session log + messages contain post-tokenization content only.
No regressions to v0.7.0 work — HL7 tab completion, mouse mode toggles,
TOOLS_JSON heredoc, streaming, @file refs, status-line existence, slash
completion, and all v0.6.x machinery remain untouched. MANIFEST unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two REPL enhancements:
1. HL7 v2.x inline tab completion. Type a segment ID or SEG.field or
SEG.field.component in any prompt and TAB completes against a built-in
schema (18 segments fully fielded: MSH, PID, PV1, PV2, EVN, MSA, ERR,
NK1, GT1, IN1, IN2, OBR, OBX, ORC, AL1, DG1, PR1, ROL; component
breakdowns for MSH.9, PID.3, PID.5, PID.11, PV1.3, NK1.4, OBX.3, IN1.4).
New slash commands /hl7 <SEG> and /hl7-fields <SEG.N> print schema
without typing. Z-segments get a "site-specific" hint instead of a
guess. Exact-match wins over prefix siblings (PID.3 completes over
PID.30; MSH completes over MSH+MSA).
2. Mouse mode. /mouse on|off and LARRY_NO_MOUSE env kill switch enable
bracketed-paste + SGR mouse reporting (mode 1006). Click-to-position
cursor in the input line is intentionally NOT implemented in this
pass — it requires per-terminal escape parsing inside bind -x which
is not reliable across iTerm2 / macOS Terminal / MobaXterm / Cygwin
in a single pass. Documented as terminal-dependent.
New file: lib/hl7-schema.sh (sourced; bash assoc arrays for the segment
+field+component tables, plus helpers hl7_segments / hl7_fields_for /
hl7_components_for / hl7_field_name).
MANIFEST + install-larry.sh updated to fetch the new lib file on
install/self-update.
Regression-safe: v0.6.9 status line, slash completion, @file completion,
streaming SSE, header capture, and all 37 prior slash commands are
unchanged. Added 3 new slash commands (/hl7, /hl7-fields, /mouse).
Verification: 15/15 automated checks on the three completion paths
(segment, field, component) — including mid-buffer completion and
exact-match preference.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a dim status line printed immediately above each `you[model]>` prompt
every turn, surfacing context-window usage + rate-limit visibility.
Two render modes auto-selected from $LARRY_AUTH_MODE:
OAuth: ─ ctx 12% (24K/1.0M) ─ 5h 1.8% reset 19:45 ─ 7d 73.7% reset Mon Jun 2 ─
API key: ─ ctx 12% (24K/200K) ─ $0.213 session ─ 14 turns ─
Implementation areas:
- call_api / call_api_stream now capture response headers via curl -D into
tempfiles. Streaming path drains its header file in the parent shell after
the SSE body completes (subshell-update problem avoided).
- New parser _parse_response_headers handles BOTH header families per Pax's
research (Deliverables/2026-05-27-anthropic-rate-limit-headers-research.md):
* API-key: RFC 3339 datetimes → converted to epoch
* OAuth: Unix epoch integer-as-string → used as-is
Both 5h and 7d buckets are displayed simultaneously; representative-claim
is honored for enforcement but not for what to render (anti-pattern noted
by Pax — Claude Code itself once shipped buggy logic that picked one).
- Static model-context-window lookup (Pax §4): opus-4-7 / sonnet-4-6 = 1M,
haiku-4-5 and legacy 4-5/4-1 families = 200K, unknown defaults to 200K.
- Safety net: first 50 OAuth response header blocks are logged to
$LARRY_HOME/log/headers.log so the empirical schema can be diff'd against
Pax's spec on Bryan's actual account. Auto-disables after limit reached.
- New /status slash command force-renders the line on demand. New env knob
LARRY_NO_STATUS=1 disables the status line entirely.
- parse_stream_to_response synthetic JSON now also carries
cache_read_input_tokens + cache_creation_input_tokens so the parent shell
can compute ctx_used = input + cache_creation + cache_read per Pax §5.
Fallback rules followed:
- First turn of a session: status line is NOT rendered (no zero-lies).
- Missing reset values: display "reset —" not a fabricated time.
- Reset already passed: display "— reset" (data stale).
- Narrow terminal (< 100 cols): drop the reset times, keep the percentages.
Verification (synthetic fixtures; no live OAuth session in this environment):
- 25 parser/renderer assertions pass (test-harness covering all 8 spec
scenarios + model lookup + token humanization).
- SSE parser still produces a valid synthetic response JSON, now including
cache fields (7 assertions pass).
- TOOLS_JSON heredoc still parses cleanly via jq.
- bash -n on larry.sh: clean.
- Pax's OAuth headers were NOT empirically observed against a live account
in this environment — only validated against the documented schema via
fixtures derived verbatim from Pax's research. The header-log safety net
is in place to verify on Bryan's account on first use.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the gap between v0.6.7's ssh_exec/ssh_status primitives and the local
nc_* tools, so Bryan's two motivating workflows compose cleanly:
1. "Compare the ADT site NetConfig on qa to dev"
2. "Grab smat files from dev and bring to qa for regression testing"
ssh_pull, ssh_push (lib/ssh-helper.sh + larry.sh):
scp via the existing ControlMaster socket — no second auth, no second TCP
handshake. Master-not-open and missing-remote-file paths fail with explicit
messages ("open the master with /ssh-setup <alias> first"). Pull caches to
/tmp/larry-pulls/<alias>.<basename>.<hash-of-remote-path> when local_path is
omitted, so repeat pulls of the same remote file are idempotent. Validates
byte counts post-transfer to catch partial transfers.
ssh_pull_smat (lib/ssh-helper.sh + larry.sh):
Cloverleaf-aware smatdb pull. Full mode scp's the entire .smatdb;
sampled mode (days_back=N) runs sqlite3 server-side via ssh_exec to extract
up to 1000 recent messages as TSV with base64-encoded MessageContent blobs
(verified end-to-end with a synthetic smatdb fixture matching nc-msgs.sh's
smat_msgs schema). Avoids transferring multi-GB archives when only N
samples are needed.
nc_diff_interface tool (newly wired):
Promotes lib/nc-diff-interface.sh into the LLM-callable tool surface. Used
by the new /nc-diff-env slash command for workflow #1.
nc_regression cross-env (lib/nc-regression.sh + larry.sh):
source_ssh_alias / target_ssh_alias args. Phase 1 (discovery) and Phase 2
(sample) run via ssh_exec + ssh_pull / ssh_pull_smat against the source
alias. Phase 3/4 (route_test) push inputs over and pull outputs back via
ssh_push / ssh_pull. Phases 5/6 (diff + summary) stay local. Reports
reference the SSH alias names rather than raw user@host strings.
/nc-diff-env and /nc-regression-env slash commands (larry.sh):
Templated prompts to Larry-the-LLM that explicitly cite the motivating
workflows, call out ssh_status / ssh_pull / nc_diff_interface and the
nc_regression cross-env fields. Registered in _LARRY_SLASH_CMDS +
_LARRY_SLASH_CMDS_DESC + /help per v0.6.7 patterns.
Bug fix unearthed during cross-env work:
lib/nc-regression.sh phase_5 / phase_6 used printf 'FORMAT' where FORMAT
begins with '- '. bash 3.2 (macOS default) reads the leading '-' as a bad
option and emits nothing — silently dropping the entire "Configuration"
section of regression-summary.md. Switched the affected lines to
printf -- 'FORMAT' so the format string is unambiguous.
Tool/slash surface deltas vs v0.6.7:
Tools: 31 → 35 (+ssh_pull, +ssh_push, +ssh_pull_smat, +nc_diff_interface)
Slash commands: 34 → 36 (+/nc-diff-env, +/nc-regression-env)
Updated tool descriptions for read_file, grep_files, nc_msgs to point at
ssh_pull / ssh_pull_smat as the cross-env pre-step so Larry-the-LLM picks
the right chain on the first attempt.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Streaming API responses (marquee). Replaced the one-shot curl/jq pipeline
with an SSE-aware path that emits text deltas as they arrive. Server-sent
events are parsed line-by-line; content_block_delta with text_delta is
flushed to stderr immediately, while tool_use blocks accumulate
input_json_delta partials until content_block_stop, validate the assembled
JSON, then execute the tool exactly as before. A synthetic response is
rebuilt from streamed blocks to keep add_assistant_blocks + cost tracking
identical to the non-streaming path. Falls back to non-streaming on parse
failure or via LARRY_NO_STREAM=1. Cite: platform.claude.com/docs/en/api/
messages-streaming for SSE event types and tool-use streaming semantics.
Fuzzy slash completion polish + descriptions. Multi-match TAB now renders
each candidate on its own line with a one-line description, cyan command +
dim description. Backed by a new _LARRY_SLASH_CMDS_DESC associative array
that mirrors print_help's wording so the two stay in sync.
Persistent command history. HISTFILE=$LARRY_HOME/.history with
HISTSIZE=1000 and HISTCONTROL=ignoredups. history -r at REPL start;
history -a after each accepted input. Skips appending /login, /ssh-pass,
and /ssh-add lines so credentials never hit the history file.
/clear slash command. Clears the terminal via the ANSI 2J/H sequence
(works without external `clear`). Distinct from /reset (which clears the
conversation, not the screen). Listed in print_help and the canonical
_LARRY_SLASH_CMDS array.
Multi-line paste auto-detection. read_user_input now reads the first line
then non-blockingly polls stdin for buffered bytes within 50ms; if more
is present, it's slurped as continuation. Also: a trailing backslash on
the first line enters a multi-line mode that ends at a blank line. The
explicit '<<' / 'EOF' heredoc still works for users who prefer it.
/copy — copy last assistant response to clipboard. Tool detection cascade:
pbcopy (macOS), wl-copy (Wayland), xclip, xsel, /dev/clipboard (Cygwin),
clip.exe (WSL/Cygwin fallback). Falls back to printing the text to stdout
with a warning when no clipboard tool is found.
/cost — running token + dollar cost. Tracks input/output/cache-read/
cache-write tokens across both streaming and non-streaming responses.
Pricing constants inline as of 2026-05 (Sonnet $3/$15, Opus $15/$75,
Haiku $1/$5, cache writes 1.25x input, cache reads 0.1x input). The /cost
report shows per-stream subtotals and a session total. Refresh constants
periodically from platform.claude.com/docs/en/about-claude/pricing.
Model name in the prompt. The prompt now reads `you[sonnet-4.6]>` (or
`you[opus-4.7]>`, etc.) derived from $LARRY_MODEL via model_short_name —
strips the `claude-` prefix and converts the trailing -N-M to .N.M. The
prompt updates immediately after /model. Color scheme preserved.
Tool-call display polish. Cyan + bold tool name, dim args one key:value
per line, long values (>120 chars) truncated with a `(use /show-last-tool
for full args)` hint. New /show-last-tool slash command prints the full
last tool call JSON + the full result body for debugging.
Friendlier error messages. Audit pass through the most common API
failures: empty response surfaces as "Network error: empty response from
<URL>"; authentication_error invites /login when OAuth/token wording
appears; rate_limit_error and overloaded_error read as "Rate limited by
Anthropic — wait a few seconds"; not_found_error includes the current
LARRY_MODEL since that's almost always the cause; jq parse errors during
tool results get wrapped as "Tool returned malformed JSON; raw body: ..."
@file inline-file syntax. The user types @<path> in any prompt; Larry
resolves each ref before send-time and appends the file contents as a
fenced block keyed by extension. Grammar supports @bare-token and
@{bracketed path with spaces}; emails (bryan@x.com) are skipped via a
look-behind on the preceding character. Validation: missing → leave
literal with warning; directory → skip; binary (null-byte scan of first
8 KB) → skip; >250 KB → truncate with a footer note. Multiple refs are
deduped. Runs BEFORE PHI tokenization so PHI markers inside attached
files still get caught. TAB after @ completes against files (find -depth
4, fzf picker when on PATH). A one-time per-session tip prints the first
time the user types @.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bug 1 (PRIMARY — unblocks OAuth on MobaXterm): jqf now strips \r from every
jq output. Root cause of the multi-day "OAuth token unavailable" cascade was
CRLF-tainted .oauth.json: `fetched_at=$(jqf ... '.fetched_at')` captured
"1716826990\r", and `$((fetched_at + expires_in))` crashed with the cryptic
"invalid arithmetic operator (error token is \"\"") — bash trying to print
the embedded CR. Single-point fix in jqf covers every caller (ensure, refresh,
status, debug, login). Added belt-and-suspenders `printf '%d'` coercion on
every numeric capture so any future non-CR junk falls back to 0 instead of
crashing arithmetic. /oauth-debug now reports CR/LF byte counts so future
CRLF taint is visible at a glance.
Bug 2 (security): .oauth.json was landing at 0644 on Cygwin/MobaXterm even
though both cmd_login and cmd_refresh called `chmod 600`. Introduced
secure_install (install -m 600 → cp+chmod → mv+chmod fallback chain) so the
mode is set atomically at placement. Also added umask 077 to cmd_refresh
(only cmd_login had it) so the .new sidecar is created tight from the start,
plus a pre-mv chmod 600 on the sidecar for fs-where-install-doesn't-stick.
On a fully POSIX FS this is now triple-redundant; on Cygwin NTFS we get as
close to 0600 as the ACL emulation will allow.
Feature 1: TAB completion for slash commands. New _LARRY_SLASH_CMDS canonical
array near read_user_input, __larry_complete_slash uses `bind -x` to read
$READLINE_LINE / $READLINE_POINT and rewrites the buffer in-place. Prefix
matching is primary; subsequence fuzzy is the fallback. Non-slash lines and
mid-arg TABs fall back to literal-tab insertion so muscle memory isn't
broken. Heredoc continuation lines DO NOT get completion (binding only fires
on the first read). /help section documents the behavior with examples.
Smoke-tested on macOS:
- CRLF-tainted .oauth.json: ensure returns access_token cleanly, status &
debug print real numbers + human timestamps (no bash arith crash).
- secure_install: file ends at 0600 even when source was 0644.
- Completion: /h→/help, /ss→lists ssh-*, /ssh-h→/ssh-hosts, /q→/quit,
/oa→/oauth-debug, /sssp→/ssh-setup (fuzzy), /xyz→silent, non-slash and
"/cmd args"→literal tab.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
call_api was swallowing every byte of oauth.sh ensure's stderr with
`2>/dev/null`, so when ensure returned an empty token there was zero
diagnostic info — just "OAuth token unavailable". With Bryan hitting an
intermittent failure on MobaXterm we'd already burned two guess-fix
cycles; this ships the data instead of another guess.
Changes:
- call_api now captures ensure's stderr to a tempfile and surfaces it
via err() when the token comes back empty, pointing the user at
/oauth-debug for full state.
- cmd_ensure validates the file parses as JSON before destructuring,
validates .access_token is non-empty before emitting, and emits a
decision trace to stderr under LARRY_OAUTH_DEBUG=1.
- New cmd_debug subcommand (oauth.sh debug) dumps: file state (mode,
size, mtime, JSON validity), parsed fetched_at + expires_in + now +
computed expiry + would_refresh decision, jq binary path + version +
Unix/Windows-native flavor, cygpath -w translation when on Cygwin,
truncated previews of access/refresh tokens (first 20 chars + length
only — safe to share), and a live LARRY_OAUTH_DEBUG=1 ensure trace.
- New /oauth-debug slash command exposes it from the REPL, documented
in /help.
- cmd_login and cmd_refresh now write to .new sidecars, validate
required keys parse, then atomically mv — guards against the
corrupted-file failure mode that would silently break ensure on a
later run.
Happy path unchanged: when the file is valid and the token is in-window
ensure prints just the access_token on stdout with no stderr.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bryan hit this on MobaXterm immediately after v0.6.3 shipped:
C:\Users\...\bin\jq.exe: Bad JSON in --rawfile c /tmp/tmp.AlsIcdX9aO:
Could not open /tmp/tmp.AlsIcdX9aO: No such file or directory
error: OAuth token unavailable; run 'larry-auth.sh login' to re-authenticate
error: empty response from API (timeout or network?)
Root cause is the same one v0.5.4 fixed for the OAuth file reads, but now
biting the new tempfile pipeline introduced for argv-overflow avoidance.
The Windows-native jq.exe shipped to MobaXterm via install-larry.sh
cannot resolve Cygwin paths like /tmp/tmp.X or /home/mobaxterm/.larry/...
when they come in as argv arguments to --rawfile / --slurpfile — it
interprets them as Windows paths and the open() fails.
v0.5.4 fixed this for stdin-read cases by piping via bash redirection
(`< "$file"`), since bash handles the cygwin→windows path open before
jq sees a file descriptor. But --rawfile / --slurpfile DO want a path
argument, so the stdin trick doesn't apply — we must give jq a path it
can actually open.
Fix: new jqpath() helper translates a cygwin path to its real Windows
equivalent via `cygpath -w` when cygpath exists (Cygwin / MobaXterm / MSYS).
On Linux and macOS cygpath is absent and the helper echoes the path
unchanged — true cross-platform no-op outside Cygwin.
Wrapped every --rawfile / --slurpfile argv path in larry.sh:
add_user_text --rawfile c ←
add_assistant_blocks --slurpfile b ←
add_user_tool_results --slurpfile b ←
agent_turn payload --rawfile system, --slurpfile messages,
--slurpfile tools ←
agent_turn tool-result aggregation --rawfile c ←
Verified on macOS: jqpath echoes paths unchanged, the 40KB prompt
smoke test from v0.6.3 still works end-to-end.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
v0.6.2 fixed the TOOLS_JSON argv overflow but four other call sites had
the same risk pattern — any of them would have crashed under Cygwin's
~32KB argv cap with large user input, large agent responses, or large
tool results:
add_user_text --arg c "$content" ← multi-paragraph prompts
add_assistant_blocks --argjson b "$blocks" ← long assistant turns
add_user_tool_results --argjson b "$blocks" ← chained tool results
agent_turn loop --arg c "$result" ← tool output (up to 250KB
for read_file, 500 lines
for ssh_exec, etc.)
agent_turn loop --arg system "$system_prompt" ← agents/*.md
total ~25KB
All five are now passed via tempfile + --rawfile (for raw strings) or
--slurpfile (for pre-parsed JSON). Same proven pattern as the v0.6.2
TOOLS_JSON fix. Tempfiles are cleaned at every return path.
Verified by pushing a 60KB user prompt through the pipeline on macOS
(also has the larger 256KB argv cap that masked these bugs locally
before, but the codepath now uses files for the large values regardless
of platform). Messages file stored the full 60025-char prompt with no
warnings.
After this commit, the only --arg / --argjson calls remaining all carry
known-small values (UUIDs, version strings, port numbers, etc.).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bryan hit this on every chat turn from MobaXterm:
jq: Argument list too long
error: OAuth token unavailable; ...
error: empty response from API
warn: turn ended with error
Root cause: agent_turn was building the API payload with
jq -n ... --argjson tools "$TOOLS_JSON" ...
which puts all 21KB of tool-definition JSON on the jq command line as
argv. Cygwin/Windows argv limit is ~32KB total — combined with the other
args this exceeded it and jq failed with E2BIG. Linux/macOS have a much
larger limit (256KB+) so this never showed in local testing.
The downstream errors were cascade noise: jq fails to write payload →
call_api runs with empty/partial payload → various error paths fire.
Fix: write TOOLS_JSON to a tempfile once per agent_turn and reference
it via --slurpfile (same pattern already used for $MESSAGES_FILE).
Tempfile is cleaned up at every return path.
Verified locally — payload now builds cleanly with no argv warnings.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
THREE bugs that Bryan hit in v0.6.0:
(1) TOOLS_JSON: unbound variable crash on any unrecognised input.
Root cause: the TOOLS_JSON assignment was a single-quoted string spanning
~40 lines, and apostrophes in tool descriptions (Anthropic's, 'codametrix',
protocol's, etc.) closed the bash string prematurely. Bash then tried to
execute fragments of the JSON as shell commands ("NAME: command not found"
warnings) and TOOLS_JSON never got assigned. With set -u, the first
reference to $TOOLS_JSON crashed the whole script.
Fix: switch to a quoted-EOF heredoc — TOOLS_JSON=$(cat <<'TOOLS_END' ...
TOOLS_END). Heredoc with single-quoted delimiter preserves content
literally — apostrophes, backslashes, all of it. Verified: all 31 tool
defs now parse as valid JSON (including the previously-broken nc_msgs).
Also fixed the pre-existing \\" → \" escape error in nc_msgs.
(2) Slash command brittleness: /ssh-add\ * pattern matched only when args
were present; /ssh-add alone fell through to the catchall and reported
"unknown command". Also failed silently with no usage message on the
too-few-args path.
Fix: rewrote all SSH slash patterns as /ssh-foo* (matches both with and
without trailing args), with a _slash_args() helper that cleanly extracts
the arg portion. Every handler now validates and prints "usage: ..." on
missing args before continuing. New _run_ssh_helper() wrapper centralises
the installed-check and swallows helper exit codes so they don't propagate
into the main loop.
(3) Backspace not working in MobaXterm/Cygwin terminals.
Root cause: terminal sends ^? (DEL) for backspace but stty erase is often
set to ^H (BS) in MobaXterm, so backspace passes through as a literal
character.
Fix: stty erase '^?' at REPL startup (harmless if already correct), AND
switch read_user_input to use `read -e -r -p` which uses libreadline for
line editing — backspace, arrow keys, history all work via readline now,
bypassing the terminal's stty config entirely. Falls back to plain read
on environments without readline support.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
NEW lib/ssh-helper.sh implements the full SSH command surface:
hosts/list show configured remote hosts
add <alias> <user@host[:port]> register a new host
remove <alias> remove + clean cred + socket
pass <alias> set/update password (hidden interactive)
setup <alias> open long-lived ControlMaster
close <alias> close ControlMaster
status [alias] show open masters + cred presence
exec <alias> <command...> run command via master
Architecture:
• $LARRY_HOME/.ssh-hosts.tsv — alias \t user@host \t port (3-col)
• $LARRY_HOME/.ssh-creds/<alias> — raw password, mode 0600
• $LARRY_HOME/.ssh-sockets/<alias>.sock — ControlMaster socket
The password is read from disk by sshpass via -f (file argument), so it
never lands in argv or environment. It is used ONCE to open the master;
all subsequent execs multiplex through the socket with no auth. Daily-
rotating passwords: just overwrite the cred file and re-run setup.
SLASH COMMANDS wired in larry.sh REPL: /ssh-hosts /ssh-add /ssh-remove
/ssh-pass /ssh-setup /ssh-close /ssh-status /ssh <alias> <cmd>.
LARRY TOOLS exposed to the LLM:
ssh_status — list aliases + open-master state
ssh_exec — run command on remote via the master socket
Both tool descriptions explicitly tell Larry the password is unreachable
and to ask Bryan to run /ssh-setup if a master is closed. Tool inputs
and outputs never contain the password. Output capped at max_lines
(default 500) with a "[ssh_exec: exit rc=N]" footer.
Bundle updated: MANIFEST + install-larry.sh both now include
lib/ssh-helper.sh. Auto-update will pull it on next launch.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bryan asked for an easier-to-remember inline PHI marker than {{phi:VALUE}}
and for name forms like SMITH^JOHN / Smith, John / John Smith / JOHN SMITH
to all collapse to the same hash. Both shipped.
INLINE SYNTAX (in addition to the legacy {{phi:VALUE}} which still works):
@@VALUE unbracketed — VALUE has no whitespace
e.g. @@12345 @@SMITH^JOHN @@V789
@@VALUE@@ bracketed — VALUE may contain spaces
e.g. @@John Smith@@ @@Smith, John@@
Parser is 2-pass to disambiguate mixed forms in the same prompt: bracketed
markers are matched first (via grep -oE with a regex that excludes leading/
trailing whitespace inside the brackets), then the unbracketed pass scans
the remaining text. Verified against:
"look for @@12345 in PID.3 for @@John Smith@@ DOB @@01/15/1985 ..."
extracts 4 markers correctly and routes each to its category.
AUTO-CATEGORY DETECTION (lib/hl7-sanitize.sh: detect_category):
pure digits 4-15 → MRN
9 digits with dashes → SSN
date-shaped → DOB
caret or comma → NAME
2+ alpha tokens → NAME
else → MANUAL
CANONICALIZATION (lib/hl7-sanitize.sh: normalize_value):
NAME: lowercase, replace ',^/' with spaces, sort unique alpha tokens
SMITH^JOHN, Smith John, John Smith, JOHN SMITH → "john smith"
DOB: parse to YYYY-MM-DD (GNU date or BSD date fallback)
SSN: strip dashes/whitespace
MRN/MANUAL: trim outer whitespace only
TABLE SCHEMA bumped to 4 columns (token / category / canonical / original).
Legacy 3-column rows still read fine — lookups key on column 3 which is
"canonical" in new rows and "value" in legacy rows (mismatches just create
a new token, no corruption). Detokenize prefers column 4, falls back to
column 3 for legacy compat.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Symptom: OAuth login succeeded on the work box but cmd_status emitted three
'jq: error: Could not open file' lines and showed empty fields. Same pattern
would have hit every subsequent chat turn via larry.sh's MESSAGES_FILE reads.
Root cause: install-larry.sh fetches a Windows-native jq.exe on cygwin/
mobaxterm platforms. Windows jq can't resolve Cygwin paths like
/home/mobaxterm/.larry/.oauth.json when they come in as argv arguments
(it interprets the leading slash as a Windows root). Bash's `>` redirection
worked because bash itself does the path open and hands jq an fd — the
read-side calls were passing the path string directly.
Fix: every read-side jq call now uses stdin redirection (`jq '...' < file`),
where bash does the open. Universal:
- Linux/macOS native jq: identical behavior (was already file-open-from-bash)
- MobaXterm/Cygwin/Git Bash with Windows jq.exe: now works
- WSL: works (Linux-native jq, same as Linux)
- Native PowerShell/cmd: doesn't apply — larry-anywhere is a bash script
Changes:
- lib/oauth.sh: new jqf() helper; 10 sites converted. Refactored cmd_refresh
to drop --slurpfile (which can only take a path) — pre-reads the previous
refresh_token, then uses --arg.
- larry.sh: add_user_text / add_assistant_blocks / add_user_tool_results
now pipe $MESSAGES_FILE via stdin too.
Verified: cmd_status against a real token file produces clean output, no
jq errors. Syntax check passes both files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Confirmed against live token endpoint (HTTP/2 200, valid sk-ant-oat01- and
sk-ant-ort01- tokens returned) that the v0.5.2 0.5.2 request body and
URLs were correct — the EXCHANGE itself works fine from my Mac. Bryan's
work-box launches still get 'rate_limit_error' from the same script.
Only meaningful differences in the working curl vs the failing one:
- Working: explicit User-Agent (claude-cli/2.1.85) + Accept: application/json
- Failing: defaults (curl/X.Y.Z, no Accept)
Anthropic's OAuth endpoint apparently checks User-Agent (or the Accept
header) and returns the misleading rate_limit_error for unrecognized
clients. Adding both headers to match what claude-cli and droidrun
send. Patched in cmd_login AND cmd_refresh.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause of every prior 'rate_limit_error' on OAuth login: Anthropic
migrated all the Claude-subscription OAuth endpoints from
console.anthropic.com / claude.ai to platform.claude.com / claude.com.
The old endpoints aren't 404 — they accept the POST and return a generic
'rate_limit_error' for every request, which is what mis-led both me and
several public community implementations.
Confirmed against two current working clients (droidrun/mobilerun and
motiful/cc-gateway, both using the same Claude Code public client_id):
AUTHORIZE_URL: claude.ai/oauth/authorize
→ claude.com/cai/oauth/authorize
TOKEN_URL: console.anthropic.com/v1/oauth/token
→ platform.claude.com/v1/oauth/token
REDIRECT_URI: console.anthropic.com/oauth/code/callback
→ platform.claude.com/oauth/code/callback
SCOPE: org:create_api_key user:profile user:inference
→ ...plus user:sessions:claude_code user:mcp_servers user:file_upload
Also updated the error-hint text to mention the misleading-rate-limit
pattern for both 'malformed code' AND 'dead endpoint' cases, and to cite
the current TOKEN_URL — so if/when these move again, the next person
hitting the same trap finds the answer in the script's own output.
The CODE#STATE parsing from 0.5.0 was correct and stays. State IS sent
in the token-exchange body (verified against droidrun's working flow).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
In 0.5.0 (and every prior version), prompt_first_run_auth was called
unconditionally at script load time, BEFORE self_update. On a never-
authenticated box, this meant a broken lib/oauth.sh trapped the user:
1. larry starts
2. no creds → auth prompt fires
3. pick OAuth → old broken oauth.sh runs → rate_limit_error
4. Ctrl-C at the API-key fallback prompt
5. script exits — self_update never ran
6. relaunch → exact same trap, forever
Fix: defer the auth-prompt call to run AFTER self_update. The auth
function DEFINITION stays where it is; only the CALL site moves. Now
on a fresh box:
1. larry starts
2. self_update phase A pulls MANIFEST and refreshes everything,
including a patched lib/oauth.sh
3. THEN the auth prompt fires, using the now-correct OAuth code
Verified: with no ANTHROPIC_API_KEY and no .oauth.json, the manifest
sync log lines appear before the "First-run authentication setup" menu.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
