cloverleaf-larry/lib/phi-sidecar.sh
Bryan Johnson 60b8f0e1c8 v0.8.2: Presidio sidecar for free-text NER (tier-5) — closes V1
The only path that closes V1 (free-text PHI gap — the dominant real-world
failure mode per Vera). Opt-in install; larry runs in v0.8.1 mode on hosts
without Presidio (MobaXterm/Cygwin per Bryan's accepted tradeoff).

New files:
- lib/phi-presidio-sidecar.py — FastAPI service on 127.0.0.1:$LARRY_PHI_PORT
  (default 41189). Presidio AnalyzerEngine + AnonymizerEngine over spaCy
  en_core_web_sm + 3 HL7-specific custom recognizers (HL7_MRN, HL7_CARET_NAME,
  HL7_PHONE_BARE). POST /redact and GET /health.
- lib/phi-sidecar.sh — lifecycle (start/stop/status/health/ensure). ensure
  is idempotent; called backgrounded from main_loop so it never blocks the
  first prompt. Honors LARRY_PHI_VENV.
- lib/phi-client.sh — bash client (phi_client_available / phi_redact_text /
  phi_redact_entities). CR-safe; 5s timeout bounds tier-5 stall.

larry.sh:
- auto_detect_phi gains tier-5: after tiers 1-4, before status summary,
  source phi-client.sh, run Presidio on a token-masked copy of the input,
  tokenize each entity through hl7-sanitize.sh tokenize-value (category
  presidio_<TYPE>) so token IDs stay stable. Honors confirm + strict modes.
  Removed the v0.7.3 early-return that skipped past tier-5 when tiers 1-4
  found nothing — pure prose now always reaches tier-5.
- Token-safe substitution: existing [[...]] tokens are pulled to sentinels,
  tier-5 value is replaced, sentinels restored — prevents the token-within-
  token corruption that naive literal-replace caused on already-tokenized
  text. Acronym guard drops HL7/clinical jargon (SSN/MRN/DOB/ADT) Presidio
  over-tags as ORGANIZATION.
- Graceful degradation: sidecar unreachable → tier-5 no-ops with a one-time
  stderr warning. /phi-sidecar slash command + completion table.

install-larry.sh:
- Probes python3 3.9+; offers to create $LARRY_HOME/phi-venv and install
  presidio + fastapi + uvicorn + en_core_web_sm. Skips silently (with a
  v0.8.1-mode note) on Cygwin/MobaXterm without python3, and on
  non-interactive pipe installs. Sets LARRY_PHI_VENV in the larry shim.

MANIFEST: three new lib files added for auto-sync.

Prototype validation (Bryan's Mac, Apple Silicon, Python 3.14):
  cold start (en_core_web_sm): ~9s   (vs ~82s if Presidio auto-grabs _lg;
                                       we pin _sm for the REPL budget)
  warm analyzer latency:       P50 20.6ms / P95 22.7ms
  end-to-end HTTP round-trip:  ~57ms warm; ~150ms first-post-startup
All comfortably under the 200ms-per-turn budget.

MobaXterm verdict: v0.8.2 is Mac/Linux-only. MobaXterm stays on v0.8.1 +
nudges, per Bryan's explicit acceptance. install-larry.sh enforces this
by platform detection; larry.sh tier-5 silently no-ops when the sidecar
is absent (which IS the MobaXterm path — no code is platform-gated).

Verification: bash -n clean on larry.sh + all 3 new lib scripts; python3
ast.parse clean on the sidecar; end-to-end tier-5 tested live against the
sidecar (pure prose, rule-pack+tier-5 combined with no token corruption,
!nophi bypass); strict-mode fail-closed abort tested; CR-taint, path-block,
and base64 round-trip batteries re-run green.

Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
2026-05-27 20:00:23 -07:00

193 lines
6.9 KiB
Bash
Executable File

#!/usr/bin/env bash
# ─────────────────────────────────────────────────────────────────────────────
# larry-anywhere v0.8.2: PHI Presidio sidecar lifecycle
#
# Manages the local Presidio FastAPI service used by auto-PHI tier-5
# (free-text NER). Started once at larry-anywhere REPL boot (best-effort —
# never blocks larry's startup), reused across turns, torn down on exit.
#
# Subcommands:
# start — launch the sidecar in the background if not already up
# stop — gracefully terminate the sidecar (TERM, then KILL)
# status — report up/down + port + pid
# health — curl /health endpoint (one-shot)
# ensure — start if not up; quick no-op if up. Idempotent. The
# primary entry point for larry.sh launch flow.
#
# Env:
# LARRY_PHI_PORT default 41189
# LARRY_PHI_HOST default 127.0.0.1
# LARRY_PHI_PYTHON default python3
# LARRY_PHI_VENV optional path to a virtualenv; if set, uses
# $LARRY_PHI_VENV/bin/python instead
# LARRY_HOME stores PID file at $LARRY_HOME/.phi-sidecar.pid
# and stderr log at $LARRY_HOME/log/phi-sidecar.log
#
# Failure handling:
# If the sidecar can't start (missing deps, port collision, model missing),
# `start` returns non-zero with a stderr explanation. Callers in larry.sh
# MUST treat sidecar absence as "tier-5 disabled" — don't block the turn.
# ─────────────────────────────────────────────────────────────────────────────
set -uo pipefail
LARRY_HOME="${LARRY_HOME:-$HOME/.larry}"
LARRY_PHI_PORT="${LARRY_PHI_PORT:-41189}"
LARRY_PHI_HOST="${LARRY_PHI_HOST:-127.0.0.1}"
LARRY_PHI_PYTHON="${LARRY_PHI_PYTHON:-python3}"
_PHI_SCRIPT_DIR="$(cd "$(dirname "$0")" 2>/dev/null && pwd)"
_PHI_SIDECAR_PY="$_PHI_SCRIPT_DIR/phi-presidio-sidecar.py"
_PHI_PID_FILE="$LARRY_HOME/.phi-sidecar.pid"
_PHI_LOG_FILE="$LARRY_HOME/log/phi-sidecar.log"
# Coerce CR-tainted port number (Cygwin defense — v0.7.5 lesson).
if [ -f "$_PHI_SCRIPT_DIR/cygwin-safe.sh" ]; then
# shellcheck source=cygwin-safe.sh
. "$_PHI_SCRIPT_DIR/cygwin-safe.sh" 2>/dev/null || true
fi
if declare -F coerce_int >/dev/null 2>&1; then
LARRY_PHI_PORT=$(coerce_int "$LARRY_PHI_PORT" 41189)
fi
_phi_python() {
if [ -n "${LARRY_PHI_VENV:-}" ] && [ -x "$LARRY_PHI_VENV/bin/python" ]; then
printf '%s' "$LARRY_PHI_VENV/bin/python"
return
fi
if command -v "$LARRY_PHI_PYTHON" >/dev/null 2>&1; then
printf '%s' "$LARRY_PHI_PYTHON"
return
fi
printf ''
}
_phi_is_up() {
# Health check via curl (lightweight). Don't trust the PID file alone —
# process could be a stale pid for an unrelated python.
curl -fsS -m 1 "http://${LARRY_PHI_HOST}:${LARRY_PHI_PORT}/health" >/dev/null 2>&1
}
cmd_status() {
if _phi_is_up; then
local body; body=$(curl -fsS -m 1 "http://${LARRY_PHI_HOST}:${LARRY_PHI_PORT}/health" 2>/dev/null)
printf 'phi-sidecar: up — %s (pid %s)\n' "$body" "$(cat "$_PHI_PID_FILE" 2>/dev/null || echo unknown)"
return 0
fi
printf 'phi-sidecar: down\n'
return 1
}
cmd_health() {
curl -fsS -m 1 "http://${LARRY_PHI_HOST}:${LARRY_PHI_PORT}/health" 2>/dev/null
local rc=$?
if [ "$rc" != "0" ]; then
printf '{"status":"down","error":"unreachable on %s:%s"}\n' "$LARRY_PHI_HOST" "$LARRY_PHI_PORT" >&2
return 1
fi
echo
return 0
}
cmd_start() {
if _phi_is_up; then
cmd_status
return 0
fi
local py; py=$(_phi_python)
if [ -z "$py" ]; then
printf 'phi-sidecar: cannot start — python3 not on PATH (set LARRY_PHI_PYTHON or LARRY_PHI_VENV)\n' >&2
return 4
fi
if [ ! -r "$_PHI_SIDECAR_PY" ]; then
printf 'phi-sidecar: cannot start — %s missing\n' "$_PHI_SIDECAR_PY" >&2
return 4
fi
# Quick dependency probe (don't load the model — that takes 9s. Just
# check imports succeed). If this fails, exit early with a clear message.
if ! "$py" -c 'import presidio_analyzer, presidio_anonymizer, fastapi, uvicorn' 2>/dev/null; then
printf 'phi-sidecar: cannot start — presidio_analyzer / presidio_anonymizer / fastapi / uvicorn not installed for %s\n' "$py" >&2
printf ' install with: %s -m pip install presidio_analyzer presidio_anonymizer fastapi uvicorn\n' "$py" >&2
printf ' then: %s -m spacy download en_core_web_sm\n' "$py" >&2
return 5
fi
mkdir -p "$(dirname "$_PHI_PID_FILE")" "$(dirname "$_PHI_LOG_FILE")" 2>/dev/null
LARRY_PHI_PORT="$LARRY_PHI_PORT" LARRY_PHI_HOST="$LARRY_PHI_HOST" \
nohup "$py" "$_PHI_SIDECAR_PY" >> "$_PHI_LOG_FILE" 2>&1 &
local pid=$!
echo "$pid" > "$_PHI_PID_FILE"
# Wait up to 30 seconds for the model to load + the FastAPI port to open.
local i
for i in $(seq 1 30); do
sleep 1
if _phi_is_up; then
printf 'phi-sidecar: started in %ds (pid %s, port %s)\n' "$i" "$pid" "$LARRY_PHI_PORT" >&2
return 0
fi
# If the python process died, surface the tail of the log.
if ! kill -0 "$pid" 2>/dev/null; then
printf 'phi-sidecar: process died during startup; tail of log:\n' >&2
tail -20 "$_PHI_LOG_FILE" >&2
rm -f "$_PHI_PID_FILE"
return 6
fi
done
printf 'phi-sidecar: did not become healthy within 30s; tail of log:\n' >&2
tail -20 "$_PHI_LOG_FILE" >&2
return 7
}
cmd_stop() {
local pid=""
[ -f "$_PHI_PID_FILE" ] && pid=$(cat "$_PHI_PID_FILE" 2>/dev/null)
if [ -z "$pid" ]; then
printf 'phi-sidecar: no pid file\n'
return 0
fi
if kill -0 "$pid" 2>/dev/null; then
kill -TERM "$pid" 2>/dev/null
local i
for i in 1 2 3 4 5; do
sleep 1
kill -0 "$pid" 2>/dev/null || break
done
if kill -0 "$pid" 2>/dev/null; then
kill -KILL "$pid" 2>/dev/null
fi
fi
rm -f "$_PHI_PID_FILE"
printf 'phi-sidecar: stopped (pid %s)\n' "$pid"
}
cmd_ensure() {
if _phi_is_up; then
return 0
fi
cmd_start
}
case "${1:-}" in
start) shift; cmd_start "$@" ;;
stop) shift; cmd_stop "$@" ;;
status) shift; cmd_status "$@" ;;
health) shift; cmd_health "$@" ;;
ensure) shift; cmd_ensure "$@" ;;
""|help|-h|--help)
cat <<USAGE
phi-sidecar.sh — larry-anywhere v0.8.2 Presidio sidecar lifecycle
start launch (background) if not up; waits up to 30s for health
stop gracefully terminate (TERM then KILL)
status up/down + pid
health one-shot curl /health
ensure start if down, no-op if up (idempotent; primary entry point)
Env: LARRY_PHI_PORT (default 41189), LARRY_PHI_HOST (default 127.0.0.1),
LARRY_PHI_PYTHON (default python3), LARRY_PHI_VENV (optional venv).
Logs: \$LARRY_HOME/log/phi-sidecar.log
PID: \$LARRY_HOME/.phi-sidecar.pid
USAGE
;;
*) printf 'phi-sidecar.sh: unknown subcommand: %s\n' "$1" >&2; exit 2 ;;
esac