v0.9.9: tty-restore + nc-regression -a + hl7-diff len10 framing & F-7 hang fix

Three fixes bundled for one self-update:
- tty-restore (larry.sh): save/restore stty around the REPL on the EXIT/INT/TERM
  trap — fixes backspace breaking (^H) after larry exits on MobaXterm where the
  client's unconditional 'stty erase' was never restored. Vera CODE-PASS.
- nc-regression -a (lib/nc-engine.sh, lib/nc-regression.sh, MANUAL.md): route_test
  now passes -a so multi-message files test ALL messages, not just the first (F-6).
- hl7-diff len10 framing + F-7 (lib/hl7-diff.sh): new --framing auto|nl|0x1c|len10
  parser for hciroutetest's default 10-digit length-prefix output; and a fix for a
  latent infinite-loop (ignored() awk fn missing 4 local decls) that hung the diff
  at 100% CPU on any 2+-message file where a default-ignored field (MSH.7) differs —
  i.e. the exact Windows->Linux ADT regression scenario. Verified no-hang (0.089s).

Consolidated by Clover; Vera CODE-PASS (Deliverables/qa/vera-gate1-cloverleaf-larry-v099-consolidated-20260723.md). Live-acceptance = Bryan's live HL7 regression run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Bryan Johnson 2026-07-23 10:17:42 -07:00
parent abc0d096d0
commit 4a1d2451f3
8 changed files with 311 additions and 41 deletions

View File

@ -4,6 +4,103 @@ All notable changes to `cloverleaf-larry` / `larry-anywhere` are recorded here.
Versioning is loose-semver; bumps trigger the in-process self-update on every Versioning is loose-semver; bumps trigger the in-process self-update on every
running client via `LARRY_BASE_URL` + `MANIFEST`. running client via `LARRY_BASE_URL` + `MANIFEST`.
## v0.9.9 — 2026-07-23
Fix: REPL leaves the terminal corrupted after exit — Backspace prints a
literal `^H` instead of deleting, and neither `stty sane` nor `reset`
recovers it (only a fresh login does). Root cause (tsk-2026-07-23-021,
live-repro'd on the Gundersen box, hci@xdcvrclvt01, v0.9.8): `main_loop`'s
"Terminal fixups" step ran `stty erase '^?'` unconditionally on every REPL
entry (a MobaXterm/Cygwin compensation for terminals that ship with erase
wrongly set to `^H`), but nothing ever restored the PRE-existing erase value
on exit — the REPL's EXIT/INT/TERM trap (`_larry_on_exit`) only tore down
mouse mode. On a terminal where erase was ALREADY correctly `^H` (i.e. the
client genuinely sends `^H` for Backspace), the blind overwrite replaced a
working setting with a wrong one and left it that way after `larry` quit.
`stty sane`/`reset` don't fix it because their own baked-in default ALSO
resolves erase to `^?` — the same wrong value larry left behind — so they
reinforce the bug rather than clear it; only a fresh pty (new login)
renegotiates the correct value. No escape-sequence-based terminal mode
(mouse tracking, bracketed paste, DECBKM, alt-screen, etc.) was involved —
mouse mode's own exit-teardown was already correct and unconditional.
Fix: `_save_tty_state`/`_restore_tty_state` now capture the FULL termios
line (`stty -g`) before the erase fixup runs and restore it verbatim from
the same exit trap that already tears down mouse mode — covering every exit
path (`/exit`, `/quit`, `/q`, EOF, an error `break`, Ctrl-C, or a killing
signal). A session that started with `erase=^H` now comes back out with
`erase=^H`; the MobaXterm/Cygwin `erase=^H`-but-key-sends-DEL case the
original fixup targets is unaffected either way. Definitive recovery for an
ALREADY-broken session (pre-fix client just exited): `stty erase '^H'`
this directly reverses the one thing larry actually changed (erase stuck at
DEL, `^?`) back to the value MobaXterm's own physical key sends (BS, `^H`
— its default "Backspace sends ^H" behavior). `stty sane` and `reset` are
NOT reliable fixes here (both re-resolve erase to the same wrong `^?`
default larry left behind) — that matches the live report exactly (both
tried, neither worked). Closing/reopening the MobaXterm tab also works
(fresh pty) but is heavier-handed than needed and loses scrollback.
Analysis: Deliverables/2026-07-23-cloverleaf-larry-tty-corruption-fix/report.md.
Fix: F-6 (tsk-2026-07-23-020) — the main scope-based regression path
(`nc-regression.sh` Phases 3/4 and the interactive `nc-engine.sh route-test`
wrapper) silently tested only message 1 of a multi-message sample file,
because neither generated a `route_test` invocation with the `-a` (all
messages) flag. Chain-walk mode already got this right. This was found by
Clover's Windows→Linux regression readiness audit for a live Bryan
regression run (Gundersen-class multi-message `.msgs` files) and would have
under-tested every thread with `--count N>1` with no warning.
- **`lib/nc-engine.sh:171`** (`cmd_route_test`) — hardcoded `-a` into the
generated `<thread> route_test -a <file>` command, mirroring the correct
chain-walk pattern (`nc-regression.sh:352`, `hciroutetest -a -d -f nl -s`).
- **`lib/nc-regression.sh:25`** and **`MANUAL.md`** — the documented default
`--route-test-cmd` example template now includes `-a`, with a comment
explaining why it's required (Phase 2 routinely samples N>1 messages/file).
Verified: **F-1** (`hl7-diff --format count` always returning 0) — audited
2026-06-08, was ALREADY fixed same-day in v0.9.2 (`9a2ed47`, "fix
F-1/F-2/F-3/F-5"). Re-confirmed here with a fresh self-test: identical pair
`count` prints `0`, exit 0; a PID-differing pair → `count` prints `2`,
exit 1. No F-1 code change was needed in this release; documented for the
record since the 2026-06-08 audit was cited as still-open.
Fix: `hl7-diff.sh` len10 framing support + F-7 (tsk-2026-07-23-len10,
Bryan-blocking) — `hciroutetest`'s DEFAULT output framing is neither
newline- nor 0x1c-delimited: each message is prefixed with a fixed
10-ASCII-digit byte-length and messages are concatenated with NO separator
between them (segments WITHIN a message remain newline-separated). Feeding
this framing through the previous nl/0x1c-only splitter silently corrupted
every message after the first. Added a new `len10` framing mode
(`looks_like_len10` auto-detect + `split_messages_len10`, seek-based via
`tail -c`/`head -c` for speed and bash-3.2/AIX/Cygwin portability) alongside
the existing `nl` and `0x1c` modes, selectable via `--framing
auto|nl|0x1c|len10` (default `auto`, tries len10 first, then 0x1c, then the
nl/MSH-block heuristic).
- **F-7** — a pre-existing (present since the v0.3.0 initial release,
unrelated to the len10 change) awk bug found while validating the len10
self-tests: the `ignored()` awk function used `n`, `i`, `ent`, `arr` without
declaring them as locals, making them true GLOBAL awk variables. `i`/`n`
collide with the outer per-message loop's own `i`/`nm` index variables, so
every call to `ignored()` that hit its early-return match branch (e.g. a
real MSH.7 timestamp difference matching the default `--ignore MSH.7` rule
— the single most common real-world hl7-diff use case) clobbered the
caller's loop index and left the outer message loop stuck re-comparing the
same message forever: a silent infinite-loop / 100%-CPU hang on any
2+-message file pair where an ignored field actually differs, not a crash,
so it was never caught by a quick eyeball check. Reproduced identically on
both `/usr/bin/awk` (BSD/onetrueawk) and gawk 5.4.1, ruling out an
implementation quirk. Fixed by declaring the 4 missing locals in the
`ignored(seg, field, comp, subc, key, key2, n, i, ent, arr)` parameter list.
Verified (self-test triad, re-run in the consolidated tree): identical
len10-framed pair → `count` prints `0`, exit 0; an MSH.7-only-differing
2+-message len10 pair → `count` prints `0`, exit 0, completes (does NOT
hang — confirms F-7 stays fixed); a real-content-differing pair → exit 1.
Analysis: Deliverables/2026-07-23-cloverleaf-larry-len10-framing.md.
## v0.9.8 — 2026-07-23 ## v0.9.8 — 2026-07-23
Fix: stale broker default + stale "LAN + Tailscale only" messaging in the Fix: stale broker default + stale "LAN + Tailscale only" messaging in the

View File

@ -23,7 +23,7 @@
# scripts/make-manifest.sh and bump VERSION. # scripts/make-manifest.sh and bump VERSION.
# Top-level scripts # Top-level scripts
larry.sh d877bf37a65e685ea27ef2503bc2dcce82cfafb255419b5d16995cee397a8cdd larry.sh 8da143113410eaf99aeea14ba12ebb13344b568d76d4711984d4c1dd75cbd4e0
larry-tunnel.sh 6b050e4eeab15669f4858eaf3b807f168f211ced07815db9521bc40a093f6aaa larry-tunnel.sh 6b050e4eeab15669f4858eaf3b807f168f211ced07815db9521bc40a093f6aaa
larry-auth.sh a220cdf7878569dc3028951ee57fc8d5e706a8ca5c6aa45347b58facb386f831 larry-auth.sh a220cdf7878569dc3028951ee57fc8d5e706a8ca5c6aa45347b58facb386f831
larry-rollback.sh 91b5e9aa6c79266bf306dcfba4ca791c07971bd6924d67a779037531648aa6d0 larry-rollback.sh 91b5e9aa6c79266bf306dcfba4ca791c07971bd6924d67a779037531648aa6d0
@ -31,9 +31,9 @@ install-larry.sh 19f42e8f6e54eaffffe33e1464a8d245721260aed047270a75987c0b25c936c
uninstall-larry.sh c53ad2d8354c7adeb243b541f027f3f481e4a8661eecfd7af14d7ca53cfcaad9 uninstall-larry.sh c53ad2d8354c7adeb243b541f027f3f481e4a8661eecfd7af14d7ca53cfcaad9
# Metadata # Metadata
VERSION ff35aa972931add85900ecabdc1eb4659e398118d3c25601d762079b1ff97dcd VERSION 050990d5e9bd0494fa852a4a9ab0d60b91c583a63307920c7887a39e692f9998
MANUAL.md 5ff54d6d5fae826f8b3da1eb3be6476076bb15f9b1417a4de285e59ea37e1b1f MANUAL.md 46cc8650bbd5daed20617dcf8626e0653da25bf2facf6781912087fcb7c43e57
CHANGELOG.md 33705f676b4069da32b3600a43697a70bd48aea03af8b7d6b72db097a7a43a72 CHANGELOG.md a14b2be47d8db54419551218df02f4d4f4428e638f5ab7de58426c3aa8c11c2e
# Agent personas (system-prompt overlays) # Agent personas (system-prompt overlays)
agents/larry.md 0a1ef737e7fc133ab35be09f79c3a4df33de814e0404b69b950932d0c8a01be1 agents/larry.md 0a1ef737e7fc133ab35be09f79c3a4df33de814e0404b69b950932d0c8a01be1
@ -74,7 +74,7 @@ lib/journal.sh 1c285df3f5677477c7b675c4f03236823005670b338a1781dd0c73d12fd1424f
# HL7 utilities # HL7 utilities
lib/hl7-sanitize.sh efd333dc3e267cf2280433e53bbb910023d0bc4b30724181b53873b410d83005 lib/hl7-sanitize.sh efd333dc3e267cf2280433e53bbb910023d0bc4b30724181b53873b410d83005
lib/hl7-desanitize.sh 2e5462a61ab1e8bd3fefb956bace8ca1ae33397a09024cbe766fa55c37a5aad6 lib/hl7-desanitize.sh 2e5462a61ab1e8bd3fefb956bace8ca1ae33397a09024cbe766fa55c37a5aad6
lib/hl7-diff.sh d2cc179bf25dd8e808d46d4211d1926f36645cec8443d0ea910675093eb89d72 lib/hl7-diff.sh 1d81163869d003bb6528818553417350c1ab885f2f04eea5d2a46a06100e789b
lib/hl7-field.sh a640f7cbd9521dc96171ee1dbdf909170262101a1d7a433f6f0ce2bea8d42b02 lib/hl7-field.sh a640f7cbd9521dc96171ee1dbdf909170262101a1d7a433f6f0ce2bea8d42b02
lib/hl7-schema.sh 2ba4057a214867ff4950f10057ee4ffd7149e1a82ba94b07b6857d77bf10d75f lib/hl7-schema.sh 2ba4057a214867ff4950f10057ee4ffd7149e1a82ba94b07b6857d77bf10d75f
@ -96,7 +96,7 @@ lib/csv-to-table.sh ca9cfbcb9f6f42bf7925bf8b8d1e25f0f7209be138d7b971e3adef3ff6b5
lib/table-to-csv.sh ad98e73687bc9e9f6ae0cd79ed5ba26c856076902865230f822dec1a1beae4b1 lib/table-to-csv.sh ad98e73687bc9e9f6ae0cd79ed5ba26c856076902865230f822dec1a1beae4b1
# NetConfig tooling # NetConfig tooling
lib/nc-engine.sh e2b12a1c019d40857b96d48d6c185b94aefadab604536ce41077ecc251b0bc58 lib/nc-engine.sh 3cf479511426c58f791016f18cdf46f0846f4746c7b1aee5f44b023ba0162c72
lib/nc-status.sh fa08c5e48704d3d17e2206dafc8522ae7668b7a2f9b97f3521e05fd0ba739443 lib/nc-status.sh fa08c5e48704d3d17e2206dafc8522ae7668b7a2f9b97f3521e05fd0ba739443
lib/nc-table.sh 4a31b49dd0106f95ed4ed33b5136eff23b178a8e339ae482a9486bfff554d2a6 lib/nc-table.sh 4a31b49dd0106f95ed4ed33b5136eff23b178a8e339ae482a9486bfff554d2a6
lib/nc-xlate.sh 8621e6f0ef55524dd6ecba91fee055cf9cdc168791e75ba7c15d9bf501fe09bf lib/nc-xlate.sh 8621e6f0ef55524dd6ecba91fee055cf9cdc168791e75ba7c15d9bf501fe09bf
@ -115,7 +115,7 @@ lib/nc-revisions.sh c27856f7decfc4c2e2c990f59eb20136fdff9cf0a52b9d9fbd9370613666
lib/nc-diff-interface.sh c922d10323f06346efa53ada68b44d32d9568ff0bd848c59af3404135f29d1ad lib/nc-diff-interface.sh c922d10323f06346efa53ada68b44d32d9568ff0bd848c59af3404135f29d1ad
lib/nc-find.sh 2264877c56100378a1b780d640dcaa806aa5501ddd204c6b6a8eb5d3e07bf966 lib/nc-find.sh 2264877c56100378a1b780d640dcaa806aa5501ddd204c6b6a8eb5d3e07bf966
lib/nc-insert-protocol.sh acc4345eface7ea77c8ea5e5bb5b2085819d48e8c7b5b2e9eec2d194a52cb046 lib/nc-insert-protocol.sh acc4345eface7ea77c8ea5e5bb5b2085819d48e8c7b5b2e9eec2d194a52cb046
lib/nc-regression.sh 70999a60608439f7bf1a3abb9f5e9854b5ea03025ef29ddbca683896346d1bce lib/nc-regression.sh 6fb3bd7c76b4b9795bcf5e7b4be8413e396db4e10926e7acabb5ccd87277e2e3
# v0.9.4: short, directly-invokable command wrappers on PATH (bin/) + dynamic # v0.9.4: short, directly-invokable command wrappers on PATH (bin/) + dynamic
# bash tab-completion. Each wrapper is a thin shim that resolves lib/ and execs # bash tab-completion. Each wrapper is a thin shim that resolves lib/ and execs

View File

@ -641,8 +641,10 @@ lib/nc-regression.sh \
--env-b /opt/cloverleaf/prod/integrator \ --env-b /opt/cloverleaf/prod/integrator \
--out /tmp/reg-2026-05-26 \ --out /tmp/reg-2026-05-26 \
--count 10 \ --count 10 \
--route-test-cmd 'cd {HCIROOT}/{HCISITE} && . ./.profile && {THREAD} route_test {INPUT} && cp *.out.* {OUTPUT_DIR}/' \ --route-test-cmd 'cd {HCIROOT}/{HCISITE} && . ./.profile && {THREAD} route_test -a {INPUT} && cp *.out.* {OUTPUT_DIR}/' \
--phase all --phase all
# NOTE: -a is required — without it only message 1 of a multi-message sample
# file is tested (F-6, tsk-2026-07-23-020). Chain-walk mode hardcodes -a already.
# Just diff existing outputs (you ran route_test manually before) # Just diff existing outputs (you ran route_test manually before)
lib/nc-regression.sh --scope site --site-a adt --site-b adt \ lib/nc-regression.sh --scope site --site-a adt --site-b adt \

View File

@ -1 +1 @@
0.9.8 0.9.9

View File

@ -100,7 +100,7 @@ set -o pipefail
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Config # Config
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
LARRY_VERSION="0.9.8" LARRY_VERSION="0.9.9"
LARRY_HOME="${LARRY_HOME:-$HOME/.larry}" LARRY_HOME="${LARRY_HOME:-$HOME/.larry}"
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
@ -6913,6 +6913,43 @@ _install_readline_tab() {
# - readline 'set enable-bracketed-paste on' (~/.inputrc). # - readline 'set enable-bracketed-paste on' (~/.inputrc).
# - MobaXterm + xterm mouse modes — known interaction with text selection: # - MobaXterm + xterm mouse modes — known interaction with text selection:
# https://forum.mobatek.net/ (search: "xterm mouse selection") # https://forum.mobatek.net/ (search: "xterm mouse selection")
# tsk-2026-07-23-021: full termios save/restore for the REPL.
#
# main_loop's "Terminal fixups" step (below, near the startup banner) runs
# `stty erase '^?'` UNCONDITIONALLY on every REPL entry, to correct terminals
# that ship with erase wrongly set to ^H while the physical key actually
# sends DEL. That fix assumes the pre-existing erase was the ^H default it
# is compensating for. When it ISN'T — e.g. a MobaXterm session that already
# had erase correctly negotiated as ^H because the client is actually
# configured to *send* ^H — the blind overwrite REPLACES a working erase
# setting with a wrong one, and nothing ever put it back: the REPL exit path
# only tore down mouse mode, never touched stty. Symptom: after `larry`
# quits, Backspace prints a literal `^H` instead of deleting. Because Linux's
# own `stty sane` / `reset` defaults ALSO resolve erase to DEL (^?) — the
# same wrong value larry just set — running `stty sane` or `reset` does NOT
# recover the terminal; only a fresh login (which renegotiates the pty from
# scratch) does. Root cause + full analysis: Deliverables/
# 2026-07-23-cloverleaf-larry-tty-corruption-fix/report.md.
#
# Fix: capture the FULL termios line with `stty -g` before main_loop makes
# ANY stty change, and restore that exact line (not just `stty sane`) from
# the same EXIT/INT/TERM trap that already tears down mouse mode — so a
# session that started with erase=^H comes back out with erase=^H, and a
# session that started with erase=^? (the case the fixup targets) is
# unaffected either way.
_LARRY_SAVED_STTY=""
_save_tty_state() {
[ -t 0 ] || return 0
command -v stty >/dev/null 2>&1 || return 0
_LARRY_SAVED_STTY=$(stty -g 2>/dev/null || true)
}
_restore_tty_state() {
[ -t 0 ] || return 0
command -v stty >/dev/null 2>&1 || return 0
if [ -n "$_LARRY_SAVED_STTY" ]; then
stty "$_LARRY_SAVED_STTY" 2>/dev/null || stty sane 2>/dev/null || true
fi
}
_LARRY_MOUSE_ACTIVE=0 _LARRY_MOUSE_ACTIVE=0
_install_mouse_mode() { _install_mouse_mode() {
# Back-compat kill switch — still a hard no. # Back-compat kill switch — still a hard no.
@ -6955,12 +6992,20 @@ _uninstall_mouse_mode() {
# block or crash the exit path. Backgrounded with a short bound so a hung # block or crash the exit path. Backgrounded with a short bound so a hung
# transport can't wedge the shutdown. # transport can't wedge the shutdown.
_larry_on_exit() { _larry_on_exit() {
# tsk-2026-07-23-021: restore the FULL termios line captured by
# _save_tty_state before main_loop's `stty erase '^?'` fixup ran. Must run
# before/independent of the mouse-mode teardown below — it undoes a
# different (termios, not escape-sequence) leaked mode. No-op if stdin
# wasn't a tty at REPL start (nothing was saved).
_restore_tty_state
_uninstall_mouse_mode _uninstall_mouse_mode
if [ -n "${LARRY_LIB_DIR:-}" ] && [ -x "$LARRY_LIB_DIR/headers-sync.sh" ]; then if [ -n "${LARRY_LIB_DIR:-}" ] && [ -x "$LARRY_LIB_DIR/headers-sync.sh" ]; then
LARRY_HOME="$LARRY_HOME" "$LARRY_LIB_DIR/headers-sync.sh" sync >/dev/null 2>&1 || true LARRY_HOME="$LARRY_HOME" "$LARRY_LIB_DIR/headers-sync.sh" sync >/dev/null 2>&1 || true
fi fi
} }
# Ensure mouse mode is disabled on REPL exit (Ctrl-C, /quit, EOF). Idempotent. # Ensure mouse mode + termios are restored on REPL exit (Ctrl-C, /quit, EOF,
# error, kill). Idempotent — both _restore_tty_state and _uninstall_mouse_mode
# are safe to call more than once or when nothing was ever changed.
trap '_larry_on_exit' EXIT INT TERM trap '_larry_on_exit' EXIT INT TERM
read_user_input() { read_user_input() {
@ -7186,10 +7231,21 @@ main_loop() {
echo "" echo ""
fi fi
# ── Terminal fixups ──────────────────────────────────────────────────────── # ── Terminal fixups ────────────────────────────────────────────────────────
# tsk-2026-07-23-021: capture the CURRENT full termios line BEFORE making
# any change below, so _larry_on_exit (trap, installed above) can put it
# back exactly on every exit path — /exit, /quit, /q, EOF, error break,
# Ctrl-C, or a killing signal. Without this the `stty erase '^?'` fixup
# below is a one-way mutation: nothing ever restored the pre-larry erase
# setting, so a terminal that already had erase correct (e.g. genuinely
# ^H because the client sends ^H) was left corrupted after larry exited —
# and because Linux's own `stty sane`/`reset` ALSO default erase to ^?,
# neither recovered it; only a fresh login (fresh pty) did.
_save_tty_state
# Some terminals (notably MobaXterm/Cygwin and certain SSH setups) ship with # Some terminals (notably MobaXterm/Cygwin and certain SSH setups) ship with
# stty erase set to ^H while the keyboard actually sends ^? (DEL) for # stty erase set to ^H while the keyboard actually sends ^? (DEL) for
# backspace, so backspace gets passed through to read() as a literal char. # backspace, so backspace gets passed through to read() as a literal char.
# Force erase=^? if we have a tty; harmless if already correct. # Force erase=^? if we have a tty; harmless if already correct. Reversible:
# _restore_tty_state (fired from the EXIT/INT/TERM trap) undoes this.
if [ -t 0 ] && command -v stty >/dev/null 2>&1; then if [ -t 0 ] && command -v stty >/dev/null 2>&1; then
stty erase '^?' 2>/dev/null || true stty erase '^?' 2>/dev/null || true
fi fi

View File

@ -7,13 +7,26 @@
# Usage: # Usage:
# hl7-diff.sh [--ignore "FIELD,FIELD,..."] [--include-fields "FIELD,..."] # hl7-diff.sh [--ignore "FIELD,FIELD,..."] [--include-fields "FIELD,..."]
# [--separator SEP] [--format text|tsv|count] # [--separator SEP] [--format text|tsv|count]
# [--framing auto|nl|0x1c|len10]
# <left_file> <right_file> # <left_file> <right_file>
# #
# Multi-message handling: # Multi-message handling:
# Files may contain multiple HL7 messages separated by either: # Files may contain multiple HL7 messages framed one of three ways:
# - len10: a fixed 10-ASCII-digit byte-length prefix immediately followed
# by "MSH", with NO delimiter between messages (this is hciroutetest's
# DEFAULT output framing — e.g. adt_in_regression_out_windows.<route>_out
# files: "0000001161MSH|...<1161 bytes total, incl. this MSH>...0000000239MSH|...").
# Segments WITHIN a message are newline-separated; the boundary between
# messages is defined solely by the byte count, never by a delimiter —
# do NOT split len10 input on newlines or 0x1c.
# - the file separator byte 0x1c (the nc_msgs --format raw default), OR # - the file separator byte 0x1c (the nc_msgs --format raw default), OR
# - blank lines between MSH-starting blocks (legacy). # - blank lines between MSH-starting blocks (legacy nl-delimited).
# The tool auto-detects. # Default --framing auto detects len10 first (leading run of exactly 10
# ASCII digits immediately followed by "MSH"), then 0x1c, then falls back to
# the nl/MSH-block heuristic. Force a specific framing with --framing if a
# file's shape could be ambiguous (e.g. a len10 file whose first message
# happens to be exactly readable as 10 digits by coincidence — vanishingly
# unlikely in practice, but the flag exists for certainty).
# #
# Defaults: # Defaults:
# --ignore MSH.7 # --ignore MSH.7
@ -45,6 +58,7 @@ die() { printf 'hl7-diff: %s\n' "$*" >&2; exit 2; }
IGNORE="MSH.7" IGNORE="MSH.7"
INCLUDE="" INCLUDE=""
FORMAT="text" FORMAT="text"
FRAMING="auto"
LEFT="" LEFT=""
RIGHT="" RIGHT=""
@ -53,7 +67,8 @@ while [ $# -gt 0 ]; do
--ignore) shift; IGNORE="$1" ;; --ignore) shift; IGNORE="$1" ;;
--include-fields) shift; INCLUDE="$1" ;; --include-fields) shift; INCLUDE="$1" ;;
--format) shift; FORMAT="$1" ;; --format) shift; FORMAT="$1" ;;
-h|--help) sed -n '2,25p' "$NC_SELF"; exit 0 ;; --framing) shift; FRAMING="$1" ;;
-h|--help) sed -n '2,33p' "$NC_SELF"; exit 0 ;;
-*) die "unknown flag: $1" ;; -*) die "unknown flag: $1" ;;
*) if [ -z "$LEFT" ]; then LEFT="$1" *) if [ -z "$LEFT" ]; then LEFT="$1"
elif [ -z "$RIGHT" ]; then RIGHT="$1" elif [ -z "$RIGHT" ]; then RIGHT="$1"
@ -65,16 +80,84 @@ done
[ -f "$LEFT" ] || die "no such left file: $LEFT" [ -f "$LEFT" ] || die "no such left file: $LEFT"
[ -f "$RIGHT" ] || die "no such right file: $RIGHT" [ -f "$RIGHT" ] || die "no such right file: $RIGHT"
case "$FORMAT" in text|tsv|count) ;; *) die "bad --format" ;; esac case "$FORMAT" in text|tsv|count) ;; *) die "bad --format" ;; esac
case "$FRAMING" in auto|nl|0x1c|len10) ;; *) die "bad --framing (want auto|nl|0x1c|len10)" ;; esac
# v0.9.10 (tsk-2026-07-23-len10, Bryan-blocking): len10 framing.
# hciroutetest's DEFAULT output framing is NOT newline- or 0x1c-delimited: each
# message is prefixed with a fixed 10-ASCII-digit byte-length, and messages are
# CONCATENATED WITH NO SEPARATOR — the next message's 10-digit prefix butts
# directly against the prior message's final byte. Segments WITHIN a message
# ARE newline-separated (converted to \r below, matching the internal
# convention the other two framings already use), but the message BOUNDARY is
# defined solely by the byte count. Splitting this on newlines or 0x1c would
# silently corrupt every message after the first.
#
# looks_like_len10: true if the first 13 bytes of a file are exactly 10 ASCII
# digits followed immediately by "MSH" (auto-detect signature).
looks_like_len10() {
local infile="$1" head13 digits msh
head13=$(LC_ALL=C head -c 13 -- "$infile" 2>/dev/null)
[ "${#head13}" -eq 13 ] || return 1
digits="${head13:0:10}"
msh="${head13:10:3}"
case "$digits" in ''|*[!0-9]*) return 1 ;; esac
[ "$msh" = "MSH" ]
}
# split_messages_len10: read 10 ASCII digits -> N (byte count) -> consume the
# next N bytes as one message -> repeat until EOF. Uses `tail -c +OFFSET |
# head -c LEN` (seek-based on regular files, not a byte-at-a-time `dd bs=1`
# loop) so it stays fast on large regression files, and stays portable to
# bash 3.2/AIX/Cygwin (no `read -N`, which is bash-4-only) per this codebase's
# existing cross-shell constraints (see lib/nc-engine.sh:53, lib/cygwin-safe.sh).
# LC_ALL=C throughout: byte-precise, immune to multi-byte locale mangling.
split_messages_len10() {
local infile="$1" outfile="$2"
local size offset digits len body_offset msg got
size=$(LC_ALL=C wc -c < "$infile" | tr -d ' ')
offset=1
: > "$outfile"
while [ "$offset" -le "$size" ]; do
digits=$(LC_ALL=C tail -c +"$offset" -- "$infile" | LC_ALL=C head -c 10)
[ "${#digits}" -eq 10 ] || {
# Fewer than 10 bytes left — trailing whitespace/EOF noise, not a new
# message. Warn only if it's non-blank (possible truncation), never die:
# a stray trailing newline is common and must not fail the whole diff.
case "$digits" in *[![:space:]]*) printf 'hl7-diff: len10: %d trailing byte(s) after last complete message in %s (ignored): %q\n' "${#digits}" "$infile" "$digits" >&2 ;; esac
break
}
case "$digits" in
*[!0-9]*) die "len10: bad length prefix '$digits' at byte offset $offset in $infile (not all-digit — file is not len10-framed, or a prior message miscounted; re-run with an explicit --framing to rule out a mis-detect)" ;;
esac
len=$((10#$digits))
body_offset=$((offset + 10))
msg=$(LC_ALL=C tail -c +"$body_offset" -- "$infile" | LC_ALL=C head -c "$len" | tr '\n' '\r')
got=$(LC_ALL=C tail -c +"$body_offset" -- "$infile" | LC_ALL=C head -c "$len" | LC_ALL=C wc -c | tr -d ' ')
[ "$got" -eq "$len" ] || printf 'hl7-diff: len10: message at offset %d claimed %d bytes but only %d were available in %s (truncated capture?)\n' "$offset" "$len" "$got" "$infile" >&2
printf '%s\x1e' "$msg" >> "$outfile"
offset=$((body_offset + len))
done
}
# Split a file into individual messages. Each MSH-block becomes one message. # Split a file into individual messages. Each MSH-block becomes one message.
# Output: one message per line, with \r preserved as 0x0d between segments, # Output: one message per line, with \r preserved as 0x0d between segments,
# messages separated by \x1e (record sep). Returns a path. # messages separated by \x1e (record sep). Returns a path.
split_messages() { split_messages() {
local infile="$1" outfile="$2" local infile="$1" outfile="$2" mode="$FRAMING"
# Try splitting by 0x1c first (raw nc_msgs format) if [ "$mode" = "auto" ]; then
if grep -q $'\x1c' "$infile" 2>/dev/null; then if looks_like_len10 "$infile"; then mode="len10"
elif grep -q $'\x1c' "$infile" 2>/dev/null; then mode="0x1c"
else mode="nl"
fi
fi
case "$mode" in
len10)
split_messages_len10 "$infile" "$outfile"
;;
0x1c)
awk -v RS=$'\x1c' 'NF>0 || $0!="" {gsub(/\n$/,""); printf "%s\x1e", $0}' "$infile" > "$outfile" awk -v RS=$'\x1c' 'NF>0 || $0!="" {gsub(/\n$/,""); printf "%s\x1e", $0}' "$infile" > "$outfile"
else ;;
nl)
# Fallback: each `MSH|` starts a new message; everything until next MSH is one message # Fallback: each `MSH|` starts a new message; everything until next MSH is one message
awk ' awk '
/^MSH\|/ { /^MSH\|/ {
@ -90,7 +173,8 @@ split_messages() {
if (msg != "") printf "%s\x1e", msg if (msg != "") printf "%s\x1e", msg
} }
' "$infile" > "$outfile" ' "$infile" > "$outfile"
fi ;;
esac
} }
# Build awk script that does the comparison. We feed it both message lists and # Build awk script that does the comparison. We feed it both message lists and
@ -105,7 +189,27 @@ split_messages "$RIGHT" "$TMP_R"
# propagate ${PIPESTATUS[0]} so awk's exit (0 identical, 1 differs) survives. # propagate ${PIPESTATUS[0]} so awk's exit (0 identical, 1 differs) survives.
awk -v IGNORE="$IGNORE" -v INCLUDE="$INCLUDE" -v FMT="$FORMAT" \ awk -v IGNORE="$IGNORE" -v INCLUDE="$INCLUDE" -v FMT="$FORMAT" \
-v LFILE="$LEFT" -v RFILE="$RIGHT" ' -v LFILE="$LEFT" -v RFILE="$RIGHT" '
function ignored(seg, field, comp, subc, key, key2) { # F-7 fix (2026-07-23, tsk-2026-07-23-len10, Bryan-blocking, discovered while
# validating len10 self-tests -- PRE-EXISTING since v0.3.0 initial release
# e08f030, unrelated to the len10 change): n, i, ent, arr were used in this
# function body but were NOT declared in the local-variable parameter list,
# so they were true GLOBAL awk variables. i and n collide with the i/nm loop
# variables the top-level END block uses to walk L_MSGS/R_MSGS. Every time
# ignored() hit the early-return match branch (the NORMAL path -- e.g. a
# real MSH.7 timestamp diff hitting the default --ignore MSH.7 rule) it left
# global i at whatever value its OWN internal for-loop had reached, silently
# resetting the outer per-message loop index in the calling scope. Net
# effect: on ANY 2+-message file pair where an ignored field actually
# differs (this is hl7-diff single most common real-world use case -- this
# is exactly the Windows-vs-Linux MSH.7 regression scenario), the outer
# message loop got stuck re-comparing the same message forever -- a silent
# infinite loop / 100 percent CPU hang, not a crash, so it was never caught
# by a quick eyeball check. Confirmed via git blame still present at HEAD
# abc0d09 (v0.9.8); reproduced identically on both /usr/bin/awk (BSD /
# onetrueawk) and gawk 5.4.1, ruling out an implementation quirk -- this is
# a genuine missing-local bug in the shipped source, fixed by declaring the
# 4 missing locals below.
function ignored(seg, field, comp, subc, key, key2, n, i, ent, arr) {
if (INCLUDE != "") { if (INCLUDE != "") {
# Inclusion mode: only fields in INCLUDE are checked # Inclusion mode: only fields in INCLUDE are checked
key = seg "." field key = seg "." field

View File

@ -168,8 +168,13 @@ cmd_resend() {
cmd_route_test() { cmd_route_test() {
local thread="$1" file="$2" local thread="$1" file="$2"
[ -n "$thread" ] && [ -f "$file" ] || die "usage: route-test <thread> <file>" [ -n "$thread" ] && [ -f "$file" ] || die "usage: route-test <thread> <file>"
local cmd="$thread route_test $file" # F-6 fix (2026-07-23, tsk-2026-07-23-020): hardcode -a so every message in
printf '\nROUTE-TEST thread=%s input=%s\n $ %s\n proceed? [y/N]: ' "$thread" "$file" "$cmd" # a multi-message input file is tested, not just the first. Without -a,
# route_test silently tests only message 1 of N (Bryan field-verified
# hciroutetest behavior, §4A of the tool-readiness audit); chain-walk mode
# (nc-regression.sh:352) already gets this right — mirror it here.
local cmd="$thread route_test -a $file"
printf '\nROUTE-TEST thread=%s input=%s (-a: all messages)\n $ %s\n proceed? [y/N]: ' "$thread" "$file" "$cmd"
read -r ans </dev/tty 2>/dev/null || ans="" read -r ans </dev/tty 2>/dev/null || ans=""
# v0.7.5: strip CR so a Cygwin pty's `Y\r` matches `^[Yy]$`. # v0.7.5: strip CR so a Cygwin pty's `Y\r` matches `^[Yy]$`.
ans="${ans//$'\r'/}" ans="${ans//$'\r'/}"

View File

@ -20,9 +20,15 @@
# Default: not set — you must pass it once for your shop's invocation pattern. # Default: not set — you must pass it once for your shop's invocation pattern.
# #
# A common pattern: a wrapper script that sources the Cloverleaf profile and # A common pattern: a wrapper script that sources the Cloverleaf profile and
# runs `<thread> route_test <INPUT>`, with output redirected to OUTPUT_DIR. # runs `<thread> route_test -a <INPUT>`, with output redirected to OUTPUT_DIR.
# Example template you might pass: # Example template you might pass:
# --route-test-cmd 'cd {HCIROOT}/{HCISITE} && . ./.profile && {THREAD} route_test {INPUT} && cp *.out.* {OUTPUT_DIR}/' # --route-test-cmd 'cd {HCIROOT}/{HCISITE} && . ./.profile && {THREAD} route_test -a {INPUT} && cp *.out.* {OUTPUT_DIR}/'
# NOTE (F-6 fix, 2026-07-23, tsk-2026-07-23-020): the -a flag is REQUIRED.
# Without it, route_test/hciroutetest silently tests only the FIRST message
# of a multi-message {INPUT} .msgs sample file (Phase 2 routinely samples
# --count N>1 messages per inbound into one file) — every subsequent message
# is never routed, never captured, never diffed, with no warning and exit 0.
# Chain-walk mode below already hardcodes -a; this template must match it.
# #
# Usage: # Usage:
# nc-regression.sh --scope <SCOPE> # nc-regression.sh --scope <SCOPE>