From 9233d23cdaeccfcbf2ced26b762ed7288d7b713d Mon Sep 17 00:00:00 2001 From: Bryan Johnson Date: Thu, 23 Jul 2026 13:22:58 -0700 Subject: [PATCH] =?UTF-8?q?v0.9.12:=20cl-format-migrate=20(table/xlate=20C?= =?UTF-8?q?RLF=E2=86=92LF=20+=20native-convert)=20+=20cdr/cds/cdd=20+=20ba?= =?UTF-8?q?re-site=20shortcuts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New `larry tools cl-format-migrate `: fixes the "' before prologue start" errors on Windows-authored tables/xlates whose CRLF line endings choke the Linux Cloverleaf parser (and Cloverleaf's own hcitblconvert/hcixltconvert). Root cause field-confirmed by Bryan: `file` shows broken=CRLF/working=LF, both version:6.0 (no format bump) — the stray CR is read as a bogus char before the prologue keyword. - Default: strip ONLY trailing CR (sed 's/\r$//'), backup-first via journal.sh, idempotent (SKIP already-LF), per-file verify (no trailing CRLF remains + prologue block intact), summary report. Bryan proved a CR-stripped table loads clean. - Opt-in --run-native-convert: chains hcitblconvert/hcixltconvert after a successful strip (Cloverleaf's real format bump), resolving the bare name against /Tables|Xlate then $HCIROOT/Tables|Xlate, one name per call; separate journal checkpoint so a native-step failure auto-rolls-back to the CR-stripped-safe state. Orphan file -> NATIVE_SKIPPED (never guesses a site). Default stays strip-only pending Bryan's hcitblconvert-post-strip confirm. - Unified tbl+xlt (both choke on CRLF identically). - QoL REPL: cdr->$HCIROOT, cds->$HCIROOT/$HCISITE, cdd->$HCIROOT/$HCISITE/data; bare site name -> setsite+cd, first-match-wins ordering so it never shadows an existing command. Vera CODE-PASS (re-gate after fixing a Major verify defect: _count_cr matched any CR byte -> false FAILED_VERIFY loop on files with a legitimate mid-line CR; now _count_trailing_cr on \r$, matching the strip). Own-fixture verified: convert/preserve-mid-line-CR/converge-to-SKIP, full regression + --run-native-convert mocked suite, bash -n 3.2.57, manifest --check clean. Live-acceptance: Bryan (Gundersen box; dry-run -> single-file --confirm pilot -> batch). --- CHANGELOG.md | 143 ++++++++++++ MANIFEST | 14 +- MANUAL.md | 49 ++++ VERSION | 2 +- bin/cl-format-migrate | 13 ++ larry.sh | 53 ++++- lib/cl-format-migrate.sh | 493 +++++++++++++++++++++++++++++++++++++++ 7 files changed, 761 insertions(+), 6 deletions(-) create mode 100755 bin/cl-format-migrate create mode 100755 lib/cl-format-migrate.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index e5e49c8..2c0395b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,149 @@ All notable changes to `cloverleaf-larry` / `larry-anywhere` are recorded here. Versioning is loose-semver; bumps trigger the in-process self-update on every running client via `LARRY_BASE_URL` + `MANIFEST`. +## v0.9.12 — 2026-07-23 + +Two independent Bryan-direct pieces of work, both from the same live session +that also produced the 2026-07-23 table/xlate format-migration design doc. + +1. **New tool: `larry tools cl-format-migrate` — CRLF->LF batch converter + for both `.tbl` AND `.xlt`, plus an opt-in chain to Cloverleaf's own + native converters.** Built on TWO rounds of Bryan-direct live evidence + from this session: + - **Round 1:** Bryan ran `file` on a broken vs. a working `.tbl` and + both are prologue `version: 6.0` — identical. The ONLY difference is + line endings (broken = CRLF, working = pure LF). + - **Round 2 (Gundersen engine box, same session):** Bryan found BOTH + native Cloverleaf converters exist in `$HCIROOT/bin` — + `hcitblconvert` (tables) and `hcixltconvert` (xlates) — and **both + fail on the CRLF files with the identical error** (`' before prologue + start` / `Please correct the error(s) and re-run.`) — the stray CR + read as a bogus char before the `prologue` keyword. So Cloverleaf's + OWN native converters choke on CRLF for the same root cause. Also + learned: neither binary has `-h`/`--help` (args are bare TABLE/XLATE + NAMES, e.g. `getwell_skipfacility_pv1_034.tbl`, not paths); resolution + is against the site's `Tables`/`Xlate` dir (`$HCIROOT/$HCISITE/Tables/ + ` then `$HCIROOT/Tables/` — mirrors `nc-table.sh`'s + existing lookup order); a bare `*` glob pulls in non-table entries and + errors, so this tool never globs at the native binary — it iterates + its own discovered file list, one name per invocation. + - `lib/cl-format-migrate.sh` (+ `bin/cl-format-migrate` on-PATH wrapper, + registered in `larry tools list`): + `cl-format-migrate [] [--type tbl|xlt|all] [--dry-run] + [--confirm yes] [--out ] [--deep-verify] + [--run-native-convert]`. `` defaults to the current directory + when omitted (same cwd-default convention as v0.9.11's + `hl7-diff-set`). + - **Step 1 (both `.tbl` and `.xlt`, the core/default step — fully + implemented, not a stub for either type anymore):** strips ONLY + trailing carriage returns (`sed 's/\r$//'` / dos2unix semantics — + never `tr -d '\r'`, which would also eat a CR that legitimately + appears mid-line). Every write goes through the existing `journal.sh` + backup-first mechanism (GL-011 invariant #3 — reused, not reinvented); + reversible via `larry-rollback.sh --entry`/`--session`. Idempotent: + files already pure-LF are SKIPPED, never touched, never counted as a + failure. Per-file VERIFY after conversion (mandatory, format-agnostic + — `.tbl`/`.xlt` share the same prologue convention): (a) zero CR bytes + remain, (b) the prologue/end_prologue/version: block is structurally + intact (same block counts pre vs. post), and (c) the actual gate — the + new file is byte-identical to the original with only trailing CRs + stripped, proving nothing else changed. A file only counts CONVERTED + if all three pass; any failure is FAILED_VERIFY (loud stop, journal + backup untouched, no silent auto-rollback — an explicit + `larry-rollback.sh` call is always the operator's decision). An + optional `--deep-verify` (`.tbl` only) additionally round-trips the + file through `table-to-csv.sh` as an EXTRA, non-gating sanity check — + nc-table.sh's known `pairs`/`lookup` bug (F-4, 2026-06-08 + tool-readiness audit) is still unfixed, so this tool does NOT depend + on it for pass/fail. + - **Step 2 (`--run-native-convert`, OPT-IN, default OFF):** after a + successful CR-strip, chain the matching native binary + (`hcitblconvert`/`hcixltconvert`, located on `$PATH` / + `$HCIROOT/bin` / `$HCIROOT/server/bin`) — "Cloverleaf's real format + bump, no format-guessing by us" — invoked with the bare filename + against a CONFIDENTLY INFERRED site (path matches `/Tables| + Xlate` with a real `NetConfig`, or the shared `$HCIROOT/Tables|Xlate` + layout; anything else is `NATIVE_SKIPPED`, never guessed). Only ever + run on a file THIS invocation just CR-stripped — an already-clean + SKIPPED file is left completely alone. A SEPARATE journal checkpoint + is taken immediately before the native call, so a native-convert + FAILURE (nonzero exit, or `before prologue`/`please correct the + error` in its output, or a truncated-empty result) auto-rolls-back + JUST that step via `larry-rollback.sh --entry` — restoring to the + already-verified CR-stripped-safe state, never leaving a file in an + uncertain post-native condition — while the CR-strip's own success is + reported and preserved independently. Kept opt-in per Larry's + explicit instruction, pending Bryan's confirmation that + `hcitblconvert` succeeds cleanly on a CR-stripped table before it + becomes the default. + - Emits a summary (converted / skipped-already-lf / failed-verify / + planned / native-converted / native-failed / native-skipped) to + stdout always, plus `report.tsv` + `report.md` when `--out DIR` is + given. Exit codes: 2 = a CR-strip verify failure occurred; 3 = a + native-convert step failed (auto-rolled-back); 2 takes priority if + both occur. + - Bash 3.2 / MobaXterm-safe throughout. Fixed one bash-3.2-only bug + found during self-test: bash 3.2's `printf` builtin misreads a format + string that itself starts with `"- "` as an (invalid) option flag + (`printf: - : invalid option`) unless `printf --` ends option parsing + first — hit in the `report.md` generator's front-matter line. + - Self-tested end-to-end with synthetic CRLF `.tbl`/`.xlt` fixtures + matching Bryan's exact reported shapes, including a mocked + `hcitblconvert`/`hcixltconvert` pair proving: a clean native-convert + success (file mutated, entry recorded), a native-convert FAILURE with + automatic rollback to the CR-stripped-safe state (verified via + `larry-rollback.sh --list`), an orphan file outside any recognizable + site layout correctly `NATIVE_SKIPPED` rather than guessed, + dry-run-with-`--run-native-convert` touching nothing, and + `--run-native-convert` without `$HCIROOT` refusing with a usage + error — plus the original round-trip/idempotent-reconvert/rollback/ + usage-error suite — all under real `/bin/bash` 3.2.57. + - **Vera-gate fix (same-day CODE-FAIL, re-gated):** verify check (a) + originally counted ANY remaining CR byte anywhere in the file + (`grep -c $'\r'`), not just a trailing CR-before-LF. The strip itself + was always correct (`sed 's/\r$//'` — strips only a trailing CRLF, + correctly PRESERVES a legitimate embedded/mid-line CR, per the tool's + own "never `tr -d '\r'`" design point). But check (a) then saw that + preserved mid-line CR and fired a FALSE `FAILED_VERIFY` — and never + converged: the file re-failed identically on every re-run (never + reached `SKIPPED`), pinning the batch exit code at 2 forever and, under + `--run-native-convert`, permanently blocking that file from the native + step (which only fires on `CONVERTED`). Did not affect Bryan's real + tables (pure trailing-CRLF, no embedded CR) but was a real correctness + bug in the verify gate. **Fix:** replaced the blanket any-CR count with + `_count_trailing_cr` (anchored on `\r$` — CR immediately before + end-of-line), used consistently for BOTH the pre-strip "does this file + need converting" decision and verify check (a), so it matches the + strip's own `\r$` semantics exactly. A file with only a legitimate + mid-line CR and no trailing CRLF is now correctly `SKIPPED` outright + (nothing for this tool to do); a file with BOTH a trailing CRLF and a + mid-line CR now converges to `CONVERTED` (mid-line CR verified + byte-for-byte preserved via `xxd`) then `SKIPPED` on every subsequent + re-run. Report-row wording updated to say "trailing-CRLF" instead of + the ambiguous "CRLF" throughout, per Vera's minor note. Re-verified the + full original self-test suite (round-trip/idempotent/rollback/ + `--run-native-convert` mocked success+failure+orphan/usage-errors) — + no regressions. + +2. **REPL QoL: `cdr`/`cds`/`cdd` cd-shortcuts + bare site-name shorthand + (Bryan-direct).** `cdr` → `cd "$HCIROOT"`, `cds` → `cd "$HCIROOT/$HCISITE"`, + `cdd` → `cd "$HCIROOT/$HCISITE/data"` (each guards against an empty + `$HCIROOT`/`$HCISITE` rather than cd-ing to a bogus path). Typing a BARE + site name at the REPL prompt (e.g. `adt`) now auto-runs the `/site adt` + equivalent, then `cd`s into it — but ONLY when the token actually + resolves to a real site (a directory under `$HCIROOT` with a `NetConfig`) + **and** isn't itself a recognized command; every existing `/command` is + tried first and always wins (the new dispatch arm's glob, `[!/]*)`, can + never match anything starting with `/`), and `cdr`/`cds`/`cdd` have their + own literal arms ahead of the generic site-name arm, so a hypothetical + site literally named `cdr` can never shadow the alias. Multi-word input + (anything containing whitespace) is explicitly excluded — ordinary + prose keeps flowing to the LLM exactly as before. Proven with a 12-case + isolated dispatch-logic harness (identical case-arm patterns) under real + bash 3.2.57, including the no-shadow proof (a same-named `cdr` site dir + does NOT hijack the alias) and the unchanged-fallthrough proof for + multi-word / non-matching bare tokens. + ## v0.9.11 — 2026-07-23 Small UX patch on top of v0.9.10's `hl7-diff-set`, both from Bryan's live diff --git a/MANIFEST b/MANIFEST index 517055f..15dc177 100644 --- a/MANIFEST +++ b/MANIFEST @@ -23,7 +23,7 @@ # scripts/make-manifest.sh and bump VERSION. # Top-level scripts -larry.sh d598cf576896504e7caf3c220a2adb8dd4ba85dd7ab6e5b23901cb4452737ef7 +larry.sh d06348673bbfe700fcb1b7871ece29863a2890cf7aa967f985551fd3f8ec0c7e larry-tunnel.sh 6b050e4eeab15669f4858eaf3b807f168f211ced07815db9521bc40a093f6aaa larry-auth.sh a220cdf7878569dc3028951ee57fc8d5e706a8ca5c6aa45347b58facb386f831 larry-rollback.sh 91b5e9aa6c79266bf306dcfba4ca791c07971bd6924d67a779037531648aa6d0 @@ -31,9 +31,9 @@ install-larry.sh 19f42e8f6e54eaffffe33e1464a8d245721260aed047270a75987c0b25c936c uninstall-larry.sh c53ad2d8354c7adeb243b541f027f3f481e4a8661eecfd7af14d7ca53cfcaad9 # Metadata -VERSION afb95cbb6514aced3e091a5d8e7e017db52ed7b7faa4dbcdce9685a8f4365c09 -MANUAL.md 88bf5f8244c5e584d889a2e9adfe54a9378475ddef927f9c8ffe2eda815186e2 -CHANGELOG.md fc2b96731ee6ba0c8d0b9c3d1c711e7318a947e72eea616a02d42ab44d0a650c +VERSION 71e12a378ec002955b38b9bdfdbc8fda0fd6c2b34a505d24d6288f1e727f5911 +MANUAL.md 750bd85fa6babd852acac69c2a0bf99b70a7b7cbb6344a21a253daa4a0c33d35 +CHANGELOG.md 7803b58ab8ffd80aacb5df5a12711194ed5272418aefcfd1bf8b418036261796 # Agent personas (system-prompt overlays) agents/larry.md 0a1ef737e7fc133ab35be09f79c3a4df33de814e0404b69b950932d0c8a01be1 @@ -95,6 +95,10 @@ lib/each-site.sh 00a31166c2cfcb610e16d4fd8bef439554b10b9e07f93f6917e5732c4ba0957 lib/len2nl.sh e84196f965ff77c49d3c8b3a776be30342f78c8b2e1c165907089d6ace908746 lib/csv-to-table.sh ca9cfbcb9f6f42bf7925bf8b8d1e25f0f7209be138d7b971e3adef3ff6b5b3dd lib/table-to-csv.sh ad98e73687bc9e9f6ae0cd79ed5ba26c856076902865230f822dec1a1beae4b1 +# v0.9.12: batch CRLF->LF format-migration for .tbl AND .xlt (journaled, +# verified, idempotent) + an opt-in --run-native-convert chain to +# hcitblconvert/hcixltconvert (auto-rollback on native-step failure). +lib/cl-format-migrate.sh c9293c4ee612e783aae57db8b24e7d6b6a2461e5876e0ca6c2d808a1b86837d2 # NetConfig tooling lib/nc-engine.sh 3cf479511426c58f791016f18cdf46f0846f4746c7b1aee5f44b023ba0162c72 @@ -190,6 +194,8 @@ bin/hl7-desanitize 78c5bda12e7374c056fac940ac1889ceca87f19a1e711dc658cc46b63b020 bin/each-site 435ed2ec2f6a48e7d15c604cebd7bb49d4744a964770682498bf12174a0e2dea bin/csv-to-table 9625f29bb874e096f0df4e19f9aa04a5beb61334d6b57f0c8b041d0e09ebf8bf bin/table-to-csv 961e45971ef3284985aedfebd5bfe601adec5273ca8ec78c9077fae83d7278ac +# v0.9.12: on-PATH wrapper for lib/cl-format-migrate.sh +bin/cl-format-migrate 93b7632fe151034933891d8d2b256245b84ebed39c69e86ef624afb9a3548217 # v0.9.4: durable checked-in demo fixture — a 3-site synthetic integrator # (epic→ancout→codamx) that exercises cross-site route fan-out + fan-in. PHI- diff --git a/MANUAL.md b/MANUAL.md index d4566e9..bc5de25 100644 --- a/MANUAL.md +++ b/MANUAL.md @@ -79,6 +79,21 @@ export HCISITE=adt export HCISITEDIR="$HCIROOT/$HCISITE" ``` +### REPL shortcuts (v0.9.12) + +Inside the `larry` REPL (not the offline `lib/` tools): + +- `cdr` → `cd "$HCIROOT"` +- `cds` → `cd "$HCIROOT/$HCISITE"` +- `cdd` → `cd "$HCIROOT/$HCISITE/data"` +- Typing a **bare site name** (e.g. `adt`) auto-runs the `/site adt` + equivalent, then `cd`s into it — but only when the token resolves to a + real site directory under `$HCIROOT` (one with a `NetConfig`) and isn't + itself a recognized command. Any existing `/command` always wins (bare + tokens never start with `/`), and `cdr`/`cds`/`cdd` always win over a + same-named site. Multi-word input (anything with a space) is never + treated as a bare site name — normal prompts keep going to the model. + --- ## Auto-update & origin @@ -858,6 +873,40 @@ OBR.10, .16, .32 / OBX.16 / DG1.3, .4 / ORC.10, .12 (orders/observation --- +## Table/xlate format migration (`lib/cl-format-migrate.sh`, v0.9.12) + +Fixes the `' before prologue start` / `Please correct the error(s) and +re-run.` parse error on old `.tbl`/`.xlt` files carrying CRLF line endings — +both Cloverleaf's own prologue parser AND its native converters +(`hcitblconvert`/`hcixltconvert`) choke on the stray CR for the same reason. + +```bash +# Preview only — writes nothing +cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt/tables --dry-run + +# Real conversion (CR-strip only — the default, proven-sufficient fix) +cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt/tables --confirm yes + +# Both .tbl and .xlt under a directory, with a machine-readable report +cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt \ + --type all --confirm yes --out /tmp/cl-format-report + +# OPT-IN: also chain Cloverleaf's own native converter after the CR-strip +# (hcitblconvert / hcixltconvert — auto-rolls-back just the native step on +# failure, keeping the already-verified CR-stripped file intact). Requires +# $HCIROOT set; the native binary must be findable on $PATH, $HCIROOT/bin, +# or $HCIROOT/server/bin. +cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt \ + --type all --confirm yes --run-native-convert +``` + +Every write is journaled (backup + diff + atomic write) exactly like every +other v3 write tool — undo with `larry-rollback.sh --entry ` or +`--session ` (printed at the end of every real run). Idempotent: an +already-pure-LF file is skipped, never touched. + +--- + ## Quick recipe: "I have to do X without internet" | Task | Command | diff --git a/VERSION b/VERSION index 8225a4b..583b27a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.9.11 +0.9.12 diff --git a/bin/cl-format-migrate b/bin/cl-format-migrate new file mode 100755 index 0000000..035b59e --- /dev/null +++ b/bin/cl-format-migrate @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +# cl-format-migrate — direct, on-PATH wrapper for lib/cl-format-migrate.sh +# (no `larry tools` prefix needed). All args pass straight through; +# -h/--help is handled by the underlying tool. +# +# cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt/tables --dry-run +# cl-format-migrate . --type tbl --confirm yes --out /tmp/cl-format-report +set -o pipefail +_self="${BASH_SOURCE[0]}"; [ -L "$_self" ] && _self="$(readlink "$_self")" +. "$(cd "$(dirname "$_self")" && pwd)/_nc_common.sh" +lib="$(_nc_resolve_lib)" || { echo "cl-format-migrate: lib/ toolkit not found (set LARRY_LIB_DIR or LARRY_HOME)" >&2; exit 1; } +if [ $# -eq 0 ]; then exec bash "$lib/cl-format-migrate.sh" --help; fi +exec bash "$lib/cl-format-migrate.sh" "$@" diff --git a/larry.sh b/larry.sh index 1d7fe69..ad96509 100755 --- a/larry.sh +++ b/larry.sh @@ -100,7 +100,7 @@ set -o pipefail # ───────────────────────────────────────────────────────────────────────────── # Config # ───────────────────────────────────────────────────────────────────────────── -LARRY_VERSION="0.9.11" +LARRY_VERSION="0.9.12" LARRY_HOME="${LARRY_HOME:-$HOME/.larry}" # ───────────────────────────────────────────────────────────────────────────── @@ -453,6 +453,7 @@ hl7-desanitize.sh|Reverse hl7-sanitize: restore original values from a token map each-site.sh|Run a command once per site under $HCIROOT (exposes $HCISITE / $HCISITEDIR) csv-to-table.sh|Convert a 2-column CSV into Cloverleaf .tbl format table-to-csv.sh|Convert a Cloverleaf .tbl file to CSV +cl-format-migrate.sh|Batch CRLF->LF format-migration for old-Cloverleaf .tbl AND .xlt files (journaled, verified, idempotent) — fixes the "' before prologue start" parse error on both. Optional --run-native-convert chains hcitblconvert/hcixltconvert on top (opt-in, auto-rollback on failure). Usage: cl-format-migrate [dir] [--type tbl|xlt|all] [--dry-run] [--confirm yes] [--out DIR] [--run-native-convert] REG } @@ -7852,6 +7853,20 @@ main_loop() { /cd\ *) local target="${input#/cd }" if cd "$target" 2>/dev/null; then larry_say "cd -> $(pwd)"; else err "no such directory: $target"; fi continue ;; + # v0.9.12 — QoL cd aliases (Bryan-direct). Guard against empty + # $HCIROOT/$HCISITE rather than cd-ing to a bogus/empty path. + cdr) if [ -z "${HCIROOT:-}" ]; then err "cdr: \$HCIROOT is not set" + elif cd "$HCIROOT" 2>/dev/null; then larry_say "cd -> $(pwd)" + else err "no such directory: $HCIROOT"; fi + continue ;; + cds) if [ -z "${HCIROOT:-}" ] || [ -z "${HCISITE:-}" ]; then err "cds: \$HCIROOT/\$HCISITE not both set" + elif cd "$HCIROOT/$HCISITE" 2>/dev/null; then larry_say "cd -> $(pwd)" + else err "no such directory: $HCIROOT/$HCISITE"; fi + continue ;; + cdd) if [ -z "${HCIROOT:-}" ] || [ -z "${HCISITE:-}" ]; then err "cdd: \$HCIROOT/\$HCISITE not both set" + elif cd "$HCIROOT/$HCISITE/data" 2>/dev/null; then larry_say "cd -> $(pwd)" + else err "no such directory: $HCIROOT/$HCISITE/data"; fi + continue ;; /load\ *) local f="${input#/load }" if [ ! -f "$f" ]; then err "no such file: $f"; continue; fi input="$(cat "$f")" @@ -8001,6 +8016,42 @@ EOF ) larry_say "/nc-regression-env: templated prompt prepared for $_src → $_tgt (scope=$_scope, out=$_out)" ;; + # v0.9.12 — bare site-name shorthand (Bryan-direct). A single-word, + # non-slash token (e.g. `adt`) that resolves to a real site directory + # under $HCIROOT (has a NetConfig) auto-runs the equivalent of + # `/site ` then cd's into it. Deliberately conservative: + # - `[!/]*)` only matches non-slash input, so it can NEVER shadow any + # /command arm above (those are tried first and win on a literal + # match regardless of where this arm sits in the case). + # - `cdr`/`cds`/`cdd` are matched by their OWN literal arms above, so + # they always win before falling through to here — no shadowing. + # - Multi-word input (anything with a space) is explicitly excluded + # — "explain the adt site" must keep going to the LLM, not be + # mistaken for a site name. + # - If $HCIROOT/$HCISITE aren't set, or the token isn't a real site + # dir (no matching NetConfig), NOTHING happens here — the arm + # falls through with no `continue`, so execution proceeds exactly + # as it did before this feature existed (normal LLM-prompt path). + [!/]*) + case "$input" in + *[[:space:]]*) : ;; # multi-word -> never a bare site name + *) + if [ -n "${HCIROOT:-}" ] && [ -d "${HCIROOT}/${input}" ] && [ -f "${HCIROOT}/${input}/NetConfig" ]; then + HCISITE="$input"; HCISITEDIR="$HCIROOT/$HCISITE" + export HCISITE HCISITEDIR + detect_cloverleaf_env + system_prompt=$(build_system_prompt) + if cd "$HCISITEDIR" 2>/dev/null; then + larry_say "HCISITE -> $HCISITE ($HCISITEDIR); cd -> $(pwd)" + else + larry_say "HCISITE -> $HCISITE ($HCISITEDIR)" + err "cd failed: $HCISITEDIR" + fi + continue + fi + ;; + esac + ;; /*) err "unknown command: $input (try /help)"; continue ;; esac diff --git a/lib/cl-format-migrate.sh b/lib/cl-format-migrate.sh new file mode 100755 index 0000000..431f62b --- /dev/null +++ b/lib/cl-format-migrate.sh @@ -0,0 +1,493 @@ +#!/usr/bin/env bash +# cl-format-migrate.sh — CRLF->LF format-migration for Cloverleaf .tbl AND +# .xlt files, plus an OPT-IN chain to Cloverleaf's own native converters +# (hcitblconvert / hcixltconvert). Per the 2026-07-23 design doc +# (Deliverables/2026-07-23-cloverleaf-table-xlate-format-migration-design.md) +# and TWO rounds of Bryan-direct live evidence from the same session: +# +# ROUND 1 (the original trigger): `file` on a broken vs. a working .tbl +# showed BOTH are prologue `version: 6.0` (identical) — the ONLY +# difference is line endings: +# getwell_skipfacility_pv1_034.tbl -> ASCII text, WITH CRLF (broken) +# getwellntw_sndingevent_msh04.tbl -> ASCII text (working) +# +# ROUND 2 (Gundersen engine box, same session): Bryan found BOTH native +# Cloverleaf converters exist in $HCIROOT/bin — `hcitblconvert` (tables) +# and `hcixltconvert` (xlates) — and BOTH FAIL on the CRLF files with the +# IDENTICAL error: +# $ hcitblconvert getwell_skipfacility_pv1_034.tbl +# getwell_skipfacility_pv1_034.tbl +# ' before prologue start +# $ hcixltconvert +# ' before prologue start +# Please correct the error(s) and re-run. +# That leading `'` is the stray CR being read as a bogus char before the +# `prologue` keyword — Cloverleaf's OWN native converters choke on CRLF +# for the SAME root-cause reason our own parser does. Also learned: +# - No -h/--help flag on either binary; args are TABLE/XLATE NAMES +# (e.g. `getwell_skipfacility_pv1_034.tbl`, WITH extension), not full +# paths. Resolution is against the site's Tables/Xlate dir: +# $HCIROOT/$HCISITE/Tables/ then $HCIROOT/Tables/ (tables; +# nc-table.sh's existing lookup order) — Xlate mirrors this with +# Xlate/xlate (nc-xlate.sh's existing lookup order). +# - A bare `*` glob pulls in non-table entries and errors — so this +# tool NEVER globs at the native binary; it iterates its OWN +# discovered file list and invokes the binary once per name. +# +# DESIGN (unified, per Larry's 2026-07-23 follow-up dispatch): +# Step 1 (both .tbl and .xlt, ALWAYS the core/default step): strip ONLY +# trailing carriage returns (`sed 's/\r$//'` / dos2unix semantics — +# never `tr -d '\r'`). Journaled (backup+diff+atomic write), verified, +# idempotent (already-pure-LF files are SKIPPED). This alone is +# PROVEN (Round 1) to make a table load. +# Step 2 (OPT-IN via --run-native-convert, default OFF): chain the +# matching native binary (hcitblconvert for .tbl, hcixltconvert for +# .xlt) on top of the CR-stripped file, so Cloverleaf's OWN converter +# does whatever real format-level work it does (metadata/version +# re-serialization) — "no format-guessing by us." Only ever run on a +# file THIS invocation just CR-stripped (never on an already-clean +# SKIPPED file — untouched files stay untouched). Kept opt-in until +# Bryan confirms hcitblconvert succeeds cleanly on a CR-stripped table +# (per Larry's explicit instruction — do NOT make this the default +# yet). A pre-native checkpoint is journaled separately from the CR- +# strip entry, so a native-convert FAILURE auto-rolls-back to the +# already-verified CR-stripped-only state (never leaves a file in an +# uncertain post-native state) while the CR-strip's own success is +# preserved and reported independently. +# +# Usage: +# cl-format-migrate.sh [] [--type tbl|xlt|all] [--dry-run] +# [--confirm yes] [--out ] [--deep-verify] +# [--run-native-convert] +# +# directory to scan RECURSIVELY for *.tbl / *.xlt. +# DEFAULTS TO THE CURRENT DIRECTORY (.) if omitted — +# same cwd-default convention as v0.9.11 hl7-diff-set. +# --type T tbl (default) | xlt | all +# --dry-run classify + report what WOULD happen; write NOTHING +# --confirm yes required to actually write on a REAL (non-dry-run) +# conversion. Without it: plan-only refusal per file, +# consistent with nc-set-field.sh / nc-provision-jumps.sh. +# --out DIR write report.tsv + report.md into DIR (created if +# needed). Always ALSO prints the summary to stdout. +# --deep-verify (.tbl only) additionally round-trip the converted +# file through table-to-csv.sh — informational only, +# NEVER gates pass/fail (nc-table.sh F-4 is unfixed). +# --run-native-convert OPT-IN, default OFF. After a successful CR-strip, +# also chain the matching native Cloverleaf converter +# (hcitblconvert / hcixltconvert), located on $PATH / +# $HCIROOT/bin / $HCIROOT/server/bin, invoked with the +# bare NAME (not a path, not a glob) against the +# inferred site. Requires $HCIROOT set. A file whose +# site can't be confidently inferred (path doesn't +# match the /Tables|Xlate or shared +# $HCIROOT/Tables|Xlate layout) is reported +# NATIVE_SKIPPED, not silently guessed at. +# +# Exit codes: +# 0 all good (including a clean --dry-run / all-already-LF run) +# 1 usage/input error +# 2 at least one file FAILED_VERIFY on the CR-strip step +# 3 at least one file's --run-native-convert step FAILED (auto-rolled-back +# to the CR-stripped-safe state — see NATIVE_FAILED in the report). +# Exit 2 takes priority if BOTH occur. +set -o pipefail + +NC_SELF="$0" +LIB_DIR="$(cd "$(dirname "$NC_SELF")" && pwd)" +JOURNAL="$LIB_DIR/journal.sh" +T2C="$LIB_DIR/table-to-csv.sh" +NC_ROLLBACK="$LIB_DIR/../larry-rollback.sh" +[ -f "$NC_ROLLBACK" ] || NC_ROLLBACK="$(command -v larry-rollback.sh 2>/dev/null || printf '%s' "$NC_ROLLBACK")" + +die() { printf 'cl-format-migrate: %s\n' "$*" >&2; exit 1; } +warn() { printf 'cl-format-migrate: %s\n' "$*" >&2; } + +# Shared CR-safety primitives (coerce_int et al) — same pattern as every +# other lib/ tool. Inline fallback if a partial install is missing it. +if [ -r "$LIB_DIR/cygwin-safe.sh" ]; then + # shellcheck disable=SC1090,SC1091 + . "$LIB_DIR/cygwin-safe.sh" +else + coerce_int() { local r="${1:-}" d="${2:-0}" c; c=$(printf '%s' "$r" | tr -cd '0-9'); printf '%s' "${c:-$d}"; } +fi + +# journal.sh enables `set -u`; restore the assumed (unset-tolerant) env after +# sourcing, same as nc-engine.sh. +[ -f "$JOURNAL" ] && . "$JOURNAL" || warn "journal.sh not available — writes will NOT be journaled/reversible; refusing to run real conversions." +set +u + +usage() { sed -n '2,93p' "$NC_SELF"; } + +DIR="." +TYPE="tbl" +DRY_RUN=0 +CONFIRM="" +OUT_DIR="" +DEEP_VERIFY=0 +RUN_NATIVE=0 +_dir_set=0 + +while [ $# -gt 0 ]; do + case "$1" in + --type) shift; TYPE="$1" ;; + --dry-run) DRY_RUN=1 ;; + --confirm) shift; CONFIRM="${1:-}" ;; + --out) shift; OUT_DIR="$1" ;; + --deep-verify) DEEP_VERIFY=1 ;; + --run-native-convert) RUN_NATIVE=1 ;; + -h|--help) usage; exit 0 ;; + -*) die "unknown flag: $1 (run --help)" ;; + *) + if [ "$_dir_set" = "0" ]; then DIR="$1"; _dir_set=1 + else die "extra arg: $1" + fi + ;; + esac + shift +done + +case "$TYPE" in tbl|xlt|all) ;; *) die "bad --type (want tbl|xlt|all)" ;; esac +[ -d "$DIR" ] || die "no such directory: $DIR" +DIR="$(cd "$DIR" && pwd)" +[ "$RUN_NATIVE" = "1" ] && [ -z "${HCIROOT:-}" ] && die "--run-native-convert requires \$HCIROOT to be set" + +# ── report plumbing ────────────────────────────────────────────────────────── +REPORT_TSV="$(mktemp)" +printf 'path\tstatus\treason\tentry_id\n' > "$REPORT_TSV" + +CONVERTED=0 +SKIPPED=0 +FAILED=0 +PLANNED=0 +NATIVE_CONVERTED=0 +NATIVE_FAILED=0 +NATIVE_SKIPPED=0 + +_report_row() { + # path status reason entry_id + printf '%s\t%s\t%s\t%s\n' "$1" "$2" "$3" "${4:-}" >> "$REPORT_TSV" + printf '%-16s %s%s\n' "$2" "$1" "$( [ -n "${3:-}" ] && printf ' — %s' "$3" )" +} + +# _count_trailing_cr FILE — count lines whose LAST byte before the newline is +# a CR (i.e. CRLF line endings — exactly what `sed 's/\r$//'` targets). +# +# v0.9.12 Vera-gate fix: this MUST match the strip's own `\r$` semantics, not +# "any CR byte anywhere" (a plain `grep -c $'\r'`). A blanket any-CR count +# also matches a legitimate mid-line/embedded CR that the strip correctly +# and deliberately PRESERVES (the tool's whole "never tr -d '\r'" design +# point) — which made the old check count that preserved byte as a failure, +# forever, on every re-run (never converges to SKIPPED). Anchoring the grep +# on `\r$` (CR immediately before end-of-line) only matches an actual CRLF +# line ending, so a file with just an embedded CR and no trailing CRLF +# correctly reads as 0 here — both for the pre-strip "does this file need +# converting" decision and for verify check (a) below. +_count_trailing_cr() { + local f="$1" n + n=$(LC_ALL=C grep -c $'\r$' "$f" 2>/dev/null) + coerce_int "$n" 0 +} + +_find_candidates() { + local dir="$1" type="$2" + case "$type" in + tbl) find "$dir" -type f -iname '*.tbl' 2>/dev/null ;; + xlt) find "$dir" -type f -iname '*.xlt' 2>/dev/null ;; + all) find "$dir" -type f \( -iname '*.tbl' -o -iname '*.xlt' \) 2>/dev/null ;; + esac | LC_ALL=C sort +} + +# ── CR-strip verify (the mandatory gate; format-agnostic — .tbl and .xlt +# share the same prologue/end_prologue/version convention, confirmed in +# the design doc §2 nc-revisions.sh cross-check) ────────────────────────── +_verify_crlf_strip() { + local orig_copy="$1" new_file="$2" + local cr; cr=$(_count_trailing_cr "$new_file") + if [ "$cr" != "0" ]; then + printf '(a) FAILED: %s trailing-CRLF line(s) still present after conversion' "$cr" + return 1 + fi + + local pro_pre pro_post epro_pre epro_post ver_pre ver_post + pro_pre=$(coerce_int "$(grep -c '^[[:space:]]*prologue[[:space:]]*$' "$orig_copy" 2>/dev/null)" 0) + pro_post=$(coerce_int "$(grep -c '^[[:space:]]*prologue[[:space:]]*$' "$new_file" 2>/dev/null)" 0) + epro_pre=$(coerce_int "$(grep -c 'end_prologue' "$orig_copy" 2>/dev/null)" 0) + epro_post=$(coerce_int "$(grep -c 'end_prologue' "$new_file" 2>/dev/null)" 0) + ver_pre=$(coerce_int "$(grep -c 'version:' "$orig_copy" 2>/dev/null)" 0) + ver_post=$(coerce_int "$(grep -c 'version:' "$new_file" 2>/dev/null)" 0) + if [ "$pro_pre" != "$pro_post" ] || [ "$epro_pre" != "$epro_post" ] || [ "$ver_pre" != "$ver_post" ]; then + printf '(b) FAILED: prologue/end_prologue/version block count changed (pre %s/%s/%s -> post %s/%s/%s)' \ + "$pro_pre" "$epro_pre" "$ver_pre" "$pro_post" "$epro_post" "$ver_post" + return 1 + fi + + # (c) the actual gate: new file == CR-stripped original, byte for byte. + if ! diff -q <(sed $'s/\r$//' "$orig_copy") "$new_file" >/dev/null 2>&1; then + printf '(c) FAILED: content differs by more than trailing-CR removal (not a pure CRLF->LF pass)' + return 1 + fi + return 0 +} + +# ── native-binary lookup ────────────────────────────────────────────────────── +_resolve_native_bin() { + local name="$1" + if command -v "$name" >/dev/null 2>&1; then command -v "$name"; return 0; fi + local d + for d in "${HCIROOT:-}/bin" "${HCIROOT:-}/server/bin"; do + [ -n "$d" ] && [ -x "$d/$name" ] && { printf '%s' "$d/$name"; return 0; } + done + return 1 +} + +# _infer_site FILE PARENTNAME1 PARENTNAME2 — infer the site name from a path +# shaped like ...///, requiring the grandparent +# to actually be a site dir (has a NetConfig) so this is never a guess. +# Prints the site name on stdout and returns 0, or returns 1 (no site — could +# still be the shared $HCIROOT/Tables|Xlate layout, handled by the caller). +_infer_site() { + local f="$1" p1="$2" p2="$3" + local parent grandparent pbase + parent="$(dirname "$f")" + pbase="$(basename "$parent")" + case "$pbase" in + "$p1"|"$p2") + grandparent="$(dirname "$parent")" + if [ -n "${HCIROOT:-}" ] && [ -f "$grandparent/NetConfig" ]; then + basename "$grandparent"; return 0 + fi + ;; + esac + return 1 +} + +# ── native-convert step (opt-in; only ever called on a file THIS run just +# CR-stripped) ───────────────────────────────────────────────────────────── +_native_convert() { + local file="$1" kind="$2" # kind = tbl | xlt + local binname parent1 parent2 + if [ "$kind" = "tbl" ]; then binname="hcitblconvert"; parent1="Tables"; parent2="tables" + else binname="hcixltconvert"; parent1="Xlate"; parent2="xlate" + fi + + local bin; bin=$(_resolve_native_bin "$binname") || { + _report_row "$file" "NATIVE_SKIPPED" "$binname not found on \$PATH, \$HCIROOT/bin, or \$HCIROOT/server/bin" "" + NATIVE_SKIPPED=$((NATIVE_SKIPPED + 1)); return 0 + } + + local site="" + if ! site=$(_infer_site "$file" "$parent1" "$parent2"); then + # Not under a /Tables|Xlate layout. Allow the shared-root layout + # ($HCIROOT/Tables or $HCIROOT/Xlate, no site subdir) with an empty site; + # anything else is a "can't confidently resolve" skip, never a guess. + local parentdir; parentdir="$(dirname "$file")" + case "$parentdir" in + "${HCIROOT:-__no_hciroot__}/$parent1"|"${HCIROOT:-__no_hciroot__}/$parent2") site="" ;; + *) + _report_row "$file" "NATIVE_SKIPPED" "cannot confidently infer site for native convert (path doesn't match /${parent1}|${parent2} or shared \$HCIROOT/${parent1}|${parent2})" "" + NATIVE_SKIPPED=$((NATIVE_SKIPPED + 1)); return 0 + ;; + esac + fi + + # Checkpoint the CR-stripped (already-verified-safe) state as its OWN + # journal entry, distinct from the CR-strip entry, so a native-convert + # failure rolls back ONLY the native step. + local precopy entry2 + precopy="$(mktemp)"; cp -p "$file" "$precopy" || { + _report_row "$file" "NATIVE_FAILED" "could not snapshot pre-native state" "" + NATIVE_FAILED=$((NATIVE_FAILED + 1)); rm -f "$precopy"; return 1 + } + entry2=$(journal_write "$file" "$precopy" 2>>"$JOURNAL_LOG") + local jw2_rc=$? + rm -f "$precopy" + if [ "$jw2_rc" != "0" ] || [ -z "$entry2" ]; then + _report_row "$file" "NATIVE_FAILED" "pre-native journal checkpoint failed (rc=$jw2_rc) — native convert NOT attempted" "" + NATIVE_FAILED=$((NATIVE_FAILED + 1)); return 1 + fi + + local base out rc + base="$(basename "$file")" + out=$( (cd "$(dirname "$file")" && HCIROOT="$HCIROOT" HCISITE="$site" "$bin" "$base") 2>&1 ) + rc=$? + + local fail_reason="" + if [ "$rc" != "0" ]; then + fail_reason="exit code $rc" + elif printf '%s' "$out" | grep -qi "before prologue\|please correct the error"; then + fail_reason="native binary reported a parse error" + elif [ ! -s "$file" ]; then + fail_reason="file is empty/truncated after native convert" + fi + + if [ -n "$fail_reason" ]; then + _report_row "$file" "NATIVE_FAILED" "$fail_reason (auto-rolled-back to CR-stripped-safe state) — output: $(printf '%s' "$out" | tr '\n' ' ' | cut -c1-200)" "$entry2" + NATIVE_FAILED=$((NATIVE_FAILED + 1)) + if [ -x "$NC_ROLLBACK" ] || [ -f "$NC_ROLLBACK" ]; then + bash "$NC_ROLLBACK" --entry "$entry2" --yes >>"$JOURNAL_LOG" 2>&1 \ + || warn "auto-rollback of $entry2 FAILED — $file may be in an uncertain state; run: larry-rollback.sh --entry $entry2" + else + warn "larry-rollback.sh not found — could not auto-rollback $entry2; $file may be in an uncertain state" + fi + return 1 + fi + + _report_row "$file" "NATIVE_CONVERTED" "$binname OK (site=${site:-}); entry $entry2; rollback: larry-rollback.sh --entry $entry2" "$entry2" + NATIVE_CONVERTED=$((NATIVE_CONVERTED + 1)) + return 0 +} + +# ── one .tbl/.xlt file — CR-strip step (shared for both types) ────────────── +_process_file() { + local file="$1" kind="$2" # kind = tbl | xlt + # v0.9.12 Vera-gate fix: trailing-CR-only detection (matches the strip's own + # `\r$` semantics) — a file whose only CR bytes are legitimate mid-line ones + # (no trailing CRLF) has nothing for this tool to do, and is correctly + # SKIPPED rather than endlessly re-attempted/re-failed. See _count_trailing_cr. + local cr; cr=$(_count_trailing_cr "$file") + + if [ "$cr" = "0" ]; then + _report_row "$file" "SKIPPED" "already pure-LF (no trailing CRLF found)" "" + SKIPPED=$((SKIPPED + 1)) + return 0 + fi + + if [ "$DRY_RUN" = "1" ]; then + _report_row "$file" "DRY_RUN" "$cr trailing-CRLF line(s) — would convert (re-run with --confirm yes)$( [ "$RUN_NATIVE" = "1" ] && printf '; would also attempt native convert' )" "" + PLANNED=$((PLANNED + 1)) + return 0 + fi + if [ "$CONFIRM" != "yes" ]; then + _report_row "$file" "REFUSED" "$cr trailing-CRLF line(s) — no --confirm yes, nothing written" "" + PLANNED=$((PLANNED + 1)) + return 0 + fi + if ! declare -f journal_write >/dev/null 2>&1; then + _report_row "$file" "FAILED_VERIFY" "journal.sh not available — refusing to write unjournaled" "" + FAILED=$((FAILED + 1)) + return 1 + fi + + local orig_copy newtmp + orig_copy="$(mktemp)"; newtmp="$(mktemp)" + if ! cp -p "$file" "$orig_copy"; then + _report_row "$file" "FAILED_VERIFY" "could not snapshot original for verify" "" + FAILED=$((FAILED + 1)); rm -f "$orig_copy" "$newtmp"; return 1 + fi + if ! sed $'s/\r$//' "$file" > "$newtmp"; then + _report_row "$file" "FAILED_VERIFY" "sed CRLF-strip failed" "" + FAILED=$((FAILED + 1)); rm -f "$orig_copy" "$newtmp"; return 1 + fi + + local entry_id jw_rc + entry_id=$(journal_write "$file" "$newtmp" 2>>"$JOURNAL_LOG") + jw_rc=$? + if [ "$jw_rc" != "0" ] || [ -z "$entry_id" ]; then + _report_row "$file" "FAILED_VERIFY" "journal_write failed (rc=$jw_rc) — file NOT modified" "" + FAILED=$((FAILED + 1)); rm -f "$orig_copy" "$newtmp"; return 1 + fi + + local reason + if reason=$(_verify_crlf_strip "$orig_copy" "$file"); then + local deep_note="" + if [ "$kind" = "tbl" ] && [ "$DEEP_VERIFY" = "1" ] && [ -f "$T2C" ]; then + if "$T2C" --with-header "$file" >/dev/null 2>&1; then + deep_note=" | deep-verify(table-to-csv): OK" + else + deep_note=" | deep-verify(table-to-csv): FAILED (informational only — F-4 known bug, not gating)" + fi + fi + _report_row "$file" "CONVERTED" "$cr trailing-CRLF line(s) stripped; entry $entry_id; rollback: larry-rollback.sh --entry $entry_id${deep_note}" "$entry_id" + CONVERTED=$((CONVERTED + 1)) + rm -f "$orig_copy" "$newtmp" + if [ "$RUN_NATIVE" = "1" ]; then + _native_convert "$file" "$kind" + fi + else + _report_row "$file" "FAILED_VERIFY" "$reason — entry $entry_id; rollback: larry-rollback.sh --entry $entry_id" "$entry_id" + FAILED=$((FAILED + 1)) + rm -f "$orig_copy" "$newtmp" + fi +} + +# ── SESSION (one for the whole batch) ──────────────────────────────────────── +SESSION="${LARRY_SESSION_ID:-clfmt-$(date +%Y-%m-%d-%H%M%S)-$$}" +export LARRY_SESSION_ID="$SESSION" +export LARRY_HOME="${LARRY_HOME:-$HOME/.larry}" +JOURNAL_LOG="$(mktemp)" + +printf '═══════════════════════════════════════════════════════════════════════\n' +printf 'cl-format-migrate — dir=%s type=%s dry-run=%s confirm=%s run-native-convert=%s\n' \ + "$DIR" "$TYPE" "$( [ "$DRY_RUN" = "1" ] && echo yes || echo no )" "$( [ "$CONFIRM" = "yes" ] && echo yes || echo no )" \ + "$( [ "$RUN_NATIVE" = "1" ] && echo yes || echo no )" +printf 'journal session: %s\n' "$SESSION" +printf '═══════════════════════════════════════════════════════════════════════\n' + +if [ "$TYPE" = "tbl" ] || [ "$TYPE" = "all" ]; then + printf '\n-- tables (.tbl) --\n' + _any=0 + while IFS= read -r f; do + [ -n "$f" ] || continue + _any=1 + _process_file "$f" tbl + done < <(_find_candidates "$DIR" tbl) + [ "$_any" = "1" ] || printf '(no .tbl files found under %s)\n' "$DIR" +fi + +if [ "$TYPE" = "xlt" ] || [ "$TYPE" = "all" ]; then + printf '\n-- xlates (.xlt) --\n' + _any=0 + while IFS= read -r f; do + [ -n "$f" ] || continue + _any=1 + _process_file "$f" xlt + done < <(_find_candidates "$DIR" xlt) + [ "$_any" = "1" ] || printf '(no .xlt files found under %s)\n' "$DIR" +fi + +# ── summary ─────────────────────────────────────────────────────────────────── +printf '\n───────────────────────────────────────────────────────────────────────\n' +printf 'SUMMARY: converted=%d skipped-already-lf=%d failed-verify=%d planned=%d native-converted=%d native-failed=%d native-skipped=%d\n' \ + "$CONVERTED" "$SKIPPED" "$FAILED" "$PLANNED" "$NATIVE_CONVERTED" "$NATIVE_FAILED" "$NATIVE_SKIPPED" +if [ "$CONVERTED" -gt 0 ] || [ "$NATIVE_CONVERTED" -gt 0 ]; then + printf 'WHOLE-BATCH ROLLBACK (all writes this run): larry-rollback.sh --session %s\n' "$SESSION" +fi + +if [ -n "$OUT_DIR" ]; then + mkdir -p "$OUT_DIR" 2>/dev/null || warn "could not create --out dir: $OUT_DIR" + if [ -d "$OUT_DIR" ]; then + cp "$REPORT_TSV" "$OUT_DIR/report.tsv" + { + printf '# cl-format-migrate report\n\n' + # v0.9.12: `printf --` — bash 3.2's printf builtin misreads a format + # string that itself starts with "- " as an (invalid) option flag + # ("printf: - : invalid option") unless "--" ends option parsing first. + printf -- '- dir: `%s`\n- type: %s\n- dry-run: %s\n- confirm: %s\n- run-native-convert: %s\n- session: %s\n- generated: %s\n\n' \ + "$DIR" "$TYPE" "$( [ "$DRY_RUN" = "1" ] && echo yes || echo no )" \ + "$( [ "$CONFIRM" = "yes" ] && echo yes || echo no )" "$( [ "$RUN_NATIVE" = "1" ] && echo yes || echo no )" \ + "$SESSION" "$(date -Iseconds 2>/dev/null || date)" + printf '## Summary\n\n' + printf '| converted | skipped-already-lf | failed-verify | planned | native-converted | native-failed | native-skipped |\n' + printf '|---|---|---|---|---|---|---|\n' + printf '| %d | %d | %d | %d | %d | %d | %d |\n\n' \ + "$CONVERTED" "$SKIPPED" "$FAILED" "$PLANNED" "$NATIVE_CONVERTED" "$NATIVE_FAILED" "$NATIVE_SKIPPED" + printf '## Per-file detail\n\n' + printf '| path | status | reason | entry_id |\n|---|---|---|---|\n' + tail -n +2 "$REPORT_TSV" | while IFS=$'\t' read -r p s r e; do + printf '| %s | %s | %s | %s |\n' "$p" "$s" "$r" "$e" + done + } > "$OUT_DIR/report.md" + printf 'report written: %s/report.tsv , %s/report.md\n' "$OUT_DIR" "$OUT_DIR" + fi +fi + +rm -f "$REPORT_TSV" "$JOURNAL_LOG" + +if [ "$FAILED" -gt 0 ]; then + exit 2 +fi +if [ "$NATIVE_FAILED" -gt 0 ]; then + exit 3 +fi +exit 0