bryan/cloverleaf-larry
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.9.12: cl-format-migrate (table/xlate CRLF→LF + native-convert) + cdr/cds/cdd + bare-site shortcuts

Browse Source 
New `larry tools cl-format-migrate <dir>`: fixes the "' before prologue start" errors on
Windows-authored tables/xlates whose CRLF line endings choke the Linux Cloverleaf parser
(and Cloverleaf's own hcitblconvert/hcixltconvert). Root cause field-confirmed by Bryan:
`file` shows broken=CRLF/working=LF, both version:6.0 (no format bump) — the stray CR is
read as a bogus char before the prologue keyword.

- Default: strip ONLY trailing CR (sed 's/\r$//'), backup-first via journal.sh, idempotent
  (SKIP already-LF), per-file verify (no trailing CRLF remains + prologue block intact),
  summary report. Bryan proved a CR-stripped table loads clean.
- Opt-in --run-native-convert: chains hcitblconvert/hcixltconvert after a successful strip
  (Cloverleaf's real format bump), resolving the bare name against <site>/Tables|Xlate then
  $HCIROOT/Tables|Xlate, one name per call; separate journal checkpoint so a native-step
  failure auto-rolls-back to the CR-stripped-safe state. Orphan file -> NATIVE_SKIPPED (never
  guesses a site). Default stays strip-only pending Bryan's hcitblconvert-post-strip confirm.
- Unified tbl+xlt (both choke on CRLF identically).
- QoL REPL: cdr->$HCIROOT, cds->$HCIROOT/$HCISITE, cdd->$HCIROOT/$HCISITE/data; bare site name
  -> setsite+cd, first-match-wins ordering so it never shadows an existing command.

Vera CODE-PASS (re-gate after fixing a Major verify defect: _count_cr matched any CR byte ->
false FAILED_VERIFY loop on files with a legitimate mid-line CR; now _count_trailing_cr on
\r$, matching the strip). Own-fixture verified: convert/preserve-mid-line-CR/converge-to-SKIP,
full regression + --run-native-convert mocked suite, bash -n 3.2.57, manifest --check clean.
Live-acceptance: Bryan (Gundersen box; dry-run -> single-file --confirm pilot -> batch).
This commit is contained in:
Bryan Johnson 2026-07-23 13:22:58 -07:00
parent 0046e88a8d
commit 9233d23cda
7 changed files with 761 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

143
CHANGELOG.md
View File

@ -4,6 +4,149 @@ All notable changes to `cloverleaf-larry` / `larry-anywhere` are recorded here.
Versioning is loose-semver; bumps trigger the in-process self-update on every
running client via `LARRY_BASE_URL` + `MANIFEST`.
## v0.9.12 — 2026-07-23
Two independent Bryan-direct pieces of work, both from the same live session
that also produced the 2026-07-23 table/xlate format-migration design doc.
1. **New tool: `larry tools cl-format-migrate` — CRLF->LF batch converter
for both `.tbl` AND `.xlt`, plus an opt-in chain to Cloverleaf's own
native converters.** Built on TWO rounds of Bryan-direct live evidence
from this session:
- **Round 1:** Bryan ran `file` on a broken vs. a working `.tbl` and
both are prologue `version: 6.0` — identical. The ONLY difference is
line endings (broken = CRLF, working = pure LF).
- **Round 2 (Gundersen engine box, same session):** Bryan found BOTH
native Cloverleaf converters exist in `$HCIROOT/bin` —
`hcitblconvert` (tables) and `hcixltconvert` (xlates) — and **both
fail on the CRLF files with the identical error** (`' before prologue
start` / `Please correct the error(s) and re-run.`) — the stray CR
read as a bogus char before the `prologue` keyword. So Cloverleaf's
OWN native converters choke on CRLF for the same root cause. Also
learned: neither binary has `-h`/`--help` (args are bare TABLE/XLATE
NAMES, e.g. `getwell_skipfacility_pv1_034.tbl`, not paths); resolution
is against the site's `Tables`/`Xlate` dir (`$HCIROOT/$HCISITE/Tables/
<name>` then `$HCIROOT/Tables/<name>` — mirrors `nc-table.sh`'s
existing lookup order); a bare `*` glob pulls in non-table entries and
errors, so this tool never globs at the native binary — it iterates
its own discovered file list, one name per invocation.
- `lib/cl-format-migrate.sh` (+ `bin/cl-format-migrate` on-PATH wrapper,
registered in `larry tools list`):
`cl-format-migrate [<dir>] [--type tbl|xlt|all] [--dry-run]
[--confirm yes] [--out <report-dir>] [--deep-verify]
[--run-native-convert]`. `<dir>` defaults to the current directory
when omitted (same cwd-default convention as v0.9.11's
`hl7-diff-set`).
- **Step 1 (both `.tbl` and `.xlt`, the core/default step — fully
implemented, not a stub for either type anymore):** strips ONLY
trailing carriage returns (`sed 's/\r$//'` / dos2unix semantics —
never `tr -d '\r'`, which would also eat a CR that legitimately
appears mid-line). Every write goes through the existing `journal.sh`
backup-first mechanism (GL-011 invariant #3 — reused, not reinvented);
reversible via `larry-rollback.sh --entry`/`--session`. Idempotent:
files already pure-LF are SKIPPED, never touched, never counted as a
failure. Per-file VERIFY after conversion (mandatory, format-agnostic
— `.tbl`/`.xlt` share the same prologue convention): (a) zero CR bytes
remain, (b) the prologue/end_prologue/version: block is structurally
intact (same block counts pre vs. post), and (c) the actual gate — the
new file is byte-identical to the original with only trailing CRs
stripped, proving nothing else changed. A file only counts CONVERTED
if all three pass; any failure is FAILED_VERIFY (loud stop, journal
backup untouched, no silent auto-rollback — an explicit
`larry-rollback.sh` call is always the operator's decision). An
optional `--deep-verify` (`.tbl` only) additionally round-trips the
file through `table-to-csv.sh` as an EXTRA, non-gating sanity check —
nc-table.sh's known `pairs`/`lookup` bug (F-4, 2026-06-08
tool-readiness audit) is still unfixed, so this tool does NOT depend
on it for pass/fail.
- **Step 2 (`--run-native-convert`, OPT-IN, default OFF):** after a
successful CR-strip, chain the matching native binary
(`hcitblconvert`/`hcixltconvert`, located on `$PATH` /
`$HCIROOT/bin` / `$HCIROOT/server/bin`) — "Cloverleaf's real format
bump, no format-guessing by us" — invoked with the bare filename
against a CONFIDENTLY INFERRED site (path matches `<site>/Tables|
Xlate` with a real `NetConfig`, or the shared `$HCIROOT/Tables|Xlate`
layout; anything else is `NATIVE_SKIPPED`, never guessed). Only ever
run on a file THIS invocation just CR-stripped — an already-clean
SKIPPED file is left completely alone. A SEPARATE journal checkpoint
is taken immediately before the native call, so a native-convert
FAILURE (nonzero exit, or `before prologue`/`please correct the
error` in its output, or a truncated-empty result) auto-rolls-back
JUST that step via `larry-rollback.sh --entry` — restoring to the
already-verified CR-stripped-safe state, never leaving a file in an
uncertain post-native condition — while the CR-strip's own success is
reported and preserved independently. Kept opt-in per Larry's
explicit instruction, pending Bryan's confirmation that
`hcitblconvert` succeeds cleanly on a CR-stripped table before it
becomes the default.
- Emits a summary (converted / skipped-already-lf / failed-verify /
planned / native-converted / native-failed / native-skipped) to
stdout always, plus `report.tsv` + `report.md` when `--out DIR` is
given. Exit codes: 2 = a CR-strip verify failure occurred; 3 = a
native-convert step failed (auto-rolled-back); 2 takes priority if
both occur.
- Bash 3.2 / MobaXterm-safe throughout. Fixed one bash-3.2-only bug
found during self-test: bash 3.2's `printf` builtin misreads a format
string that itself starts with `"- "` as an (invalid) option flag
(`printf: - : invalid option`) unless `printf --` ends option parsing
first — hit in the `report.md` generator's front-matter line.
- Self-tested end-to-end with synthetic CRLF `.tbl`/`.xlt` fixtures
matching Bryan's exact reported shapes, including a mocked
`hcitblconvert`/`hcixltconvert` pair proving: a clean native-convert
success (file mutated, entry recorded), a native-convert FAILURE with
automatic rollback to the CR-stripped-safe state (verified via
`larry-rollback.sh --list`), an orphan file outside any recognizable
site layout correctly `NATIVE_SKIPPED` rather than guessed,
dry-run-with-`--run-native-convert` touching nothing, and
`--run-native-convert` without `$HCIROOT` refusing with a usage
error — plus the original round-trip/idempotent-reconvert/rollback/
usage-error suite — all under real `/bin/bash` 3.2.57.
- **Vera-gate fix (same-day CODE-FAIL, re-gated):** verify check (a)
originally counted ANY remaining CR byte anywhere in the file
(`grep -c $'\r'`), not just a trailing CR-before-LF. The strip itself
was always correct (`sed 's/\r$//'` — strips only a trailing CRLF,
correctly PRESERVES a legitimate embedded/mid-line CR, per the tool's
own "never `tr -d '\r'`" design point). But check (a) then saw that
preserved mid-line CR and fired a FALSE `FAILED_VERIFY` — and never
converged: the file re-failed identically on every re-run (never
reached `SKIPPED`), pinning the batch exit code at 2 forever and, under
`--run-native-convert`, permanently blocking that file from the native
step (which only fires on `CONVERTED`). Did not affect Bryan's real
tables (pure trailing-CRLF, no embedded CR) but was a real correctness
bug in the verify gate. **Fix:** replaced the blanket any-CR count with
`_count_trailing_cr` (anchored on `\r$` — CR immediately before
end-of-line), used consistently for BOTH the pre-strip "does this file
need converting" decision and verify check (a), so it matches the
strip's own `\r$` semantics exactly. A file with only a legitimate
mid-line CR and no trailing CRLF is now correctly `SKIPPED` outright
(nothing for this tool to do); a file with BOTH a trailing CRLF and a
mid-line CR now converges to `CONVERTED` (mid-line CR verified
byte-for-byte preserved via `xxd`) then `SKIPPED` on every subsequent
re-run. Report-row wording updated to say "trailing-CRLF" instead of
the ambiguous "CRLF" throughout, per Vera's minor note. Re-verified the
full original self-test suite (round-trip/idempotent/rollback/
`--run-native-convert` mocked success+failure+orphan/usage-errors) —
no regressions.
2. **REPL QoL: `cdr`/`cds`/`cdd` cd-shortcuts + bare site-name shorthand
(Bryan-direct).** `cdr` → `cd "$HCIROOT"`, `cds` → `cd "$HCIROOT/$HCISITE"`,
`cdd` → `cd "$HCIROOT/$HCISITE/data"` (each guards against an empty
`$HCIROOT`/`$HCISITE` rather than cd-ing to a bogus path). Typing a BARE
site name at the REPL prompt (e.g. `adt`) now auto-runs the `/site adt`
equivalent, then `cd`s into it — but ONLY when the token actually
resolves to a real site (a directory under `$HCIROOT` with a `NetConfig`)
**and** isn't itself a recognized command; every existing `/command` is
tried first and always wins (the new dispatch arm's glob, `[!/]*)`, can
never match anything starting with `/`), and `cdr`/`cds`/`cdd` have their
own literal arms ahead of the generic site-name arm, so a hypothetical
site literally named `cdr` can never shadow the alias. Multi-word input
(anything containing whitespace) is explicitly excluded — ordinary
prose keeps flowing to the LLM exactly as before. Proven with a 12-case
isolated dispatch-logic harness (identical case-arm patterns) under real
bash 3.2.57, including the no-shadow proof (a same-named `cdr` site dir
does NOT hijack the alias) and the unchanged-fallthrough proof for
multi-word / non-matching bare tokens.
## v0.9.11 — 2026-07-23
Small UX patch on top of v0.9.10's `hl7-diff-set`, both from Bryan's live

14
MANIFEST
View File

@ -23,7 +23,7 @@
# scripts/make-manifest.sh and bump VERSION.
# Top-level scripts
larry.sh d598cf576896504e7caf3c220a2adb8dd4ba85dd7ab6e5b23901cb4452737ef7
larry.sh d06348673bbfe700fcb1b7871ece29863a2890cf7aa967f985551fd3f8ec0c7e
larry-tunnel.sh 6b050e4eeab15669f4858eaf3b807f168f211ced07815db9521bc40a093f6aaa
larry-auth.sh a220cdf7878569dc3028951ee57fc8d5e706a8ca5c6aa45347b58facb386f831
larry-rollback.sh 91b5e9aa6c79266bf306dcfba4ca791c07971bd6924d67a779037531648aa6d0
@ -31,9 +31,9 @@ install-larry.sh 19f42e8f6e54eaffffe33e1464a8d245721260aed047270a75987c0b25c936c
uninstall-larry.sh c53ad2d8354c7adeb243b541f027f3f481e4a8661eecfd7af14d7ca53cfcaad9
# Metadata
VERSION afb95cbb6514aced3e091a5d8e7e017db52ed7b7faa4dbcdce9685a8f4365c09
MANUAL.md 88bf5f8244c5e584d889a2e9adfe54a9378475ddef927f9c8ffe2eda815186e2
CHANGELOG.md fc2b96731ee6ba0c8d0b9c3d1c711e7318a947e72eea616a02d42ab44d0a650c
VERSION 71e12a378ec002955b38b9bdfdbc8fda0fd6c2b34a505d24d6288f1e727f5911
MANUAL.md 750bd85fa6babd852acac69c2a0bf99b70a7b7cbb6344a21a253daa4a0c33d35
CHANGELOG.md 7803b58ab8ffd80aacb5df5a12711194ed5272418aefcfd1bf8b418036261796
# Agent personas (system-prompt overlays)
agents/larry.md 0a1ef737e7fc133ab35be09f79c3a4df33de814e0404b69b950932d0c8a01be1
@ -95,6 +95,10 @@ lib/each-site.sh 00a31166c2cfcb610e16d4fd8bef439554b10b9e07f93f6917e5732c4ba0957
lib/len2nl.sh e84196f965ff77c49d3c8b3a776be30342f78c8b2e1c165907089d6ace908746
lib/csv-to-table.sh ca9cfbcb9f6f42bf7925bf8b8d1e25f0f7209be138d7b971e3adef3ff6b5b3dd
lib/table-to-csv.sh ad98e73687bc9e9f6ae0cd79ed5ba26c856076902865230f822dec1a1beae4b1
# v0.9.12: batch CRLF->LF format-migration for .tbl AND .xlt (journaled,
# verified, idempotent) + an opt-in --run-native-convert chain to
# hcitblconvert/hcixltconvert (auto-rollback on native-step failure).
lib/cl-format-migrate.sh c9293c4ee612e783aae57db8b24e7d6b6a2461e5876e0ca6c2d808a1b86837d2
# NetConfig tooling
lib/nc-engine.sh 3cf479511426c58f791016f18cdf46f0846f4746c7b1aee5f44b023ba0162c72
@ -190,6 +194,8 @@ bin/hl7-desanitize 78c5bda12e7374c056fac940ac1889ceca87f19a1e711dc658cc46b63b020
bin/each-site 435ed2ec2f6a48e7d15c604cebd7bb49d4744a964770682498bf12174a0e2dea
bin/csv-to-table 9625f29bb874e096f0df4e19f9aa04a5beb61334d6b57f0c8b041d0e09ebf8bf
bin/table-to-csv 961e45971ef3284985aedfebd5bfe601adec5273ca8ec78c9077fae83d7278ac
# v0.9.12: on-PATH wrapper for lib/cl-format-migrate.sh
bin/cl-format-migrate 93b7632fe151034933891d8d2b256245b84ebed39c69e86ef624afb9a3548217
# v0.9.4: durable checked-in demo fixture — a 3-site synthetic integrator
# (epic→ancout→codamx) that exercises cross-site route fan-out + fan-in. PHI-

49
MANUAL.md
View File

@ -79,6 +79,21 @@ export HCISITE=adt
export HCISITEDIR="$HCIROOT/$HCISITE"
```
### REPL shortcuts (v0.9.12)
Inside the `larry` REPL (not the offline `lib/` tools):
- `cdr` → `cd "$HCIROOT"`
- `cds` → `cd "$HCIROOT/$HCISITE"`
- `cdd` → `cd "$HCIROOT/$HCISITE/data"`
- Typing a **bare site name** (e.g. `adt`) auto-runs the `/site adt`
equivalent, then `cd`s into it — but only when the token resolves to a
real site directory under `$HCIROOT` (one with a `NetConfig`) and isn't
itself a recognized command. Any existing `/command` always wins (bare
tokens never start with `/`), and `cdr`/`cds`/`cdd` always win over a
same-named site. Multi-word input (anything with a space) is never
treated as a bare site name — normal prompts keep going to the model.
---
## Auto-update & origin
@ -858,6 +873,40 @@ OBR.10, .16, .32 / OBX.16 / DG1.3, .4 / ORC.10, .12 (orders/observation
---
## Table/xlate format migration (`lib/cl-format-migrate.sh`, v0.9.12)
Fixes the `' before prologue start` / `Please correct the error(s) and
re-run.` parse error on old `.tbl`/`.xlt` files carrying CRLF line endings —
both Cloverleaf's own prologue parser AND its native converters
(`hcitblconvert`/`hcixltconvert`) choke on the stray CR for the same reason.
```bash
# Preview only — writes nothing
cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt/tables --dry-run
# Real conversion (CR-strip only — the default, proven-sufficient fix)
cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt/tables --confirm yes
# Both .tbl and .xlt under a directory, with a machine-readable report
cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt \
--type all --confirm yes --out /tmp/cl-format-report
# OPT-IN: also chain Cloverleaf's own native converter after the CR-strip
# (hcitblconvert / hcixltconvert — auto-rolls-back just the native step on
# failure, keeping the already-verified CR-stripped file intact). Requires
# $HCIROOT set; the native binary must be findable on $PATH, $HCIROOT/bin,
# or $HCIROOT/server/bin.
cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt \
--type all --confirm yes --run-native-convert
```
Every write is journaled (backup + diff + atomic write) exactly like every
other v3 write tool — undo with `larry-rollback.sh --entry <id>` or
`--session <id>` (printed at the end of every real run). Idempotent: an
already-pure-LF file is skipped, never touched.
---
## Quick recipe: "I have to do X without internet"
| Task | Command |

2
VERSION
View File

@ -1 +1 @@
0.9.11
0.9.12

13
bin/cl-format-migrate Executable file
View File

@ -0,0 +1,13 @@
#!/usr/bin/env bash
# cl-format-migrate — direct, on-PATH wrapper for lib/cl-format-migrate.sh
# (no `larry tools` prefix needed). All args pass straight through;
# -h/--help is handled by the underlying tool.
#
# cl-format-migrate /opt/cloverleaf/cis2025.01/integrator/adt/tables --dry-run
# cl-format-migrate . --type tbl --confirm yes --out /tmp/cl-format-report
set -o pipefail
_self="${BASH_SOURCE[0]}"; [ -L "$_self" ] && _self="$(readlink "$_self")"
. "$(cd "$(dirname "$_self")" && pwd)/_nc_common.sh"
lib="$(_nc_resolve_lib)" || { echo "cl-format-migrate: lib/ toolkit not found (set LARRY_LIB_DIR or LARRY_HOME)" >&2; exit 1; }
if [ $# -eq 0 ]; then exec bash "$lib/cl-format-migrate.sh" --help; fi
exec bash "$lib/cl-format-migrate.sh" "$@"

53
larry.sh
View File

@ -100,7 +100,7 @@ set -o pipefail
# ─────────────────────────────────────────────────────────────────────────────
# Config
# ─────────────────────────────────────────────────────────────────────────────
LARRY_VERSION="0.9.11"
LARRY_VERSION="0.9.12"
LARRY_HOME="${LARRY_HOME:-$HOME/.larry}"
# ─────────────────────────────────────────────────────────────────────────────
@ -453,6 +453,7 @@ hl7-desanitize.sh|Reverse hl7-sanitize: restore original values from a token map
each-site.sh|Run a command once per site under $HCIROOT (exposes $HCISITE / $HCISITEDIR)
csv-to-table.sh|Convert a 2-column CSV into Cloverleaf .tbl format
table-to-csv.sh|Convert a Cloverleaf .tbl file to CSV
cl-format-migrate.sh|Batch CRLF->LF format-migration for old-Cloverleaf .tbl AND .xlt files (journaled, verified, idempotent) — fixes the "' before prologue start" parse error on both. Optional --run-native-convert chains hcitblconvert/hcixltconvert on top (opt-in, auto-rollback on failure). Usage: cl-format-migrate [dir] [--type tbl|xlt|all] [--dry-run] [--confirm yes] [--out DIR] [--run-native-convert]
REG
}
@ -7852,6 +7853,20 @@ main_loop() {
/cd\ *) local target="${input#/cd }"
if cd "$target" 2>/dev/null; then larry_say "cd -> $(pwd)"; else err "no such directory: $target"; fi
continue ;;
# v0.9.12 — QoL cd aliases (Bryan-direct). Guard against empty
# $HCIROOT/$HCISITE rather than cd-ing to a bogus/empty path.
cdr) if [ -z "${HCIROOT:-}" ]; then err "cdr: \$HCIROOT is not set"
elif cd "$HCIROOT" 2>/dev/null; then larry_say "cd -> $(pwd)"
else err "no such directory: $HCIROOT"; fi
continue ;;
cds) if [ -z "${HCIROOT:-}" ] || [ -z "${HCISITE:-}" ]; then err "cds: \$HCIROOT/\$HCISITE not both set"
elif cd "$HCIROOT/$HCISITE" 2>/dev/null; then larry_say "cd -> $(pwd)"
else err "no such directory: $HCIROOT/$HCISITE"; fi
continue ;;
cdd) if [ -z "${HCIROOT:-}" ] || [ -z "${HCISITE:-}" ]; then err "cdd: \$HCIROOT/\$HCISITE not both set"
elif cd "$HCIROOT/$HCISITE/data" 2>/dev/null; then larry_say "cd -> $(pwd)"
else err "no such directory: $HCIROOT/$HCISITE/data"; fi
continue ;;
/load\ *) local f="${input#/load }"
if [ ! -f "$f" ]; then err "no such file: $f"; continue; fi
input="$(cat "$f")"
@ -8001,6 +8016,42 @@ EOF
)
larry_say "/nc-regression-env: templated prompt prepared for $_src → $_tgt (scope=$_scope, out=$_out)"
;;
# v0.9.12 — bare site-name shorthand (Bryan-direct). A single-word,
# non-slash token (e.g. `adt`) that resolves to a real site directory
# under $HCIROOT (has a NetConfig) auto-runs the equivalent of
# `/site <name>` then cd's into it. Deliberately conservative:
# - `[!/]*)` only matches non-slash input, so it can NEVER shadow any
# /command arm above (those are tried first and win on a literal
# match regardless of where this arm sits in the case).
# - `cdr`/`cds`/`cdd` are matched by their OWN literal arms above, so
# they always win before falling through to here — no shadowing.
# - Multi-word input (anything with a space) is explicitly excluded
# — "explain the adt site" must keep going to the LLM, not be
# mistaken for a site name.
# - If $HCIROOT/$HCISITE aren't set, or the token isn't a real site
# dir (no matching NetConfig), NOTHING happens here — the arm
# falls through with no `continue`, so execution proceeds exactly
# as it did before this feature existed (normal LLM-prompt path).
[!/]*)
case "$input" in
*[[:space:]]*) : ;; # multi-word -> never a bare site name
*)
if [ -n "${HCIROOT:-}" ] && [ -d "${HCIROOT}/${input}" ] && [ -f "${HCIROOT}/${input}/NetConfig" ]; then
HCISITE="$input"; HCISITEDIR="$HCIROOT/$HCISITE"
export HCISITE HCISITEDIR
detect_cloverleaf_env
system_prompt=$(build_system_prompt)
if cd "$HCISITEDIR" 2>/dev/null; then
larry_say "HCISITE -> $HCISITE ($HCISITEDIR); cd -> $(pwd)"
else
larry_say "HCISITE -> $HCISITE ($HCISITEDIR)"
err "cd failed: $HCISITEDIR"
fi
continue
fi
;;
esac
;;
/*) err "unknown command: $input (try /help)"; continue ;;
esac

493
lib/cl-format-migrate.sh Executable file
View File

@ -0,0 +1,493 @@
#!/usr/bin/env bash
# cl-format-migrate.sh — CRLF->LF format-migration for Cloverleaf .tbl AND
# .xlt files, plus an OPT-IN chain to Cloverleaf's own native converters
# (hcitblconvert / hcixltconvert). Per the 2026-07-23 design doc
# (Deliverables/2026-07-23-cloverleaf-table-xlate-format-migration-design.md)
# and TWO rounds of Bryan-direct live evidence from the same session:
#
# ROUND 1 (the original trigger): `file` on a broken vs. a working .tbl
# showed BOTH are prologue `version: 6.0` (identical) — the ONLY
# difference is line endings:
# getwell_skipfacility_pv1_034.tbl -> ASCII text, WITH CRLF (broken)
# getwellntw_sndingevent_msh04.tbl -> ASCII text (working)
#
# ROUND 2 (Gundersen engine box, same session): Bryan found BOTH native
# Cloverleaf converters exist in $HCIROOT/bin — `hcitblconvert` (tables)
# and `hcixltconvert` (xlates) — and BOTH FAIL on the CRLF files with the
# IDENTICAL error:
# $ hcitblconvert getwell_skipfacility_pv1_034.tbl
# getwell_skipfacility_pv1_034.tbl
# ' before prologue start
# $ hcixltconvert
# ' before prologue start
# Please correct the error(s) and re-run.
# That leading `'` is the stray CR being read as a bogus char before the
# `prologue` keyword — Cloverleaf's OWN native converters choke on CRLF
# for the SAME root-cause reason our own parser does. Also learned:
# - No -h/--help flag on either binary; args are TABLE/XLATE NAMES
# (e.g. `getwell_skipfacility_pv1_034.tbl`, WITH extension), not full
# paths. Resolution is against the site's Tables/Xlate dir:
# $HCIROOT/$HCISITE/Tables/<name> then $HCIROOT/Tables/<name> (tables;
# nc-table.sh's existing lookup order) — Xlate mirrors this with
# Xlate/xlate (nc-xlate.sh's existing lookup order).
# - A bare `*` glob pulls in non-table entries and errors — so this
# tool NEVER globs at the native binary; it iterates its OWN
# discovered file list and invokes the binary once per name.
#
# DESIGN (unified, per Larry's 2026-07-23 follow-up dispatch):
# Step 1 (both .tbl and .xlt, ALWAYS the core/default step): strip ONLY
# trailing carriage returns (`sed 's/\r$//'` / dos2unix semantics —
# never `tr -d '\r'`). Journaled (backup+diff+atomic write), verified,
# idempotent (already-pure-LF files are SKIPPED). This alone is
# PROVEN (Round 1) to make a table load.
# Step 2 (OPT-IN via --run-native-convert, default OFF): chain the
# matching native binary (hcitblconvert for .tbl, hcixltconvert for
# .xlt) on top of the CR-stripped file, so Cloverleaf's OWN converter
# does whatever real format-level work it does (metadata/version
# re-serialization) — "no format-guessing by us." Only ever run on a
# file THIS invocation just CR-stripped (never on an already-clean
# SKIPPED file — untouched files stay untouched). Kept opt-in until
# Bryan confirms hcitblconvert succeeds cleanly on a CR-stripped table
# (per Larry's explicit instruction — do NOT make this the default
# yet). A pre-native checkpoint is journaled separately from the CR-
# strip entry, so a native-convert FAILURE auto-rolls-back to the
# already-verified CR-stripped-only state (never leaves a file in an
# uncertain post-native state) while the CR-strip's own success is
# preserved and reported independently.
#
# Usage:
# cl-format-migrate.sh [<dir>] [--type tbl|xlt|all] [--dry-run]
# [--confirm yes] [--out <report-dir>] [--deep-verify]
# [--run-native-convert]
#
# <dir> directory to scan RECURSIVELY for *.tbl / *.xlt.
# DEFAULTS TO THE CURRENT DIRECTORY (.) if omitted —
# same cwd-default convention as v0.9.11 hl7-diff-set.
# --type T tbl (default) | xlt | all
# --dry-run classify + report what WOULD happen; write NOTHING
# --confirm yes required to actually write on a REAL (non-dry-run)
# conversion. Without it: plan-only refusal per file,
# consistent with nc-set-field.sh / nc-provision-jumps.sh.
# --out DIR write report.tsv + report.md into DIR (created if
# needed). Always ALSO prints the summary to stdout.
# --deep-verify (.tbl only) additionally round-trip the converted
# file through table-to-csv.sh — informational only,
# NEVER gates pass/fail (nc-table.sh F-4 is unfixed).
# --run-native-convert OPT-IN, default OFF. After a successful CR-strip,
# also chain the matching native Cloverleaf converter
# (hcitblconvert / hcixltconvert), located on $PATH /
# $HCIROOT/bin / $HCIROOT/server/bin, invoked with the
# bare NAME (not a path, not a glob) against the
# inferred site. Requires $HCIROOT set. A file whose
# site can't be confidently inferred (path doesn't
# match the <site>/Tables|Xlate or shared
# $HCIROOT/Tables|Xlate layout) is reported
# NATIVE_SKIPPED, not silently guessed at.
#
# Exit codes:
# 0 all good (including a clean --dry-run / all-already-LF run)
# 1 usage/input error
# 2 at least one file FAILED_VERIFY on the CR-strip step
# 3 at least one file's --run-native-convert step FAILED (auto-rolled-back
# to the CR-stripped-safe state — see NATIVE_FAILED in the report).
# Exit 2 takes priority if BOTH occur.
set -o pipefail
NC_SELF="$0"
LIB_DIR="$(cd "$(dirname "$NC_SELF")" && pwd)"
JOURNAL="$LIB_DIR/journal.sh"
T2C="$LIB_DIR/table-to-csv.sh"
NC_ROLLBACK="$LIB_DIR/../larry-rollback.sh"
[ -f "$NC_ROLLBACK" ] || NC_ROLLBACK="$(command -v larry-rollback.sh 2>/dev/null || printf '%s' "$NC_ROLLBACK")"
die() { printf 'cl-format-migrate: %s\n' "$*" >&2; exit 1; }
warn() { printf 'cl-format-migrate: %s\n' "$*" >&2; }
# Shared CR-safety primitives (coerce_int et al) — same pattern as every
# other lib/ tool. Inline fallback if a partial install is missing it.
if [ -r "$LIB_DIR/cygwin-safe.sh" ]; then
# shellcheck disable=SC1090,SC1091
. "$LIB_DIR/cygwin-safe.sh"
else
coerce_int() { local r="${1:-}" d="${2:-0}" c; c=$(printf '%s' "$r" | tr -cd '0-9'); printf '%s' "${c:-$d}"; }
fi
# journal.sh enables `set -u`; restore the assumed (unset-tolerant) env after
# sourcing, same as nc-engine.sh.
[ -f "$JOURNAL" ] && . "$JOURNAL" || warn "journal.sh not available — writes will NOT be journaled/reversible; refusing to run real conversions."
set +u
usage() { sed -n '2,93p' "$NC_SELF"; }
DIR="."
TYPE="tbl"
DRY_RUN=0
CONFIRM=""
OUT_DIR=""
DEEP_VERIFY=0
RUN_NATIVE=0
_dir_set=0
while [ $# -gt 0 ]; do
case "$1" in
--type) shift; TYPE="$1" ;;
--dry-run) DRY_RUN=1 ;;
--confirm) shift; CONFIRM="${1:-}" ;;
--out) shift; OUT_DIR="$1" ;;
--deep-verify) DEEP_VERIFY=1 ;;
--run-native-convert) RUN_NATIVE=1 ;;
-h|--help) usage; exit 0 ;;
-*) die "unknown flag: $1 (run --help)" ;;
*)
if [ "$_dir_set" = "0" ]; then DIR="$1"; _dir_set=1
else die "extra arg: $1"
fi
;;
esac
shift
done
case "$TYPE" in tbl|xlt|all) ;; *) die "bad --type (want tbl|xlt|all)" ;; esac
[ -d "$DIR" ] || die "no such directory: $DIR"
DIR="$(cd "$DIR" && pwd)"
[ "$RUN_NATIVE" = "1" ] && [ -z "${HCIROOT:-}" ] && die "--run-native-convert requires \$HCIROOT to be set"
# ── report plumbing ──────────────────────────────────────────────────────────
REPORT_TSV="$(mktemp)"
printf 'path\tstatus\treason\tentry_id\n' > "$REPORT_TSV"
CONVERTED=0
SKIPPED=0
FAILED=0
PLANNED=0
NATIVE_CONVERTED=0
NATIVE_FAILED=0
NATIVE_SKIPPED=0
_report_row() {
# path status reason entry_id
printf '%s\t%s\t%s\t%s\n' "$1" "$2" "$3" "${4:-}" >> "$REPORT_TSV"
printf '%-16s %s%s\n' "$2" "$1" "$( [ -n "${3:-}" ] && printf ' — %s' "$3" )"
}
# _count_trailing_cr FILE — count lines whose LAST byte before the newline is
# a CR (i.e. CRLF line endings — exactly what `sed 's/\r$//'` targets).
#
# v0.9.12 Vera-gate fix: this MUST match the strip's own `\r$` semantics, not
# "any CR byte anywhere" (a plain `grep -c $'\r'`). A blanket any-CR count
# also matches a legitimate mid-line/embedded CR that the strip correctly
# and deliberately PRESERVES (the tool's whole "never tr -d '\r'" design
# point) — which made the old check count that preserved byte as a failure,
# forever, on every re-run (never converges to SKIPPED). Anchoring the grep
# on `\r$` (CR immediately before end-of-line) only matches an actual CRLF
# line ending, so a file with just an embedded CR and no trailing CRLF
# correctly reads as 0 here — both for the pre-strip "does this file need
# converting" decision and for verify check (a) below.
_count_trailing_cr() {
local f="$1" n
n=$(LC_ALL=C grep -c $'\r$' "$f" 2>/dev/null)
coerce_int "$n" 0
}
_find_candidates() {
local dir="$1" type="$2"
case "$type" in
tbl) find "$dir" -type f -iname '*.tbl' 2>/dev/null ;;
xlt) find "$dir" -type f -iname '*.xlt' 2>/dev/null ;;
all) find "$dir" -type f \( -iname '*.tbl' -o -iname '*.xlt' \) 2>/dev/null ;;
esac | LC_ALL=C sort
}
# ── CR-strip verify (the mandatory gate; format-agnostic — .tbl and .xlt
# share the same prologue/end_prologue/version convention, confirmed in
# the design doc §2 nc-revisions.sh cross-check) ──────────────────────────
_verify_crlf_strip() {
local orig_copy="$1" new_file="$2"
local cr; cr=$(_count_trailing_cr "$new_file")
if [ "$cr" != "0" ]; then
printf '(a) FAILED: %s trailing-CRLF line(s) still present after conversion' "$cr"
return 1
fi
local pro_pre pro_post epro_pre epro_post ver_pre ver_post
pro_pre=$(coerce_int "$(grep -c '^[[:space:]]*prologue[[:space:]]*$' "$orig_copy" 2>/dev/null)" 0)
pro_post=$(coerce_int "$(grep -c '^[[:space:]]*prologue[[:space:]]*$' "$new_file" 2>/dev/null)" 0)
epro_pre=$(coerce_int "$(grep -c 'end_prologue' "$orig_copy" 2>/dev/null)" 0)
epro_post=$(coerce_int "$(grep -c 'end_prologue' "$new_file" 2>/dev/null)" 0)
ver_pre=$(coerce_int "$(grep -c 'version:' "$orig_copy" 2>/dev/null)" 0)
ver_post=$(coerce_int "$(grep -c 'version:' "$new_file" 2>/dev/null)" 0)
if [ "$pro_pre" != "$pro_post" ] || [ "$epro_pre" != "$epro_post" ] || [ "$ver_pre" != "$ver_post" ]; then
printf '(b) FAILED: prologue/end_prologue/version block count changed (pre %s/%s/%s -> post %s/%s/%s)' \
"$pro_pre" "$epro_pre" "$ver_pre" "$pro_post" "$epro_post" "$ver_post"
return 1
fi
# (c) the actual gate: new file == CR-stripped original, byte for byte.
if ! diff -q <(sed $'s/\r$//' "$orig_copy") "$new_file" >/dev/null 2>&1; then
printf '(c) FAILED: content differs by more than trailing-CR removal (not a pure CRLF->LF pass)'
return 1
fi
return 0
}
# ── native-binary lookup ──────────────────────────────────────────────────────
_resolve_native_bin() {
local name="$1"
if command -v "$name" >/dev/null 2>&1; then command -v "$name"; return 0; fi
local d
for d in "${HCIROOT:-}/bin" "${HCIROOT:-}/server/bin"; do
[ -n "$d" ] && [ -x "$d/$name" ] && { printf '%s' "$d/$name"; return 0; }
done
return 1
}
# _infer_site FILE PARENTNAME1 PARENTNAME2 — infer the site name from a path
# shaped like .../<site>/<PARENTNAME1-or-2>/<file>, requiring the grandparent
# to actually be a site dir (has a NetConfig) so this is never a guess.
# Prints the site name on stdout and returns 0, or returns 1 (no site — could
# still be the shared $HCIROOT/Tables|Xlate layout, handled by the caller).
_infer_site() {
local f="$1" p1="$2" p2="$3"
local parent grandparent pbase
parent="$(dirname "$f")"
pbase="$(basename "$parent")"
case "$pbase" in
"$p1"|"$p2")
grandparent="$(dirname "$parent")"
if [ -n "${HCIROOT:-}" ] && [ -f "$grandparent/NetConfig" ]; then
basename "$grandparent"; return 0
fi
;;
esac
return 1
}
# ── native-convert step (opt-in; only ever called on a file THIS run just
# CR-stripped) ─────────────────────────────────────────────────────────────
_native_convert() {
local file="$1" kind="$2" # kind = tbl | xlt
local binname parent1 parent2
if [ "$kind" = "tbl" ]; then binname="hcitblconvert"; parent1="Tables"; parent2="tables"
else binname="hcixltconvert"; parent1="Xlate"; parent2="xlate"
fi
local bin; bin=$(_resolve_native_bin "$binname") || {
_report_row "$file" "NATIVE_SKIPPED" "$binname not found on \$PATH, \$HCIROOT/bin, or \$HCIROOT/server/bin" ""
NATIVE_SKIPPED=$((NATIVE_SKIPPED + 1)); return 0
}
local site=""
if ! site=$(_infer_site "$file" "$parent1" "$parent2"); then
# Not under a <site>/Tables|Xlate layout. Allow the shared-root layout
# ($HCIROOT/Tables or $HCIROOT/Xlate, no site subdir) with an empty site;
# anything else is a "can't confidently resolve" skip, never a guess.
local parentdir; parentdir="$(dirname "$file")"
case "$parentdir" in
"${HCIROOT:-__no_hciroot__}/$parent1"|"${HCIROOT:-__no_hciroot__}/$parent2") site="" ;;
*)
_report_row "$file" "NATIVE_SKIPPED" "cannot confidently infer site for native convert (path doesn't match <site>/${parent1}|${parent2} or shared \$HCIROOT/${parent1}|${parent2})" ""
NATIVE_SKIPPED=$((NATIVE_SKIPPED + 1)); return 0
;;
esac
fi
# Checkpoint the CR-stripped (already-verified-safe) state as its OWN
# journal entry, distinct from the CR-strip entry, so a native-convert
# failure rolls back ONLY the native step.
local precopy entry2
precopy="$(mktemp)"; cp -p "$file" "$precopy" || {
_report_row "$file" "NATIVE_FAILED" "could not snapshot pre-native state" ""
NATIVE_FAILED=$((NATIVE_FAILED + 1)); rm -f "$precopy"; return 1
}
entry2=$(journal_write "$file" "$precopy" 2>>"$JOURNAL_LOG")
local jw2_rc=$?
rm -f "$precopy"
if [ "$jw2_rc" != "0" ] || [ -z "$entry2" ]; then
_report_row "$file" "NATIVE_FAILED" "pre-native journal checkpoint failed (rc=$jw2_rc) — native convert NOT attempted" ""
NATIVE_FAILED=$((NATIVE_FAILED + 1)); return 1
fi
local base out rc
base="$(basename "$file")"
out=$( (cd "$(dirname "$file")" && HCIROOT="$HCIROOT" HCISITE="$site" "$bin" "$base") 2>&1 )
rc=$?
local fail_reason=""
if [ "$rc" != "0" ]; then
fail_reason="exit code $rc"
elif printf '%s' "$out" | grep -qi "before prologue\|please correct the error"; then
fail_reason="native binary reported a parse error"
elif [ ! -s "$file" ]; then
fail_reason="file is empty/truncated after native convert"
fi
if [ -n "$fail_reason" ]; then
_report_row "$file" "NATIVE_FAILED" "$fail_reason (auto-rolled-back to CR-stripped-safe state) — output: $(printf '%s' "$out" | tr '\n' ' ' | cut -c1-200)" "$entry2"
NATIVE_FAILED=$((NATIVE_FAILED + 1))
if [ -x "$NC_ROLLBACK" ] || [ -f "$NC_ROLLBACK" ]; then
bash "$NC_ROLLBACK" --entry "$entry2" --yes >>"$JOURNAL_LOG" 2>&1 \
|| warn "auto-rollback of $entry2 FAILED — $file may be in an uncertain state; run: larry-rollback.sh --entry $entry2"
else
warn "larry-rollback.sh not found — could not auto-rollback $entry2; $file may be in an uncertain state"
fi
return 1
fi
_report_row "$file" "NATIVE_CONVERTED" "$binname OK (site=${site:-<shared>}); entry $entry2; rollback: larry-rollback.sh --entry $entry2" "$entry2"
NATIVE_CONVERTED=$((NATIVE_CONVERTED + 1))
return 0
}
# ── one .tbl/.xlt file — CR-strip step (shared for both types) ──────────────
_process_file() {
local file="$1" kind="$2" # kind = tbl | xlt
# v0.9.12 Vera-gate fix: trailing-CR-only detection (matches the strip's own
# `\r$` semantics) — a file whose only CR bytes are legitimate mid-line ones
# (no trailing CRLF) has nothing for this tool to do, and is correctly
# SKIPPED rather than endlessly re-attempted/re-failed. See _count_trailing_cr.
local cr; cr=$(_count_trailing_cr "$file")
if [ "$cr" = "0" ]; then
_report_row "$file" "SKIPPED" "already pure-LF (no trailing CRLF found)" ""
SKIPPED=$((SKIPPED + 1))
return 0
fi
if [ "$DRY_RUN" = "1" ]; then
_report_row "$file" "DRY_RUN" "$cr trailing-CRLF line(s) — would convert (re-run with --confirm yes)$( [ "$RUN_NATIVE" = "1" ] && printf '; would also attempt native convert' )" ""
PLANNED=$((PLANNED + 1))
return 0
fi
if [ "$CONFIRM" != "yes" ]; then
_report_row "$file" "REFUSED" "$cr trailing-CRLF line(s) — no --confirm yes, nothing written" ""
PLANNED=$((PLANNED + 1))
return 0
fi
if ! declare -f journal_write >/dev/null 2>&1; then
_report_row "$file" "FAILED_VERIFY" "journal.sh not available — refusing to write unjournaled" ""
FAILED=$((FAILED + 1))
return 1
fi
local orig_copy newtmp
orig_copy="$(mktemp)"; newtmp="$(mktemp)"
if ! cp -p "$file" "$orig_copy"; then
_report_row "$file" "FAILED_VERIFY" "could not snapshot original for verify" ""
FAILED=$((FAILED + 1)); rm -f "$orig_copy" "$newtmp"; return 1
fi
if ! sed $'s/\r$//' "$file" > "$newtmp"; then
_report_row "$file" "FAILED_VERIFY" "sed CRLF-strip failed" ""
FAILED=$((FAILED + 1)); rm -f "$orig_copy" "$newtmp"; return 1
fi
local entry_id jw_rc
entry_id=$(journal_write "$file" "$newtmp" 2>>"$JOURNAL_LOG")
jw_rc=$?
if [ "$jw_rc" != "0" ] || [ -z "$entry_id" ]; then
_report_row "$file" "FAILED_VERIFY" "journal_write failed (rc=$jw_rc) — file NOT modified" ""
FAILED=$((FAILED + 1)); rm -f "$orig_copy" "$newtmp"; return 1
fi
local reason
if reason=$(_verify_crlf_strip "$orig_copy" "$file"); then
local deep_note=""
if [ "$kind" = "tbl" ] && [ "$DEEP_VERIFY" = "1" ] && [ -f "$T2C" ]; then
if "$T2C" --with-header "$file" >/dev/null 2>&1; then
deep_note=" | deep-verify(table-to-csv): OK"
else
deep_note=" | deep-verify(table-to-csv): FAILED (informational only — F-4 known bug, not gating)"
fi
fi
_report_row "$file" "CONVERTED" "$cr trailing-CRLF line(s) stripped; entry $entry_id; rollback: larry-rollback.sh --entry $entry_id${deep_note}" "$entry_id"
CONVERTED=$((CONVERTED + 1))
rm -f "$orig_copy" "$newtmp"
if [ "$RUN_NATIVE" = "1" ]; then
_native_convert "$file" "$kind"
fi
else
_report_row "$file" "FAILED_VERIFY" "$reason — entry $entry_id; rollback: larry-rollback.sh --entry $entry_id" "$entry_id"
FAILED=$((FAILED + 1))
rm -f "$orig_copy" "$newtmp"
fi
}
# ── SESSION (one for the whole batch) ────────────────────────────────────────
SESSION="${LARRY_SESSION_ID:-clfmt-$(date +%Y-%m-%d-%H%M%S)-$$}"
export LARRY_SESSION_ID="$SESSION"
export LARRY_HOME="${LARRY_HOME:-$HOME/.larry}"
JOURNAL_LOG="$(mktemp)"
printf '═══════════════════════════════════════════════════════════════════════\n'
printf 'cl-format-migrate — dir=%s type=%s dry-run=%s confirm=%s run-native-convert=%s\n' \
"$DIR" "$TYPE" "$( [ "$DRY_RUN" = "1" ] && echo yes || echo no )" "$( [ "$CONFIRM" = "yes" ] && echo yes || echo no )" \
"$( [ "$RUN_NATIVE" = "1" ] && echo yes || echo no )"
printf 'journal session: %s\n' "$SESSION"
printf '═══════════════════════════════════════════════════════════════════════\n'
if [ "$TYPE" = "tbl" ] || [ "$TYPE" = "all" ]; then
printf '\n-- tables (.tbl) --\n'
_any=0
while IFS= read -r f; do
[ -n "$f" ] || continue
_any=1
_process_file "$f" tbl
done < <(_find_candidates "$DIR" tbl)
[ "$_any" = "1" ] || printf '(no .tbl files found under %s)\n' "$DIR"
fi
if [ "$TYPE" = "xlt" ] || [ "$TYPE" = "all" ]; then
printf '\n-- xlates (.xlt) --\n'
_any=0
while IFS= read -r f; do
[ -n "$f" ] || continue
_any=1
_process_file "$f" xlt
done < <(_find_candidates "$DIR" xlt)
[ "$_any" = "1" ] || printf '(no .xlt files found under %s)\n' "$DIR"
fi
# ── summary ───────────────────────────────────────────────────────────────────
printf '\n───────────────────────────────────────────────────────────────────────\n'
printf 'SUMMARY: converted=%d skipped-already-lf=%d failed-verify=%d planned=%d native-converted=%d native-failed=%d native-skipped=%d\n' \
"$CONVERTED" "$SKIPPED" "$FAILED" "$PLANNED" "$NATIVE_CONVERTED" "$NATIVE_FAILED" "$NATIVE_SKIPPED"
if [ "$CONVERTED" -gt 0 ] || [ "$NATIVE_CONVERTED" -gt 0 ]; then
printf 'WHOLE-BATCH ROLLBACK (all writes this run): larry-rollback.sh --session %s\n' "$SESSION"
fi
if [ -n "$OUT_DIR" ]; then
mkdir -p "$OUT_DIR" 2>/dev/null || warn "could not create --out dir: $OUT_DIR"
if [ -d "$OUT_DIR" ]; then
cp "$REPORT_TSV" "$OUT_DIR/report.tsv"
{
printf '# cl-format-migrate report\n\n'
# v0.9.12: `printf --` — bash 3.2's printf builtin misreads a format
# string that itself starts with "- " as an (invalid) option flag
# ("printf: - : invalid option") unless "--" ends option parsing first.
printf -- '- dir: `%s`\n- type: %s\n- dry-run: %s\n- confirm: %s\n- run-native-convert: %s\n- session: %s\n- generated: %s\n\n' \
"$DIR" "$TYPE" "$( [ "$DRY_RUN" = "1" ] && echo yes || echo no )" \
"$( [ "$CONFIRM" = "yes" ] && echo yes || echo no )" "$( [ "$RUN_NATIVE" = "1" ] && echo yes || echo no )" \
"$SESSION" "$(date -Iseconds 2>/dev/null || date)"
printf '## Summary\n\n'
printf '| converted | skipped-already-lf | failed-verify | planned | native-converted | native-failed | native-skipped |\n'
printf '|---|---|---|---|---|---|---|\n'
printf '| %d | %d | %d | %d | %d | %d | %d |\n\n' \
"$CONVERTED" "$SKIPPED" "$FAILED" "$PLANNED" "$NATIVE_CONVERTED" "$NATIVE_FAILED" "$NATIVE_SKIPPED"
printf '## Per-file detail\n\n'
printf '| path | status | reason | entry_id |\n|---|---|---|---|\n'
tail -n +2 "$REPORT_TSV" | while IFS=$'\t' read -r p s r e; do
printf '| %s | %s | %s | %s |\n' "$p" "$s" "$r" "$e"
done
} > "$OUT_DIR/report.md"
printf 'report written: %s/report.tsv , %s/report.md\n' "$OUT_DIR" "$OUT_DIR"
fi
fi
rm -f "$REPORT_TSV" "$JOURNAL_LOG"
if [ "$FAILED" -gt 0 ]; then
exit 2
fi
if [ "$NATIVE_FAILED" -gt 0 ]; then
exit 3
fi
exit 0