v0.8.9: manifest-sync live progress indicator — silent ~3-min relaunch no longer looks frozen
Root cause: sync_from_manifest fully downloads all 48 manifest entries sequentially (authenticated HTTPS via proxy + Cloudflare), then cmp-compares locally to find the few that changed — 48 silent round-trips, ~3 min, no output. Add _sync_progress/_sync_progress_done: live in-place "checking N/48 <file>" (switching to "downloading N/48 <file>" on real changes) via \r\033[K only — MobaXterm-safe (no scroll-region/cursor-save/abs-pos). Gates on [ -t 2 ]; non-TTY emits a plain heartbeat every 10 files (no \r). Current filename shown so a hang is visible by name; per-file curl --max-time bounds each stall. Hash-skip speedup deferred: MANIFEST is paths-only (no hashes), so local skip-unchanged needs a manifest-format + release-tooling change — filed for v0.9.x. Sync correctness unchanged. Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
This commit is contained in:
parent
5ed82db770
commit
b80f2fb29d
50
CHANGELOG.md
50
CHANGELOG.md
@ -4,6 +4,56 @@ All notable changes to `cloverleaf-larry` / `larry-anywhere` are recorded here.
|
||||
Versioning is loose-semver; bumps trigger the in-process self-update on every
|
||||
running client via `LARRY_BASE_URL` + `MANIFEST`.
|
||||
|
||||
## v0.8.9 — 2026-05-27
|
||||
|
||||
Manifest-sync live progress indicator (Clover). Symptom: Bryan's auto-update
|
||||
relaunch is very slow and **looks frozen** — a multi-minute silent gap between
|
||||
`update found: X -> Y … relaunching` and `manifest sync: N updated, 0 failed, M
|
||||
total`, with NO output in between. Observed: `[21:32:29]` → `[21:35:37]` = ~3 min
|
||||
of silence to update just 3 of 48 files; earlier `[20:19:42]` → `[20:20:32]` =
|
||||
~50s for 22 files. Bryan cannot tell working-but-slow from hung.
|
||||
|
||||
**Root cause (confirmed by reading `sync_from_manifest`).** Phase-A sync does NOT
|
||||
do cheap HEAD/hash-checks — it **fully downloads EVERY manifest entry** over an
|
||||
authenticated HTTPS round-trip (Gitea via the corporate proxy + Cloudflare),
|
||||
then uses `cmp -s` locally to decide whether the bytes actually changed
|
||||
(discarding unchanged ones). With 48 entries that is **48 sequential full
|
||||
downloads**, every relaunch, regardless of how few files changed. The loop emits
|
||||
nothing until the trailing summary `log` line → the entire window is silent →
|
||||
looks hung. The "3 files changed" in the summary is just how many survived the
|
||||
`cmp`; all 48 were still fetched.
|
||||
|
||||
- **Live in-place progress over the WHOLE sync.** New `_sync_progress` /
|
||||
`_sync_progress_done` helpers render `checking N/48 lib/foo.sh` per entry,
|
||||
rewriting the line via `\r\033[K` (carriage-return + clear-line). When a fetch
|
||||
actually lands a changed file, the frame switches to `downloading N/48
|
||||
lib/foo.sh` so real writes are distinguishable from the common unchanged case.
|
||||
The current filename is always shown, so a genuine stall is VISIBLE — you see
|
||||
exactly which file it is stuck on instead of a blank freeze.
|
||||
- **MobaXterm-safe escapes only.** Uses solely `\r` + `ESC[K` (the same
|
||||
primitive already at the readline prompt, audited safe in the v0.8.7 escape
|
||||
inventory). Deliberately NO DECSTBM scroll-region, cursor save/restore, or
|
||||
absolute-row addressing — the exact escapes MobaXterm mis-honors. Verified
|
||||
under a pty: only `ESC[0m`, `ESC[2m`, `ESC[K` ever emitted; zero forbidden
|
||||
escapes. Independent of `LARRY_NO_STREAM` / mouse mode (gates only on a TTY).
|
||||
- **Non-TTY safe.** Gates on `[ -t 2 ]`. When stderr is a pipe/log it emits a
|
||||
plain newline-terminated heartbeat every 10 files (no `\r`), so captured logs
|
||||
stay clean — verified zero `\r` bytes in piped output.
|
||||
- **Heartbeat for hangs.** The per-file fetch already carries curl `--max-time`
|
||||
(5s for VERSION, 15s otherwise); a stuck file now shows its name, times out,
|
||||
counts as a fail, and the loop advances — never an infinite silent stall.
|
||||
- **Speedup deferred (by design).** The clean fix — fetch the manifest once,
|
||||
compare per-file hashes locally, and SKIP unchanged files with NO per-file
|
||||
network round-trip — requires the MANIFEST to carry hashes. It currently
|
||||
carries **paths only** (no hashes/sizes), so the optimization would require a
|
||||
manifest-format + release-tooling change (higher risk, separate change). Filed
|
||||
as a follow-up: `MANIFEST` should emit `path<TAB>sha256` so v0.9.x can turn 48
|
||||
network round-trips into 1 fetch + local compare. This release ships the
|
||||
indicator (Bryan's explicit ask); correctness of the sync is unchanged.
|
||||
|
||||
Note: the v0.8.9 relaunch itself is still the slow path (the indicator helps
|
||||
from the NEXT update onward), but you now see live forward progress during it.
|
||||
|
||||
## v0.8.8 — 2026-05-27
|
||||
|
||||
Force unconditional 429 header capture (Clover). Symptom: Bryan's MobaXterm
|
||||
|
||||
84
larry.sh
84
larry.sh
@ -65,7 +65,7 @@ set -o pipefail
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Config
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
LARRY_VERSION="0.8.8"
|
||||
LARRY_VERSION="0.8.9"
|
||||
LARRY_HOME="${LARRY_HOME:-$HOME/.larry}"
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
@ -474,6 +474,65 @@ _record_origin() {
|
||||
_LARRY_LAST_ORIGIN_URL="$2"
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# v0.8.9: manifest-sync progress indicator.
|
||||
#
|
||||
# WHY: phase-A sync (sync_from_manifest) fetches EVERY manifest entry over an
|
||||
# authenticated HTTPS round-trip (Gitea via corporate proxy + Cloudflare), then
|
||||
# uses `cmp` to decide if the byte stream actually changed. With 48 entries that
|
||||
# is 48 sequential round-trips — ~3 min on Bryan's work-box — and it was
|
||||
# COMPLETELY SILENT between "update found … relaunching" and "manifest sync: …".
|
||||
# Looked frozen. This indicator shows live per-file forward progress so a real
|
||||
# hang is visible (you see WHICH file it stalls on) and a slow-but-working sync
|
||||
# is obviously alive.
|
||||
#
|
||||
# MobaXterm-safety: uses ONLY carriage-return + clear-line (\r\033[K), the same
|
||||
# primitive already used at the readline prompt (see read_user_input) and
|
||||
# audited MobaXterm-safe in the v0.8.7 escape inventory. Deliberately NO
|
||||
# DECSTBM scroll-region, cursor-save/restore, or absolute-row addressing — those
|
||||
# are exactly the escapes MobaXterm mis-honors (see render_status_line note).
|
||||
#
|
||||
# These run BEFORE lib/cygwin-safe.sh is sourced (self_update is early), so we
|
||||
# strip CRs inline via parameter expansion rather than calling strip_cr.
|
||||
#
|
||||
# Output goes to stderr (fd 2) to sit alongside log()/warn()/err(), and the TTY
|
||||
# gate is `[ -t 2 ]`. Non-TTY (pipe/log redirect): no \r — we emit a plain
|
||||
# newline-terminated heartbeat line every $_SYNC_PROGRESS_PLAIN_EVERY files so a
|
||||
# captured log shows progress without \r garbage.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
_SYNC_PROGRESS_PLAIN_EVERY=10 # non-TTY: emit a plain heartbeat every N files
|
||||
|
||||
# _sync_progress PHASE CUR TOTAL LABEL — render one in-place progress frame.
|
||||
# PHASE : short verb, e.g. "syncing" / "checking" / "downloading"
|
||||
# CUR : current 1-based index
|
||||
# TOTAL : denominator
|
||||
# LABEL : current filename (or any short context string)
|
||||
# TTY: rewrites the current line via \r\033[K (no scroll, no newline).
|
||||
# non-TTY: prints a plain line only on the first, every-Nth, and (caller uses
|
||||
# _sync_progress_done for) final frame, so logs stay readable.
|
||||
_sync_progress() {
|
||||
local phase="$1" cur="$2" total="$3" label="${4:-}"
|
||||
# Strip any CR that rode in on the label (defensive; pre-cygwin-safe).
|
||||
label="${label//$'\r'/}"
|
||||
if [ -t 2 ]; then
|
||||
printf '\r\033[K%s[%s]%s %s %s/%s %s' \
|
||||
"$C_DIM" "$(date +%H:%M:%S)" "$C_RESET" "$phase" "$cur" "$total" "$label" >&2
|
||||
else
|
||||
# Plain mode: heartbeat on first frame and every Nth frame. No \r.
|
||||
if [ "$cur" -eq 1 ] || [ $(( cur % _SYNC_PROGRESS_PLAIN_EVERY )) -eq 0 ]; then
|
||||
printf '%s[%s]%s %s %s/%s\n' \
|
||||
"$C_DIM" "$(date +%H:%M:%S)" "$C_RESET" "$phase" "$cur" "$total" >&2
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# _sync_progress_done — clear the in-place progress line on a TTY so the
|
||||
# following summary log line lands clean. No-op on a non-TTY (nothing to clear).
|
||||
_sync_progress_done() {
|
||||
[ -t 2 ] && printf '\r\033[K' >&2
|
||||
return 0
|
||||
}
|
||||
|
||||
sync_from_manifest() {
|
||||
local base="$1"
|
||||
local manifest="$LARRY_HOME/.manifest.new"
|
||||
@ -491,6 +550,16 @@ sync_from_manifest() {
|
||||
local self="$0"
|
||||
case "$self" in /*) ;; *) self="$PWD/$self" ;; esac
|
||||
|
||||
# v0.8.9: pre-count manifest entries so the progress indicator has a
|
||||
# denominator. Cheap local pass (no network) over the just-fetched manifest.
|
||||
local total=0 _l
|
||||
while IFS= read -r _l; do
|
||||
case "$_l" in ''|'#'*) continue ;; esac
|
||||
_l="${_l%%[[:space:]]*}"
|
||||
[ -z "$_l" ] && continue
|
||||
total=$((total + 1))
|
||||
done < "$manifest"
|
||||
|
||||
local count=0 updated=0 failed=0 path tmp dest
|
||||
while IFS= read -r path; do
|
||||
case "$path" in ''|'#'*) continue ;; esac
|
||||
@ -502,6 +571,13 @@ sync_from_manifest() {
|
||||
# the running script mid-execution.
|
||||
[ "$path" = "larry.sh" ] && continue
|
||||
|
||||
# v0.8.9: live progress BEFORE the network round-trip, so a fetch that
|
||||
# hangs (slow file / proxy stall) shows exactly which file it is stuck on
|
||||
# rather than freezing silently. fetch_validate's --max-time bounds each
|
||||
# hang to ${_kind} fetch timeout (15s); on timeout it fails loud, counts as
|
||||
# a fail, and the loop advances to the next file — never an infinite stall.
|
||||
_sync_progress checking "$count" "$total" "$path"
|
||||
|
||||
dest="$LARRY_HOME/$path"
|
||||
tmp="$dest.new"
|
||||
mkdir -p "$(dirname "$dest")" 2>/dev/null
|
||||
@ -517,6 +593,9 @@ sync_from_manifest() {
|
||||
esac
|
||||
if fetch_validate "$base/$path" "$tmp" "$_kind" 15 && [ -s "$tmp" ]; then
|
||||
if [ ! -f "$dest" ] || ! cmp -s "$dest" "$tmp"; then
|
||||
# Distinguish a real change (a download that lands) from the common
|
||||
# case of an unchanged file, so the operator sees actual writes.
|
||||
_sync_progress downloading "$count" "$total" "$path"
|
||||
mv "$tmp" "$dest"
|
||||
case "$path" in *.sh) chmod +x "$dest" 2>/dev/null || true ;; esac
|
||||
updated=$((updated + 1))
|
||||
@ -530,6 +609,9 @@ sync_from_manifest() {
|
||||
done < "$manifest"
|
||||
rm -f "$manifest"
|
||||
|
||||
# v0.8.9: clear the in-place progress line so the summary lands clean.
|
||||
_sync_progress_done
|
||||
|
||||
if [ "$updated" -gt 0 ] || [ "$failed" -gt 0 ]; then
|
||||
log "manifest sync: $updated updated, $failed failed, $count total (from $base)"
|
||||
fi
|
||||
|
||||
Loading…
Reference in New Issue
Block a user