v0.8.9: manifest-sync live progress indicator — silent ~3-min relaunch no longer looks frozen
Root cause: sync_from_manifest fully downloads all 48 manifest entries sequentially (authenticated HTTPS via proxy + Cloudflare), then cmp-compares locally to find the few that changed — 48 silent round-trips, ~3 min, no output. Add _sync_progress/_sync_progress_done: live in-place "checking N/48 <file>" (switching to "downloading N/48 <file>" on real changes) via \r\033[K only — MobaXterm-safe (no scroll-region/cursor-save/abs-pos). Gates on [ -t 2 ]; non-TTY emits a plain heartbeat every 10 files (no \r). Current filename shown so a hang is visible by name; per-file curl --max-time bounds each stall. Hash-skip speedup deferred: MANIFEST is paths-only (no hashes), so local skip-unchanged needs a manifest-format + release-tooling change — filed for v0.9.x. Sync correctness unchanged. Co-Authored-By: Clover (Claude Opus 4.7) <noreply@anthropic.com>
This commit is contained in:
parent
5ed82db770
commit
b80f2fb29d
50
CHANGELOG.md
50
CHANGELOG.md
@ -4,6 +4,56 @@ All notable changes to `cloverleaf-larry` / `larry-anywhere` are recorded here.
|
|||||||
Versioning is loose-semver; bumps trigger the in-process self-update on every
|
Versioning is loose-semver; bumps trigger the in-process self-update on every
|
||||||
running client via `LARRY_BASE_URL` + `MANIFEST`.
|
running client via `LARRY_BASE_URL` + `MANIFEST`.
|
||||||
|
|
||||||
|
## v0.8.9 — 2026-05-27
|
||||||
|
|
||||||
|
Manifest-sync live progress indicator (Clover). Symptom: Bryan's auto-update
|
||||||
|
relaunch is very slow and **looks frozen** — a multi-minute silent gap between
|
||||||
|
`update found: X -> Y … relaunching` and `manifest sync: N updated, 0 failed, M
|
||||||
|
total`, with NO output in between. Observed: `[21:32:29]` → `[21:35:37]` = ~3 min
|
||||||
|
of silence to update just 3 of 48 files; earlier `[20:19:42]` → `[20:20:32]` =
|
||||||
|
~50s for 22 files. Bryan cannot tell working-but-slow from hung.
|
||||||
|
|
||||||
|
**Root cause (confirmed by reading `sync_from_manifest`).** Phase-A sync does NOT
|
||||||
|
do cheap HEAD/hash-checks — it **fully downloads EVERY manifest entry** over an
|
||||||
|
authenticated HTTPS round-trip (Gitea via the corporate proxy + Cloudflare),
|
||||||
|
then uses `cmp -s` locally to decide whether the bytes actually changed
|
||||||
|
(discarding unchanged ones). With 48 entries that is **48 sequential full
|
||||||
|
downloads**, every relaunch, regardless of how few files changed. The loop emits
|
||||||
|
nothing until the trailing summary `log` line → the entire window is silent →
|
||||||
|
looks hung. The "3 files changed" in the summary is just how many survived the
|
||||||
|
`cmp`; all 48 were still fetched.
|
||||||
|
|
||||||
|
- **Live in-place progress over the WHOLE sync.** New `_sync_progress` /
|
||||||
|
`_sync_progress_done` helpers render `checking N/48 lib/foo.sh` per entry,
|
||||||
|
rewriting the line via `\r\033[K` (carriage-return + clear-line). When a fetch
|
||||||
|
actually lands a changed file, the frame switches to `downloading N/48
|
||||||
|
lib/foo.sh` so real writes are distinguishable from the common unchanged case.
|
||||||
|
The current filename is always shown, so a genuine stall is VISIBLE — you see
|
||||||
|
exactly which file it is stuck on instead of a blank freeze.
|
||||||
|
- **MobaXterm-safe escapes only.** Uses solely `\r` + `ESC[K` (the same
|
||||||
|
primitive already at the readline prompt, audited safe in the v0.8.7 escape
|
||||||
|
inventory). Deliberately NO DECSTBM scroll-region, cursor save/restore, or
|
||||||
|
absolute-row addressing — the exact escapes MobaXterm mis-honors. Verified
|
||||||
|
under a pty: only `ESC[0m`, `ESC[2m`, `ESC[K` ever emitted; zero forbidden
|
||||||
|
escapes. Independent of `LARRY_NO_STREAM` / mouse mode (gates only on a TTY).
|
||||||
|
- **Non-TTY safe.** Gates on `[ -t 2 ]`. When stderr is a pipe/log it emits a
|
||||||
|
plain newline-terminated heartbeat every 10 files (no `\r`), so captured logs
|
||||||
|
stay clean — verified zero `\r` bytes in piped output.
|
||||||
|
- **Heartbeat for hangs.** The per-file fetch already carries curl `--max-time`
|
||||||
|
(5s for VERSION, 15s otherwise); a stuck file now shows its name, times out,
|
||||||
|
counts as a fail, and the loop advances — never an infinite silent stall.
|
||||||
|
- **Speedup deferred (by design).** The clean fix — fetch the manifest once,
|
||||||
|
compare per-file hashes locally, and SKIP unchanged files with NO per-file
|
||||||
|
network round-trip — requires the MANIFEST to carry hashes. It currently
|
||||||
|
carries **paths only** (no hashes/sizes), so the optimization would require a
|
||||||
|
manifest-format + release-tooling change (higher risk, separate change). Filed
|
||||||
|
as a follow-up: `MANIFEST` should emit `path<TAB>sha256` so v0.9.x can turn 48
|
||||||
|
network round-trips into 1 fetch + local compare. This release ships the
|
||||||
|
indicator (Bryan's explicit ask); correctness of the sync is unchanged.
|
||||||
|
|
||||||
|
Note: the v0.8.9 relaunch itself is still the slow path (the indicator helps
|
||||||
|
from the NEXT update onward), but you now see live forward progress during it.
|
||||||
|
|
||||||
## v0.8.8 — 2026-05-27
|
## v0.8.8 — 2026-05-27
|
||||||
|
|
||||||
Force unconditional 429 header capture (Clover). Symptom: Bryan's MobaXterm
|
Force unconditional 429 header capture (Clover). Symptom: Bryan's MobaXterm
|
||||||
|
|||||||
84
larry.sh
84
larry.sh
@ -65,7 +65,7 @@ set -o pipefail
|
|||||||
# ─────────────────────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
# Config
|
# Config
|
||||||
# ─────────────────────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
LARRY_VERSION="0.8.8"
|
LARRY_VERSION="0.8.9"
|
||||||
LARRY_HOME="${LARRY_HOME:-$HOME/.larry}"
|
LARRY_HOME="${LARRY_HOME:-$HOME/.larry}"
|
||||||
|
|
||||||
# ─────────────────────────────────────────────────────────────────────────────
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
@ -474,6 +474,65 @@ _record_origin() {
|
|||||||
_LARRY_LAST_ORIGIN_URL="$2"
|
_LARRY_LAST_ORIGIN_URL="$2"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
# v0.8.9: manifest-sync progress indicator.
|
||||||
|
#
|
||||||
|
# WHY: phase-A sync (sync_from_manifest) fetches EVERY manifest entry over an
|
||||||
|
# authenticated HTTPS round-trip (Gitea via corporate proxy + Cloudflare), then
|
||||||
|
# uses `cmp` to decide if the byte stream actually changed. With 48 entries that
|
||||||
|
# is 48 sequential round-trips — ~3 min on Bryan's work-box — and it was
|
||||||
|
# COMPLETELY SILENT between "update found … relaunching" and "manifest sync: …".
|
||||||
|
# Looked frozen. This indicator shows live per-file forward progress so a real
|
||||||
|
# hang is visible (you see WHICH file it stalls on) and a slow-but-working sync
|
||||||
|
# is obviously alive.
|
||||||
|
#
|
||||||
|
# MobaXterm-safety: uses ONLY carriage-return + clear-line (\r\033[K), the same
|
||||||
|
# primitive already used at the readline prompt (see read_user_input) and
|
||||||
|
# audited MobaXterm-safe in the v0.8.7 escape inventory. Deliberately NO
|
||||||
|
# DECSTBM scroll-region, cursor-save/restore, or absolute-row addressing — those
|
||||||
|
# are exactly the escapes MobaXterm mis-honors (see render_status_line note).
|
||||||
|
#
|
||||||
|
# These run BEFORE lib/cygwin-safe.sh is sourced (self_update is early), so we
|
||||||
|
# strip CRs inline via parameter expansion rather than calling strip_cr.
|
||||||
|
#
|
||||||
|
# Output goes to stderr (fd 2) to sit alongside log()/warn()/err(), and the TTY
|
||||||
|
# gate is `[ -t 2 ]`. Non-TTY (pipe/log redirect): no \r — we emit a plain
|
||||||
|
# newline-terminated heartbeat line every $_SYNC_PROGRESS_PLAIN_EVERY files so a
|
||||||
|
# captured log shows progress without \r garbage.
|
||||||
|
# ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
_SYNC_PROGRESS_PLAIN_EVERY=10 # non-TTY: emit a plain heartbeat every N files
|
||||||
|
|
||||||
|
# _sync_progress PHASE CUR TOTAL LABEL — render one in-place progress frame.
|
||||||
|
# PHASE : short verb, e.g. "syncing" / "checking" / "downloading"
|
||||||
|
# CUR : current 1-based index
|
||||||
|
# TOTAL : denominator
|
||||||
|
# LABEL : current filename (or any short context string)
|
||||||
|
# TTY: rewrites the current line via \r\033[K (no scroll, no newline).
|
||||||
|
# non-TTY: prints a plain line only on the first, every-Nth, and (caller uses
|
||||||
|
# _sync_progress_done for) final frame, so logs stay readable.
|
||||||
|
_sync_progress() {
|
||||||
|
local phase="$1" cur="$2" total="$3" label="${4:-}"
|
||||||
|
# Strip any CR that rode in on the label (defensive; pre-cygwin-safe).
|
||||||
|
label="${label//$'\r'/}"
|
||||||
|
if [ -t 2 ]; then
|
||||||
|
printf '\r\033[K%s[%s]%s %s %s/%s %s' \
|
||||||
|
"$C_DIM" "$(date +%H:%M:%S)" "$C_RESET" "$phase" "$cur" "$total" "$label" >&2
|
||||||
|
else
|
||||||
|
# Plain mode: heartbeat on first frame and every Nth frame. No \r.
|
||||||
|
if [ "$cur" -eq 1 ] || [ $(( cur % _SYNC_PROGRESS_PLAIN_EVERY )) -eq 0 ]; then
|
||||||
|
printf '%s[%s]%s %s %s/%s\n' \
|
||||||
|
"$C_DIM" "$(date +%H:%M:%S)" "$C_RESET" "$phase" "$cur" "$total" >&2
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# _sync_progress_done — clear the in-place progress line on a TTY so the
|
||||||
|
# following summary log line lands clean. No-op on a non-TTY (nothing to clear).
|
||||||
|
_sync_progress_done() {
|
||||||
|
[ -t 2 ] && printf '\r\033[K' >&2
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
sync_from_manifest() {
|
sync_from_manifest() {
|
||||||
local base="$1"
|
local base="$1"
|
||||||
local manifest="$LARRY_HOME/.manifest.new"
|
local manifest="$LARRY_HOME/.manifest.new"
|
||||||
@ -491,6 +550,16 @@ sync_from_manifest() {
|
|||||||
local self="$0"
|
local self="$0"
|
||||||
case "$self" in /*) ;; *) self="$PWD/$self" ;; esac
|
case "$self" in /*) ;; *) self="$PWD/$self" ;; esac
|
||||||
|
|
||||||
|
# v0.8.9: pre-count manifest entries so the progress indicator has a
|
||||||
|
# denominator. Cheap local pass (no network) over the just-fetched manifest.
|
||||||
|
local total=0 _l
|
||||||
|
while IFS= read -r _l; do
|
||||||
|
case "$_l" in ''|'#'*) continue ;; esac
|
||||||
|
_l="${_l%%[[:space:]]*}"
|
||||||
|
[ -z "$_l" ] && continue
|
||||||
|
total=$((total + 1))
|
||||||
|
done < "$manifest"
|
||||||
|
|
||||||
local count=0 updated=0 failed=0 path tmp dest
|
local count=0 updated=0 failed=0 path tmp dest
|
||||||
while IFS= read -r path; do
|
while IFS= read -r path; do
|
||||||
case "$path" in ''|'#'*) continue ;; esac
|
case "$path" in ''|'#'*) continue ;; esac
|
||||||
@ -502,6 +571,13 @@ sync_from_manifest() {
|
|||||||
# the running script mid-execution.
|
# the running script mid-execution.
|
||||||
[ "$path" = "larry.sh" ] && continue
|
[ "$path" = "larry.sh" ] && continue
|
||||||
|
|
||||||
|
# v0.8.9: live progress BEFORE the network round-trip, so a fetch that
|
||||||
|
# hangs (slow file / proxy stall) shows exactly which file it is stuck on
|
||||||
|
# rather than freezing silently. fetch_validate's --max-time bounds each
|
||||||
|
# hang to ${_kind} fetch timeout (15s); on timeout it fails loud, counts as
|
||||||
|
# a fail, and the loop advances to the next file — never an infinite stall.
|
||||||
|
_sync_progress checking "$count" "$total" "$path"
|
||||||
|
|
||||||
dest="$LARRY_HOME/$path"
|
dest="$LARRY_HOME/$path"
|
||||||
tmp="$dest.new"
|
tmp="$dest.new"
|
||||||
mkdir -p "$(dirname "$dest")" 2>/dev/null
|
mkdir -p "$(dirname "$dest")" 2>/dev/null
|
||||||
@ -517,6 +593,9 @@ sync_from_manifest() {
|
|||||||
esac
|
esac
|
||||||
if fetch_validate "$base/$path" "$tmp" "$_kind" 15 && [ -s "$tmp" ]; then
|
if fetch_validate "$base/$path" "$tmp" "$_kind" 15 && [ -s "$tmp" ]; then
|
||||||
if [ ! -f "$dest" ] || ! cmp -s "$dest" "$tmp"; then
|
if [ ! -f "$dest" ] || ! cmp -s "$dest" "$tmp"; then
|
||||||
|
# Distinguish a real change (a download that lands) from the common
|
||||||
|
# case of an unchanged file, so the operator sees actual writes.
|
||||||
|
_sync_progress downloading "$count" "$total" "$path"
|
||||||
mv "$tmp" "$dest"
|
mv "$tmp" "$dest"
|
||||||
case "$path" in *.sh) chmod +x "$dest" 2>/dev/null || true ;; esac
|
case "$path" in *.sh) chmod +x "$dest" 2>/dev/null || true ;; esac
|
||||||
updated=$((updated + 1))
|
updated=$((updated + 1))
|
||||||
@ -530,6 +609,9 @@ sync_from_manifest() {
|
|||||||
done < "$manifest"
|
done < "$manifest"
|
||||||
rm -f "$manifest"
|
rm -f "$manifest"
|
||||||
|
|
||||||
|
# v0.8.9: clear the in-place progress line so the summary lands clean.
|
||||||
|
_sync_progress_done
|
||||||
|
|
||||||
if [ "$updated" -gt 0 ] || [ "$failed" -gt 0 ]; then
|
if [ "$updated" -gt 0 ] || [ "$failed" -gt 0 ]; then
|
||||||
log "manifest sync: $updated updated, $failed failed, $count total (from $base)"
|
log "manifest sync: $updated updated, $failed failed, $count total (from $base)"
|
||||||
fi
|
fi
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user