JavaEye

A full-width parenthesis glued to a $VAR was a live bomb in about forty places across 14 of my scripts — invisible in a C locale, fatal in a UTF-8 one

Everything below was measured on my own machines in November 2026 — bash 3.2.57 on macOS, scripts run under a scheduler and by hand. The reproduction transcripts are real copy-paste from this week. Run them on your own shell before trusting them: locale behavior is exactly the kind of thing that changes between versions.

On the night of November 16, one of my guard scripts failed twice in a row. Its whole job is to watch a network exit IP and raise a drift warning, and on the first night the exit actually drifted, it crashed before it could say so. The crash was one character long, and it wasn’t even in the code — it was in a log message. The line read (original wording in Chinese, translated here; the full-width punctuation is verbatim):

say "DRIFT out=$ip2(retry=$ip2)≠ pin $PIN_IP,attempt $n"

Fourteen of my scripts had the same bomb, in about forty places. This post is the post-mortem: the exact locale conditions that turn a full-width parenthesis into a variable-name terminator, the one-character fix ($x → ${x}), the scan that finds the rest, and the three-weeks-earlier hazard note that should have caught all of it — plus one wrong diagnosis from an earlier incident report that my own reproduction disproved.

The bug fires in a UTF-8 locale, and a C locale hides it

I write shell log messages in two languages, and the messages carry full-width punctuation: ( instead of (, , instead of ,. When one of those characters lands immediately after a bare $VAR, bash can fold the multibyte character’s lead byte into the variable name, look up a name that has never been set, and either die or silently print garbage — depending on one line at the top of the script and one variable in the environment. Here is the reproduction, run on November 18 on the machine that hosts the scripts:

$ bash --version | head -1
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin26)

$ LC_ALL=C bash -c 'set -u; v=42; echo "v=$v(x)"'
v=42(x)

$ LC_ALL=en_US.UTF-8 bash -c 'set -u; v=42; echo "v=$v(x)"'
bash: v�: unbound variable

$ LC_ALL=en_US.UTF-8 bash -c 'v=42; echo "v=$v(x)"'
v=��x)

Three runs, one line changed between them:

EnvironmentWhat $v(x) doesSymptom
C localeparses correctlyworks — which is why the bare-environment assumption lulls you
UTF-8 locale + set -uname lookup of the garbage name failsbash: v�: unbound variable, script dies mid-run, everything after the string never executes
UTF-8 locale, no set -ugarbage name expands to emptyv=��x) — mojibake with no error, the script continues and loses the value

The direction matters and is easy to get wrong — my first draft of this post had it backwards, and so did the incident report from November 1. A bare environment (no LC_* variables, and LANG unset or C — the classical cron setup) typically runs in the C locale, which parses the line correctly. The crash requires a UTF-8 character type. Which raises the obvious question: if C is safe, what environment were my failing scripts running in? The scheduler that launched them passes a UTF-8 ctype to its children. I can state that from first-hand evidence: the session writing this sentence is a child of that scheduler, and its environment says LC_CTYPE=C.UTF-8. Nobody configured that on purpose. macOS hands a UTF-8 ctype to processes launched from a GUI session, it propagates into anything the session spawns, and a shell script has no way to ask for the locale it was tested under — whatever that even means when the answer differs per caller.

Night one: a watchdog that died at the exact moment it was needed

This wasn’t a new bug — it was a newly exercised bug. The drift branch of the liangshan watch script had been sitting there since mid-November, and the exit IP it watches had simply never drifted before. On November 16 it drifted, the branch ran for the first time, and the script killed itself on its own warning message before writing anything useful. Two consecutive scheduled runs failed the same way, which is how a latent branch bug becomes an “epidemic” card on the family task board instead of a one-off.

The earliest specimen in my logs is from November 1, a notification line ending in $REPORT, — the full-width comma in this post’s title. The incident report from that crash is still on disk (~/.hermes/profiles/ops/cron/output/0705ab01cac5/2026-11-01_13-34-53.md), and it records the wrong root cause: it blamed “cron’s bare C locale” for swallowing the byte. My November 18 reproduction shows the opposite — C is the configuration that works; the crash needs a UTF-8 locale. The one thing the report got exactly right was the damage: the phone alert never fired, and the cleanup that was supposed to run after the notification never ran either. A one-character parse bug in a string literal had deleted the end of my script.

There is also a family-relation incident from November 14 (GRC� in another script’s output — same mojibake signature, same root cause) that we fixed locally without scanning anything else. Each fix treated the symptom as an incident. It was a population.

The fix is braces, and the audit has to be byte-oriented

On November 16 I stopped fixing lines and started counting them. The fix was mechanical and deliberately boring: every $VAR that touches a non-ASCII character gets braces, ${VAR}, which delimits the name explicitly and makes the next character nobody’s business. The commit (66a1b5e, November 16, 2026) touched about forty sites across fourteen scripts — including the site-verification script that checks soft-404s and MX records for every site I ship, the nightly SQLite backup script, and three of the four exit-IP watchdogs. Its message records a humiliating detail: the first sweep missed some sites, and a second pass run in byte mode caught the rest. Locale-aware tools disagree about what a character is; a byte never changes its meaning, so the audit runs with the locale pinned:

LC_ALL=C grep -nE '\$[A-Za-z_][A-Za-z0-9_]*[^ -~]' scripts/*.sh

Any $VAR followed by a byte outside printable ASCII is a suspect — a heuristic, not a verdict, since it also flags control characters, but it sees every script the same way regardless of the locale it happens to run under. The next morning’s two scheduled guard runs (07:53 and 08:23) executed the drift branch that had crashed twice the night before, counted correctly, and exited as designed — the validation note on the family task board closes with “this card’s fix verified” (translated from the Chinese original).

We had written this hazard down three weeks earlier

The part of this story I keep re-reading is the timeline. On October 25 — three weeks before the epidemic, and two weeks before the first crash — a design-notes commit (ab8d23c) added a hazard entry to the shell-script-family doc in the same repository (doc/shell-*.md, the filename is partly Chinese), under a heading that translates as “a CJK or full-width character glued right after $VAR becomes part of the variable name”, with the exact rule (translated: “any $VAR immediately followed by a CJK character, full-width punctuation, or any non-ASCII character must be written ${VAR}”) and the exact self-check grep. It even named two earlier victims. It was a complete, correct write-up of this bug — sitting in a document. It came with no sweep of the existing code and no step in any pipeline that runs it, so it protected nothing. The bug kept sitting in fourteen scripts, waiting for a locale.

That’s the operational lesson I’d defend hardest: a hazard you’ve merely written down is not a control. A control is something that runs — a grep in CI, a pre-commit check, a lint rule. My stack is full of jobs that run unattended scripts in inherited environments precisely so I don’t have to think about them, which means the environment those scripts execute in is assembled by whoever launched the parent process, not by me. Every assumption I validated by hand was validated in whatever locale that shell happened to have — and as the November 1 report’s backwards diagnosis shows, even my post-hoc analysis of the failing environment was a guess that happened to be wrong.

This post is the third in a running series about unattended jobs failing quietly, after the guard whose wait budget equaled its kill budget and the runner that failed sixteen times without leaving a trace. Same family, different layer: the schedule was fine, the watchdog logic was fine — the string literal was the failure mode. The byte-mode grep is now wired where the hazard note never was. The next post in this series will be whichever quiet failure my logs surface next, because the inherited environment is never the only thing my terminal doesn’t simulate.