Commit1ba27853Recorded22 Aug 2026Repositorysigil-doc

test: make every setsid invocation wait, and fix four gates that lied

Message

The 22 setsid call sites under test/ split 16 bare against 6 that waited. A bare setsid <cmd> does not reliably block: util-linux forks when the setsid PROCESS is itself a process-group leader, which bash makes it under job control, and the parent returns 0 at once while the child runs on. Measured here under bash -i on a pty:

    bare  exit=0  elapsed_ms=5     logbytes=0
    -w    exit=0  elapsed_ms=3011  logbytes=16

and in the if ! setsid ... form the bare version reported SUCCESS for a child that exited 1. This repo has no CI; every gate here is run by hand, so the interactive prompt is the normal case rather than the exotic one. All 16 now use -w; gate-sd-p3.sh was the in-repo precedent.

Note for whoever reads the original ticket: command substitution and pipelines block even on the fork branch, because the shell waits for EOF and the grandchild holds the write end (measured 3020ms). The racy shape is redirect-to-a-file. gate-d2.sh:25 and gate-d3.sh:25 were never racy.

FOUR GATES THAT COULD NOT FAIL, OR THAT LIED:

  • gate-d2.sh's cache-reuse arm is an ABSENCE assertion -- stat, load the manuscript, stat again, require no change -- with nothing in front of it. On the fork branch the load has not run when the second stat is taken. Reproduced in that exact shape: bare -> GREEN on an artifact the child demonstrably had rewritten, -w -> RED. It now carries two controls asserting different things: that a rebuild lands at the watched path, and that the stat probe sees an in-place rewrite. Induced red confirmed.
  • gate-d3.sh arm 2 asserted rg -q '7|8|'. The empty alternative matches every non-empty line; measured, it accepted ERROR: everything exploded, a wrong example count, and a zero-length rendering. Both counts are now derived from the fixture (7 @example plus 1 @project-check), with controls in both directions. Neither end is anchored, because the eval echoes its own return value: the real line is 7|8|531#<undefined>.
  • gate-sd-p1.sh arm 1 asserted a hardcoded 11 passed (11 total) against a suite that declares and passes 12. It has been RED ON MASTER, accusing healthy product code -- confirmed by running a pristine copy from git show HEAD:. The count is now derived, with two assertions rather than one: every declared test must be COLLECTED, and every collected test must have PASSED. gate-sd-p4.sh:45 hardcodes 17 the same way and is the next to go stale; surveyed, not changed here.
  • sabotage-sd-p1.sh's closing run -- the only thing in the driver proving red->green -- asserted nothing about its output and printed its completion marker regardless. It now requires the runner's own banner and summary with passed == total, no reported failures, and a floor derived from the legs' own collected counts. Five controls run before it.

Also: sabotage-sd-p1.sh:91 expanded $label inside the same local that bound it, so under set -u every leg wrote to $work/.log and printed label: unbound variable -- seven times in test/transcripts/sd-p1/ sabotage.txt. Split; the run now produces nine distinct per-leg logs and zero unbound-variable errors.

New test/gate-setsid-waits.sh keeps the class closed: it derives the population from every *.sh in the repo and refuses any invocation whose option run lacks -w/--wait. 9/9 planted bare forms flagged, including ones inside quoted strings, eval and sh -c; 0/8 waiting or inert forms misflagged. Its one blind spot -- invocation through a variable -- is stated in the file rather than left to be discovered.

Evidence, all on a quiet box with load recorded at both ends: gate-d2 green; gate-d3 D3_GATE_GREEN_25_25 including arm 10; gate-sd-p4 SD_P4_GATE_GREEN; gate-sd-p1 SD_P1_GATE_GREEN at 12/12; sabotage-sd-p1 8 legs red, closing suite green over 12 collected tests; sabotage-sd-p4 8 legs red, SD_P4_SABOTAGE_COMPLETE, all targets verified against HEAD by sha256 afterwards.

Changed
 test/gate-d2.sh           |  73 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 test/gate-d3.sh           |  53 +++++++++++++++++++++++++++++++++++++++++++++---
 test/gate-sd-p1.sh        |  47 ++++++++++++++++++++++++++++++++++++------
 test/gate-sd-p4.sh        |   6 +++---
 test/gate-setsid-waits.sh | 185 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 test/sabotage-sd-p1.sh    | 140 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 test/sabotage-sd-p4.sh    |   4 ++--
 7 files changed, 486 insertions(+), 22 deletions(-)
Diff
test/gate-d2.shmodified
@@ -20,17 +20,84 @@ export XDG_CACHE_HOME="$cache_dir/real"
20
21
# Compile the package and run the focused contract first. The real-language
22
# controls below then exercise the public #lang path against those exact bytes.
−23
setsid "$sigil" -L src -L test test test/test-reader.sgl --report
+23
setsid -w "$sigil" -L src -L test test test/test-reader.sgl --report
24
−25
observed=$(setsid "$sigil" -L src -L test eval '(begin (import (sigil doc)) (load "test/fixtures/practical-sigil-chapter.sgd") (display (document-title (eval (quote document)))) (display "|") (display (eval (quote tip-calls))) (display "|") (display (document->plain (eval (quote document)))))')
+25
observed=$(setsid -w "$sigil" -L src -L test eval '(begin (import (sigil doc)) (load "test/fixtures/practical-sigil-chapter.sgd") (display (document-title (eval (quote document)))) (display "|") (display (eval (quote tip-calls))) (display "|") (display (document->plain (eval (quote document)))))')
26
case "$observed" in
27
*"A Useful First Script|2|"*"macro-expanded phrase"*"computed inline expression"*"Computed Section 2"*) ;;
28
*) echo "D2 authoring positive control failed" >&2; exit 1 ;;
29
esac
30
reader_sgb=$(find "$cache_dir/real" -path '*/sigil/doc/reader.sgb' -print -quit)
31
test -n "$reader_sgb"
+32
# POSITIVE CONTROLS FOR THE ABSENCE ASSERTION BELOW -- RUN FIRST, DELIBERATELY.
+33
# Until they pass, the assertion they guard means nothing, so they do not run
+34
# after it.
+35
#
+36
# `test "$before" = "$after"` below claims that loading the manuscript through
+37
# the public #lang path did NOT recompile the reader. That is an assertion that
+38
# NOTHING happened, and it is satisfied for free by anything that stops the
+39
# `eval` from running at all.
+40
#
+41
# It was reachable, not hypothetical. Until 2026-08-22 that line was a bare
+42
# `setsid`, which forks and returns 0 at once whenever the setsid process is a
+43
# process-group leader -- which is what bash makes it at an interactive prompt,
+44
# and by hand at a prompt is the only way anything in this repo is run.
+45
#
+46
# WHAT WAS MEASURED, STATED PRECISELY SO NOBODY READS IT AS "THIS GATE IS RED":
+47
# the measurement was of this ARM'S SHAPE -- stat, run a child that rewrites the
+48
# artifact, stat again -- with a synthetic 3-second child, under `bash -i` on a
+49
# pty. The bare form reported GREEN for an artifact the child demonstrably HAD
+50
# rewritten; `-w` reported RED. It says the shape can produce a false green. It
+51
# says nothing about whether the real manuscript recompiles the real reader, and
+52
# the assertion below is expected to hold.
+53
#
+54
# `-w` closes the race. These two controls close the other half, and they assert
+55
# DIFFERENT things:
+56
#
+57
# (a) A REBUILD LANDS AT THIS EXACT PATH. Without it, a recompile that wrote
+58
# somewhere else would leave `before = after` true while the very thing the
+59
# arm rules out had happened.
+60
# (b) THE `stat` PROBE SEES A REWRITE IN PLACE. That is the actual failure mode
+61
# -- an existing artifact overwritten, not a deleted one recreated -- and
+62
# (a) alone cannot show it, because a file that was removed and recreated
+63
# changes its stat triple by construction the moment it exists again.
+64
#
+65
# Both work in a THROWAWAY COPY of the cache, so the working tree is untouched
+66
# and no restore step has to be reached. The copy is removed afterwards: it is
+67
# under `build/` and therefore invisible to git, which is not the same as free.
+68
control_cache="$cache_dir/control"
+69
cp -a "$cache_dir/real" "$control_cache"
+70
control_sgb=$(find "$control_cache" -path '*/sigil/doc/reader.sgb' -print -quit)
+71
test -n "$control_sgb" || { echo "positive control: no reader.sgb in the copied cache" >&2; exit 1; }
+72
+73
# (b) first, because it needs no compiler and a failure here invalidates (a) too.
+74
inplace_before=$(stat -c '%y:%s:%i' "$control_sgb")
+75
printf '\0' >>"$control_sgb"
+76
inplace_after=$(stat -c '%y:%s:%i' "$control_sgb")
+77
test "$inplace_before" != "$inplace_after" || {
+78
echo "POSITIVE CONTROL (b) FAILED: the stat probe cannot see an in-place rewrite." >&2
+79
echo " before: $inplace_before" >&2
+80
echo " after: $inplace_after" >&2
+81
echo " The 'before = after' assertion below therefore proves nothing." >&2; exit 1; }
+82
+83
# (a) now: delete the compiled reader from the copy and re-run the same
+84
# manuscript against that cache. It must be written back to the same path.
+85
rm -f "$control_sgb"
+86
XDG_CACHE_HOME="$control_cache" setsid -w "$sigil" -L src -L test \
+87
eval -f test/fixtures/practical-sigil-chapter.sgd >"$cache_dir/control.log" 2>&1 || {
+88
echo "positive control (a): the forced-recompile run failed" >&2
+89
tail -30 "$cache_dir/control.log" >&2; exit 1; }
+90
test -f "$control_sgb" || {
+91
echo "POSITIVE CONTROL (a) FAILED: a recompile did not land at the path this arm watches." >&2
+92
echo " expected: $control_sgb" >&2
+93
echo " A rebuild could therefore happen without moving 'before'/'after' at all." >&2; exit 1; }
+94
rm -rf "$control_cache"
+95
echo "D2 cache-reuse probe armed: in-place rewrite visible to stat ($inplace_before -> $inplace_after); a rebuild lands at the watched path"
+96
+97
# THE ASSERTION ITSELF, now that both halves of the probe have been shown to fire.
98
before=$(stat -c '%y:%s:%i' "$reader_sgb")
−33
setsid "$sigil" -L src -L test eval -f test/fixtures/practical-sigil-chapter.sgd
+99
setsid -w "$sigil" -L src -L test eval -f test/fixtures/practical-sigil-chapter.sgd
100
after=$(stat -c '%y:%s:%i' "$reader_sgb")
101
test "$before" = "$after"
+102
103
echo "D2_REAL_LANGUAGE_GREEN"
test/gate-d3.shmodified
@@ -22,15 +22,62 @@ expect_fail() {
22
rg -q "$pattern" "$log"
23
}
24
−25
setsid "$sigil" -L src -L test test test/test-examples.sgl --report | tee "$gate_dir/model.log"
+25
setsid -w "$sigil" -L src -L test test test/test-examples.sgl --report | tee "$gate_dir/model.log"
26
rg -q '5 passed' "$gate_dir/model.log"; pass 1 MODEL_5_5
27
28
# Loading, traversal, and rendering are passive: the deliberately infinite
29
# timeout subject is present but these operations terminate without a verifier.
−30
setsid "$sigil" -L src -L test eval \
+30
setsid -w "$sigil" -L src -L test eval \
31
'(begin (import (sigil doc) (sigil doc examples)) (load "test/fixtures/d3-examples.sgd") (define d (eval (quote document))) (display (length (document-examples d))) (display "|") (display (length (document-events d))) (display "|") (display (string-length (document->plain d))))' \
32
>"$gate_dir/passive.log"
−33
rg -q '7|8|' "$gate_dir/passive.log"; pass 2 PASSIVE_LOAD_TRAVERSE_RENDER
+33
# ARM 2'S ASSERTION WAS `rg -q '7|8|'`, WHICH MATCHES EVERY NON-EMPTY LINE.
+34
#
+35
# The intent was the probe's literal `|`-delimited output. As a REGEX, `7|8|` is
+36
# "7, or 8, or the empty string", and the empty alternative matches anything.
+37
# Measured 2026-08-22: it matches `total nonsense here`, and it matches
+38
# `ERROR: everything exploded`. The only input it refused was a completely empty
+39
# file -- which is what the bare `setsid` above it made likely, so the race was
+40
# the only thing giving this arm any teeth at all. Making the command wait, on
+41
# its own, would have achieved nothing here.
+42
#
+43
# BOTH COUNTS ARE NOW DERIVED FROM THE FIXTURE rather than written in, so a
+44
# fixture that gains an example fails here instead of going silently unchecked.
+45
# The event stream carries one event per example plus one per project-check.
+46
d3_fixture=test/fixtures/d3-examples.sgd
+47
expect_examples=$(grep -cE '^@example\[' "$d3_fixture")
+48
expect_checks=$(grep -cE '^@project-check\[' "$d3_fixture")
+49
expect_events=$((expect_examples + expect_checks))
+50
test "$expect_examples" -ge 1 || {
+51
echo "derived 0 examples from $d3_fixture; the derivation is broken, not the product" >&2; exit 1; }
+52
# NEITHER END IS ANCHORED, and both halves of that are measured rather than
+53
# assumed. The compiler may emit chatter on the same stdout line the probe's
+54
# first `display` continues; and the `eval` ECHOES ITS OWN RETURN VALUE after the
+55
# three displays, so the real line is `7|8|531#<undefined>` and a `$` anchor
+56
# fails on it. A `[^0-9]` bound on each side is what makes `17|8|...` and
+57
# `...|8|0` both fail while the real output passes.
+58
passive_pattern="(^|[^0-9])${expect_examples}\|${expect_events}\|[1-9][0-9]*([^0-9]|\$)"
+59
# POSITIVE CONTROLS, BOTH DIRECTIONS AND THEN SOME. The pattern must REFUSE what
+60
# the old one accepted, ACCEPT the real observed line -- otherwise a pattern that
+61
# refuses everything satisfies the first check for free -- and refuse a zero
+62
# length, which is the shape a stalled or empty render produces.
+63
#
+64
# The "known-good" control is the line this probe ACTUALLY EMITS, echoed return
+65
# value and all, not a tidied version of it. A control built from what the output
+66
# ought to look like is how a pattern ends up passing its own controls and
+67
# failing the product.
+68
printf 'ERROR: everything exploded\n' >"$gate_dir/passive-control-bad.log"
+69
if rg -q "$passive_pattern" "$gate_dir/passive-control-bad.log"; then
+70
echo "arm 2's pattern matches arbitrary output; it is not an assertion" >&2; exit 1; fi
+71
printf '%s|%s|0#<undefined>\n' "$expect_examples" "$expect_events" >"$gate_dir/passive-control-zero.log"
+72
if rg -q "$passive_pattern" "$gate_dir/passive-control-zero.log"; then
+73
echo "arm 2's pattern accepts a zero-length rendering" >&2; exit 1; fi
+74
printf '%s|%s|531#<undefined>\n' "$expect_examples" "$expect_events" >"$gate_dir/passive-control-ok.log"
+75
rg -q "$passive_pattern" "$gate_dir/passive-control-ok.log" || {
+76
echo "arm 2's pattern rejects the shape this probe really emits; it is a wall, not a check" >&2; exit 1; }
+77
rg -q "$passive_pattern" "$gate_dir/passive.log" || {
+78
echo "passive load/traverse/render did not report ${expect_examples}|${expect_events}|<non-zero length>" >&2
+79
echo "observed:" >&2; cat "$gate_dir/passive.log" >&2; exit 1; }
+80
pass 2 "PASSIVE_LOAD_TRAVERSE_RENDER ${expect_examples} examples, ${expect_events} events, both derived from $d3_fixture"
81
82
verify=("$sigil" -L src -L test tools/sigil-doc-verify.sgl test/fixtures/practical-sigil-chapter.sgd)
83
"${verify[@]}" --no-cache --example stdout | tee "$gate_dir/stdout.log"
test/gate-sd-p1.shmodified
@@ -40,14 +40,49 @@ test -f test/transcripts/sd-p1/classification-before.txt
40
suite_log="$gate_dir/root-profile-suite.log"
41
# NOT `--report`: that flag captures the suite's stdout, and this arm asserts on
42
# a denominator the suite PRINTS.
−43
if ! setsid "$sigil" -L src -L test test test/test-root-profile.sgl >"$suite_log" 2>&1; then
+43
if ! setsid -w "$sigil" -L src -L test test test/test-root-profile.sgl >"$suite_log" 2>&1; then
44
echo "root-profile suite exited non-zero" >&2; tail -40 "$suite_log" >&2; exit 1
45
fi
46
# Strip ANSI colour before matching, so the assertion depends on the runner's
47
# words rather than on its escape sequences.
48
sed -e 's/\x1b\[[0-9;]*m//g' "$suite_log" >"$suite_log.clean"
−49
rg -q 'Tests:\s+11 passed \(11 total\)' "$suite_log.clean" || {
−50
echo "root-profile suite did not report 11 of 11 tests" >&2
+49
# THE COUNT IS DERIVED FROM THE SUITE FILE, NOT WRITTEN IN.
+50
#
+51
# This line asserted a hardcoded `Tests: 11 passed (11 total)`. A TWELFTH test
+52
# was added to test/test-root-profile.sgl and the literal went stale, so this
+53
# gate has been RED ON MASTER against a suite that passes 12/12 -- the gate
+54
# lying about a healthy product. Confirmed 2026-08-22 by running a pristine copy
+55
# built from `git show HEAD:`: exit 1, "did not report 11 of 11 tests", with the
+56
# runner reporting `Tests: 12 passed (12 total)` and `All tests passed`.
+57
#
+58
# That is the same defect class as an assertion that cannot fail, pointing the
+59
# other way: a check that has stopped describing its subject. folio's
+60
# test/gate-f1a.sh states the rule in its own header and derives its count
+61
# exactly like this -- "Every count this gate checks is DERIVED FROM THE ARTIFACT
+62
# it describes, never written in as a literal. [...] That drift is the single
+63
# most common defect in this estate's gates." Adding a test must not require
+64
# editing a number here, and now it does not.
+65
#
+66
# NOTE FOR WHOEVER TOUCHES THE SIBLING: test/gate-sd-p4.sh:45 hardcodes 17 the
+67
# same way. It still matches its suite, so it is green today and is the next one
+68
# to go stale. Surveyed, deliberately not changed in this branch.
+69
expected_tests=$(grep -cE '^\(test "' test/test-root-profile.sgl)
+70
test "$expected_tests" -ge 1 || {
+71
echo "derived 0 tests from test/test-root-profile.sgl; the DERIVATION is broken, not the suite" >&2
+72
exit 1; }
+73
reported=$(rg -o 'Tests:[[:space:]]+[0-9]+ passed' "$suite_log.clean" | head -1 | grep -oE '[0-9]+' | head -1)
+74
collected=$(rg -o '\([0-9]+ total\)' "$suite_log.clean" | head -1 | grep -oE '[0-9]+' | head -1)
+75
test -n "$reported" -a -n "$collected" || {
+76
echo "root-profile suite printed no summary line" >&2
+77
tail -40 "$suite_log.clean" >&2; exit 1; }
+78
# Every test the FILE declares must have been collected, and every test collected
+79
# must have passed. The first catches a test that silently stopped running; the
+80
# second catches a failure the summary would otherwise bury.
+81
test "$collected" = "$expected_tests" || {
+82
echo "test/test-root-profile.sgl declares $expected_tests tests but the runner collected $collected" >&2
+83
tail -40 "$suite_log.clean" >&2; exit 1; }
+84
test "$reported" = "$collected" || {
+85
echo "root-profile suite reported $reported passed of $collected collected" >&2
86
tail -40 "$suite_log.clean" >&2; exit 1; }
87
rg -q 'All tests passed' "$suite_log.clean" || {
88
echo "root-profile suite did not report all tests passing" >&2
@@ -58,7 +93,7 @@ roots=$(rg -o 'SD_P1_ROOTS_CHECKED [0-9]+' "$suite_log.clean" | head -1 | awk '{
93
test -n "$roots" || {
94
echo "suite printed no root-count denominator" >&2; tail -40 "$suite_log.clean" >&2; exit 1; }
95
test "$roots" -ge 2 || { echo "location walk covered only $roots roots" >&2; exit 1; }
−61
pass 1 "root-profile suite 11/11, locations verified at $roots roots"
+96
pass 1 "root-profile suite $reported/$expected_tests, locations verified at $roots roots"
97
98
# ---------------------------------------------------------------- ARM 02
99
# The classification change is CONFINED to the two colliding spellings.
@@ -75,7 +110,7 @@ rg -q 'PRE_SD_P1_CONTROL_UNBOUND' "$before" || {
110
echo "the BEFORE transcript carries no pre-SD-P1 positive control; it cannot be trusted" >&2
111
exit 1; }
112
after_log="$gate_dir/classification-after.txt"
−78
setsid "$sigil" -L src -L test eval -f test/sd-p1-classification-probe.sgl >"$after_log" 2>&1
+113
setsid -w "$sigil" -L src -L test eval -f test/sd-p1-classification-probe.sgl >"$after_log" 2>&1
114
# Derived, not hardcoded -- see test/sd-p1-before-transcript.sh for why.
115
cases=$(rg -c '^\(report "' test/sd-p1-classification-probe.sgl || echo 0)
116
test "$cases" -ge 1 || { echo "could not count the probe's cases" >&2; exit 1; }
@@ -177,7 +212,7 @@ cat >"$gate_dir/delegate-probe.sgl" <<'PROBE'
212
(display "SD_P1_DELEGATION roots=") (display (length (cadr parts)))
213
(display " forms=") (display (length (car parts))) (newline)
214
PROBE
−180
setsid "$sigil" -L src -L "$gate_dir" eval -f "$gate_dir/delegate-probe.sgl" >"$delegate_log" 2>&1 || {
+215
setsid -w "$sigil" -L src -L "$gate_dir" eval -f "$gate_dir/delegate-probe.sgl" >"$delegate_log" 2>&1 || {
216
echo "delegation probe failed" >&2; cat "$delegate_log" >&2; exit 1; }
217
rg -q 'SD_P1_DELEGATION roots=2 forms=0' "$delegate_log" || {
218
echo "delegation probe did not deliver two roots from the public module" >&2
test/gate-sd-p4.shmodified
@@ -36,7 +36,7 @@ suite_log="$gate_dir/passive-suite.log"
36
# NOT `--report`: that flag captures the suite's stdout, and this arm asserts on
37
# a denominator the suite PRINTS. Running with it made this gate exit silently
38
# on a suite that had actually passed 14/14.
−39
if ! setsid "$sigil" -L src -L test test test/test-passive-parse.sgl >"$suite_log" 2>&1; then
+39
if ! setsid -w "$sigil" -L src -L test test test/test-passive-parse.sgl >"$suite_log" 2>&1; then
40
echo "passive suite exited non-zero" >&2; tail -40 "$suite_log" >&2; exit 1
41
fi
42
# Strip ANSI colour before matching, so the assertion depends on the runner's
@@ -65,7 +65,7 @@ witness_control="$gate_dir/witness-control.log"
65
# (a) POSITIVE CONTROL FIRST. The fixture's markers must appear when the
66
# document is genuinely evaluated through the real #lang path. Until this
67
# passes, the absence assertion below means nothing.
−68
setsid "$sigil" -L src eval -f test/fixtures/sd-p4-witness.sgd >"$witness_control" 2>&1
+68
setsid -w "$sigil" -L src eval -f test/fixtures/sd-p4-witness.sgd >"$witness_control" 2>&1
69
rg -q 'SDP4-WITNESS-AUTHOR-EXPRESSION-RAN' "$witness_control" || {
70
echo "positive control: evaluating the fixture did NOT run its author expression" >&2
71
cat "$witness_control" >&2; exit 1; }
@@ -75,7 +75,7 @@ rg -q 'SDP4-WITNESS-AUTHOR-COMMAND-RAN' "$witness_control" || {
75
76
# (b) THE ASSERTION. Passive inspection, in a fresh process importing only the
77
# inspection surface, must run none of it.
−78
setsid "$sigil" -L src -L test eval -f test/sd-p4-witness-probe.sgl >"$witness_passive" 2>&1
+78
setsid -w "$sigil" -L src -L test eval -f test/sd-p4-witness-probe.sgl >"$witness_passive" 2>&1
79
rg -q 'SD_P4_WITNESS_PROBE_INSPECTED' "$witness_passive" || {
80
echo "witness probe did not run to completion" >&2; cat "$witness_passive" >&2; exit 1; }
81
# ...and it must have actually traversed something.
test/gate-setsid-waits.shadded
@@ -0,0 +1,185 @@
+1
#!/usr/bin/env bash
+2
#
+3
# No shell script in this repository may invoke the session wrapper without
+4
# telling it to wait.
+5
#
+6
# WHY THIS IS A GATE AND NOT A ONE-TIME SWEEP.
+7
#
+8
# `setsid <cmd> > log` does not reliably block. util-linux forks when the setsid
+9
# PROCESS is itself a process-group leader and the parent returns 0 at once while
+10
# the child runs on. Whether that happens is a property of the CALLING SHELL, not
+11
# of the command: bash puts every job in its own process group when job control
+12
# is on, so the same line waits inside a script and does NOT wait when a human
+13
# runs it at an interactive prompt. Measured on this host 2026-08-22, `bash -i`
+14
# over a pty:
+15
#
+16
# bare exit=0 elapsed_ms=5 logbytes=0
+17
# -w exit=0 elapsed_ms=3011 logbytes=16
+18
#
+19
# and in the `if ! ...` form the bare version reported SUCCESS for a child that
+20
# exited 1. Every gate in this repo is run by hand -- there is no CI here -- so
+21
# the interactive prompt is the normal case, not the exotic one.
+22
#
+23
# A sweep fixes today's sites and cannot see tomorrow's. Every bare site this
+24
# repo ever had was written next to another one.
+25
#
+26
# THE POPULATION IS DERIVED FROM THE ARTIFACT: every `*.sh` in the repository,
+27
# found by `find`, not a list anybody maintains. The number of files scanned is
+28
# printed, because a scan that found zero files would otherwise look identical to
+29
# a clean one.
+30
#
+31
# WHAT THIS CANNOT SEE, stated rather than left for someone to discover:
+32
# invocation through a variable or an alias (`S=setsid; $S "$sigil" ...`). This
+33
# is a text scanner and that construct defeats it. It is not used anywhere in
+34
# this repo today; if it ever is, this gate will not notice, and that is a
+35
# genuine hole rather than a hypothetical one.
+36
#
+37
# It DOES see invocations inside quoted strings -- `eval "..."`, `sh -c '...'`,
+38
# and lines that merely happen to contain an apostrophe. An earlier version
+39
# skipped anything with odd quote parity to its left, which meant one apostrophe
+40
# anywhere earlier on the line disarmed the check for a real, executed call. A
+41
# prose mention that gets flagged costs a rewording; a missed call site costs a
+42
# false green, so the strictness runs that way deliberately.
+43
set -euo pipefail
+44
+45
repo_dir="$(cd "$(dirname "$0")/.." && pwd)"
+46
cd "$repo_dir"
+47
+48
command -v awk >/dev/null || { echo "awk is required by this gate" >&2; exit 1; }
+49
command -v find >/dev/null || { echo "find is required by this gate" >&2; exit 1; }
+50
+51
work=$(mktemp -d)
+52
trap 'rm -rf "$work"' EXIT INT TERM
+53
+54
# The detector. An occurrence is ACCEPTED only when `-w`/`--wait` appears among
+55
# the option tokens that immediately follow it, before the command word. `-f` /
+56
# `--fork` alone is NOT acceptance: `-f` FORCES the racy branch. Anything the
+57
# parser cannot account for is REPORTED rather than waved through -- a detector
+58
# that fails open is the thing it exists to catch.
+59
#
+60
# Comment lines are skipped, and only comment lines: a `#`-led line cannot
+61
# execute, and this gate's own prose has to spell the bad pattern out.
+62
#
+63
# `\`-continuations are JOINED first, so an invocation whose options sit on the
+64
# next physical line is read as one logical line rather than reported as bare.
+65
cat >"$work/detect.awk" <<'AWK'
+66
{
+67
line = $0
+68
while (line ~ /\\$/ && (getline nxt) > 0) {
+69
sub(/\\$/, "", line)
+70
sub(/^[[:space:]]+/, " ", nxt)
+71
line = line nxt
+72
}
+73
stripped = line
+74
sub(/^[ \t]*/, "", stripped)
+75
if (substr(stripped, 1, 1) == "#") next
+76
rest = line
+77
while (match(rest, /(^|[^-A-Za-z0-9_])setsid[ \t]+/)) {
+78
rest = substr(rest, RSTART + RLENGTH)
+79
waits = 0
+80
tail = rest
+81
# Consume the leading option run. The trailing `[ \t]*$` alternative is what
+82
# lets an option that ends the line count -- without it, `... -w` as the last
+83
# token on a line was reported as bare.
+84
while (match(tail, /^-[^ \t]+([ \t]+|$)/)) {
+85
opt = substr(tail, RSTART, RLENGTH)
+86
sub(/[ \t]+$/, "", opt)
+87
if (opt == "--wait") waits = 1
+88
else if (opt !~ /^--/ && opt ~ /w/) waits = 1
+89
tail = substr(tail, RLENGTH + 1)
+90
}
+91
if (!waits) { print FILENAME ":" FNR ":" line }
+92
}
+93
}
+94
AWK
+95
+96
scan() { # scan <dir> -- one record per bare occurrence
+97
find "$1" -name '.git' -prune -o -type f -name '*.sh' -print \
+98
| sort \
+99
| while IFS= read -r file; do awk -f "$work/detect.awk" "$file"; done
+100
}
+101
+102
# --------------------------------------------------------------- CONTROLS
+103
# Planted into a throwaway directory, NEVER into the working tree, so there is
+104
# no restore step that has to be reached.
+105
#
+106
# THE FIXTURES ARE ASSEMBLED, NEVER SPELLED. This gate scans its own repository,
+107
# and a fixture written out literally here would be found by that scan and
+108
# reported as a real call site -- a check matching its own prose. Building the
+109
# lines through `$s` means the offending text does not appear anywhere in this
+110
# file outside a comment.
+111
ctl="$work/controls"
+112
mkdir -p "$ctl"
+113
s=setsid
+114
{
+115
printf '%s "$sigil" test suite.sgl >"$log" 2>&1\n' "$s"
+116
printf 'if ! %s "$sigil" eval -f probe.sgl >"$log"; then echo no; fi\n' "$s"
+117
printf 'out=$(%s -f "$sigil" eval "(display 1)")\n' "$s"
+118
printf '%s --fork "$sigil" build\n' "$s"
+119
printf ' indented=$(%s "$sigil" --version)\n' "$s"
+120
printf 'echo "the parser%ss output"; %s "$sigil" test suite.sgl\n' "'" "$s"
+121
printf 'eval "%s $sigil test >log"\n' "$s"
+122
printf "sh -c '%s \$sigil test'\n" "$s"
+123
printf '%s "$sigil" -w test suite.sgl\n' "$s"
+124
} >"$ctl/bad.sh"
+125
{
+126
printf '%s -w "$sigil" test suite.sgl >"$log" 2>&1\n' "$s"
+127
printf '%s -f --wait "${verify[@]}" --example timeout\n' "$s"
+128
printf '%s --wait "$sigil" build\n' "$s"
+129
printf 'FOO=bar %s -w "$sigil" eval -f probe.sgl\n' "$s"
+130
printf '%s -fw "$sigil" eval -f probe.sgl\n' "$s"
+131
printf '%s -w\n' "$s"
+132
printf '%s \\\n -w "$sigil" test suite.sgl\n' "$s"
+133
printf '# %s "$sigil" test suite.sgl <- a comment, and comments do not run\n' "$s"
+134
} >"$ctl/good.sh"
+135
+136
# The fixtures must contain what they claim to, or every control below passes
+137
# over an empty file.
+138
bad_lines=9
+139
good_forms=8
+140
good_physical=9 # one form deliberately spans two physical lines
+141
test "$(grep -c . "$ctl/bad.sh")" = "$bad_lines" || { echo "bad.sh fixture is not $bad_lines lines" >&2; exit 1; }
+142
test "$(grep -c . "$ctl/good.sh")" = "$good_physical" || { echo "good.sh fixture is not $good_physical physical lines" >&2; exit 1; }
+143
+144
ctl_bare=$(scan "$ctl" || true)
+145
ctl_bad=$(printf '%s\n' "$ctl_bare" | grep -c '/bad\.sh:' || true)
+146
ctl_good=$(printf '%s\n' "$ctl_bare" | grep -c '/good\.sh:' || true)
+147
+148
# Nine planted bare forms: plain, `if !`, `-f` only, `--fork` only, indented in a
+149
# substitution, one hidden behind an apostrophe, one inside an `eval` string, one
+150
# inside an `sh -c` string, and one where `-w` sits AFTER the command word and so
+151
# is an argument to the command rather than an option to the wrapper.
+152
test "$ctl_bad" = "$bad_lines" || {
+153
echo "POSITIVE CONTROL FAILED: detector flagged $ctl_bad of $bad_lines planted bare invocations" >&2
+154
printf '%s\n' "$ctl_bare" | grep '/bad\.sh:' >&2 || true
+155
exit 1; }
+156
# Eight waiting or inert forms, including `-w` as the final token and `-w` on a
+157
# continuation line -- both of which an earlier version reported as bare.
+158
test "$ctl_good" = "0" || {
+159
echo "NEGATIVE CONTROL FAILED: detector flagged $ctl_good of $good_forms waiting/inert form(s) as bare" >&2
+160
printf '%s\n' "$ctl_bare" | grep '/good\.sh:' >&2; exit 1; }
+161
echo "detector armed: $bad_lines/$bad_lines planted bare invocations flagged (quoted, eval'd and post-command -w included), 0/$good_forms waiting or inert forms misflagged"
+162
+163
# --------------------------------------------------------------- POPULATION
+164
files=$(find . -name '.git' -prune -o -type f -name '*.sh' -print | wc -l)
+165
test "$files" -ge 1 || { echo "scanned 0 shell scripts; the population derivation is broken" >&2; exit 1; }
+166
occurrences=$(grep -rn --include='*.sh' 'setsid' . 2>/dev/null | wc -l)
+167
echo "population: $files shell script(s); $occurrences line(s) mention it"
+168
+169
bare=$(scan "." || true)
+170
if [ -n "$bare" ]; then
+171
echo "BARE INVOCATION(S) that do not reliably wait:" >&2
+172
printf '%s\n' "$bare" | sed 's/^/ /' >&2
+173
echo >&2
+174
echo "VERIFY THIS GATE'S OWN PRECONDITIONS BEFORE ACTING ON THIS MESSAGE:" >&2
+175
echo " - the detector's controls passed above, so it sees what it claims to see;" >&2
+176
echo " - the line quoted is the logical line, with continuations already joined;" >&2
+177
echo " - an occurrence inside a quoted string is reported deliberately, because" >&2
+178
echo " eval and sh -c strings are real invocations. If yours is prose, reword it." >&2
+179
echo "Then: pass -w so it waits and propagates the child's exit status, or drop the" >&2
+180
echo "wrapper entirely -- the sigil test runner does not need a new session." >&2
+181
echo "See folio topics/setsid-does-not-reliably-wait." >&2
+182
exit 1
+183
fi
+184
+185
echo "SETSID_WAITS_GREEN 0 bare invocation(s) across $files shell script(s), $occurrences line(s) mentioning it"
test/sabotage-sd-p1.shmodified
@@ -79,11 +79,26 @@ PY
79
test "$before_hash" != "$after_hash" || { echo "SABOTAGE DID NOT CHANGE $file" >&2; exit 1; }
80
}
81
+82
# The COLLECTED test count, measured from the leg runs below rather than
+83
# written in. The closing green check at the end of this driver needs a
+84
# denominator; a literal one goes stale the first time somebody adds a test,
+85
# and a stale denominator in a check is how a gate stops describing its
+86
# subject and becomes a second thing to keep in sync.
+87
collected_total=0
+88
89
# expect_suite_red <label> [<expected failing test name fragment>]
90
expect_suite_red() {
−84
local label="$1" expect="${2:-}" log="$work/$(echo "$label" | tr ' /' '__').log" status
+91
local label="$1" expect="${2:-}" status log
+92
# SPLIT DELIBERATELY. `local a="$1" b="$(... $a ...)"` expands EVERY
+93
# assignment before `local` runs, so `$a` is still UNBOUND at that point.
+94
# Under `set -u` this printed `label: unbound variable` and evaluated to
+95
# `$work/.log` for every leg, so all seven legs overwrote one another's
+96
# log and the per-leg evidence this driver is built around did not exist.
+97
# It is in the committed transcript seven times
+98
# (test/transcripts/sd-p1/sabotage.txt, reported at "line 84").
+99
log="$work/$(printf '%s' "$label" | tr ' /' '__').log"
100
set +e
−86
setsid "$sigil" -L src -L test test "$suite" >"$log" 2>&1
+101
setsid -w "$sigil" -L src -L test test "$suite" >"$log" 2>&1
102
status=$?
103
set -e
104
sed -e 's/\x1b\[[0-9;]*m//g' "$log" >"$log.clean"
@@ -93,6 +108,12 @@ expect_suite_red() {
108
rg -q '[0-9]+ passed, [0-9]+ failed' "$log.clean" || {
109
echo "$label: no runner summary -- the suite did not run; this is NOT a demonstrated red" >&2
110
tail -40 "$log.clean" >&2; exit 1; }
+111
# Record this run's collected count (passed + failed) for the closing check.
+112
local summary p f
+113
summary=$(rg -o '[0-9]+ passed, [0-9]+ failed' "$log.clean" | head -1)
+114
p=${summary%% passed*}
+115
f=${summary#*, }; f=${f%% failed*}
+116
if [ $((p + f)) -gt "$collected_total" ]; then collected_total=$((p + f)); fi
117
test "$status" -ne 0 || { echo "$label: SABOTAGE DID NOT TURN THE SUITE RED" >&2; exit 1; }
118
rg -o '[0-9]+ passed, [0-9]+ failed' "$log.clean" | head -1
119
rg -o '✗ .*' "$log.clean" | head -6 || true
@@ -193,7 +214,7 @@ echo "neutered -- replaced with the CURRENT behaviour, control marker kept --"
214
echo "while the gate's scoring and exit logic are left untouched. If arm 2"
215
echo "still passes, it was passing for free."
216
after_now="$work/after-now.txt"
−196
setsid "$sigil" -L src -L test eval -f test/sd-p1-classification-probe.sgl >"$after_now" 2>&1
+217
setsid -w "$sigil" -L src -L test eval -f test/sd-p1-classification-probe.sgl >"$after_now" 2>&1
218
# DERIVED, not hardcoded. This exact guard shipped with a stale "8 cases" and
219
# aborted leg 8 on a probe that had run all 9 correctly -- the fourth instance
220
# of this hazard in one session. It failed SAFE (it refused to forge with input
@@ -230,6 +251,115 @@ echo "green closes the loop: sabotage proves green->red, this proves red->green.
251
for f in "$parser" "$reader" "$suite" "$before"; do
252
printf ' %s %s\n' "$(hash_of "$f")" "$f"
253
done
−233
setsid "$sigil" -L src -L test test "$suite" 2>&1 | sed -e 's/\x1b\[[0-9;]*m//g' | tail -8
254
−235
echo "SD_P1_SABOTAGE_COMPLETE 8 legs"
+255
# THE CLOSING RUN ASSERTED NOTHING ABOUT ITS OUTPUT UNTIL 2026-08-22, AND IT IS
+256
# THE ONLY THING IN THIS DRIVER THAT PROVES RED->GREEN.
+257
#
+258
# It was `setsid "$sigil" ... | sed ... | tail -8`, with no marker check of any
+259
# kind, and `SD_P1_SABOTAGE_COMPLETE 8 legs` printed on the next line.
+260
#
+261
# WHAT WAS AND WAS NOT WRONG WITH IT. The first version of this comment blamed a
+262
# `setsid` race, a leader acted on that, and it was wrong -- so the correction
+263
# lives here rather than being quietly dropped.
+264
#
+265
# NOT wrong: the exit status. `set -o pipefail` is on, so a suite exiting
+266
# non-zero would have aborted the script. And a PIPELINE blocks even when
+267
# `setsid` takes its fork branch, because the shell waits for EOF on the pipe
+268
# and the grandchild holds the write end. Measured on the forced fork branch:
+269
# 3020 ms for the piped form against a 3-second child, versus 10 ms for the
+270
# same command redirected to a file. This line was never racy. The racy shape
+271
# is redirect-to-a-file, which is a different set of call sites.
+272
#
+273
# Wrong, and still disqualifying: NOTHING ABOUT THE CONTENT WAS CHECKED. A
+274
# runner that exited 0 having collected zero tests, or having printed nothing
+275
# at all, satisfied every condition this block imposed, and the completion
+276
# marker went out regardless. A step whose whole job is to prove red->green has
+277
# to read what the runner actually said.
+278
#
+279
# It now asserts the RUNNER'S OWN WORDS from a log on disk, requires the run to
+280
# exit 0, and -- because an assertion that has never been shown to fire cannot
+281
# certify anything -- is armed against BOTH a truncated log and a known-good
+282
# summary before the real run happens.
+283
closing_log="$work/closing-green.log"
+284
+285
# assert_suite_green <plain-log> <minimum collected count>
+286
# 0 = the log shows a complete green run; 1 = refuse, with the reason.
+287
assert_suite_green() {
+288
local log="$1" floor="$2" passed total
+289
rg -q 'All tests passed' "$log" || { echo " closing check: no 'All tests passed' banner" >&2; return 1; }
+290
passed=$(rg -o 'Tests:[[:space:]]+[0-9]+ passed' "$log" | head -1 | grep -oE '[0-9]+' | head -1)
+291
total=$(rg -o '\([0-9]+ total\)' "$log" | head -1 | grep -oE '[0-9]+' | head -1)
+292
test -n "$passed" -a -n "$total" || { echo " closing check: the runner printed no summary line" >&2; return 1; }
+293
# Self-consistency: every collected test must be a passing one. This needs no
+294
# literal and cannot go stale.
+295
test "$passed" = "$total" || { echo " closing check: $passed passed of $total collected" >&2; return 1; }
+296
# ...and a FLOOR, because "0 passed (0 total)" is self-consistent too. A bound
+297
# satisfied by a suite that collected nothing is not a bound.
+298
#
+299
# WHAT THIS FLOOR IS AND IS NOT. It is derived from the same suite it then
+300
# bounds -- the maximum collected count observed across the legs above -- so it
+301
# cannot go stale when a test is added, and it cannot become unsatisfiable
+302
# (sabotage reduces or preserves collection, never increases it). It also
+303
# cannot detect a suite that shrank: gut the file from 11 tests to 1 and every
+304
# leg still reddens on the survivor, the floor becomes 1, and this reports
+305
# "green over 1 collected tests". It catches total collapse to zero and nothing
+306
# weaker. `test/gate-sd-p1.sh` arm 1 is what pins the absolute count.
+307
test "$total" -ge "$floor" || {
+308
echo " closing check: collected $total, but the legs above collected $floor" >&2; return 1; }
+309
if rg -q '[1-9][0-9]* failed' "$log"; then
+310
echo " closing check: the runner reported failures" >&2; return 1; fi
+311
return 0
+312
}
+313
+314
test "$collected_total" -ge 1 || {
+315
echo "no leg recorded a collected test count; the closing check has no denominator" >&2; exit 1; }
+316
+317
# POSITIVE CONTROL 1 -- it must REFUSE the exact artifact the fork branch
+318
# produces. `Running tests...` and nothing else is the real 18-byte log from the
+319
# 2026-08-11 measurement.
+320
printf 'Running tests...\n' >"$closing_log.truncated"
+321
if assert_suite_green "$closing_log.truncated" "$collected_total" 2>/dev/null; then
+322
echo "CLOSING CHECK CANNOT FAIL: it accepted a truncated log" >&2; exit 1; fi
+323
# POSITIVE CONTROL 2 -- and it must ACCEPT a known-good summary, or control 1
+324
# passes for free: a check that refuses everything refuses truncation too.
+325
printf 'Tests: %s passed (%s total)\nAll tests passed\n' "$collected_total" "$collected_total" \
+326
>"$closing_log.control"
+327
assert_suite_green "$closing_log.control" "$collected_total" || {
+328
echo "CLOSING CHECK REFUSES A KNOWN-GOOD SUMMARY -- it is a wall, not a check" >&2; exit 1; }
+329
# POSITIVE CONTROL 3 -- the floor must bite. A self-consistent empty run is the
+330
# shape a stalled or mis-filtered runner produces, and it must not be accepted.
+331
printf 'Tests: 0 passed (0 total)\nAll tests passed\n' >"$closing_log.empty"
+332
if assert_suite_green "$closing_log.empty" "$collected_total" 2>/dev/null; then
+333
echo "CLOSING CHECK CANNOT FAIL: it accepted a run that collected 0 tests" >&2; exit 1; fi
+334
# POSITIVE CONTROL 4 -- the `passed == total` arm must bite on its own. Controls
+335
# 1-3 are all refused by the banner or the floor before this arm is reached, so
+336
# without this one the arm could be deleted and every control would still pass.
+337
printf 'Tests: %s passed (%s total)\nAll tests passed\n' \
+338
"$((collected_total - 1))" "$collected_total" >"$closing_log.short"
+339
if assert_suite_green "$closing_log.short" "$collected_total" 2>/dev/null; then
+340
echo "CLOSING CHECK CANNOT FAIL: it accepted a run where not every collected test passed" >&2
+341
exit 1; fi
+342
# POSITIVE CONTROL 5 -- and so must the `N failed` arm, which controls 1-4 also
+343
# leave untouched: a log can be self-consistent, banner-carrying, at or above the
+344
# floor, AND still report failures.
+345
printf 'Tests: %s passed (%s total)\nAll tests passed\n2 failed\n' \
+346
"$collected_total" "$collected_total" >"$closing_log.failing"
+347
if assert_suite_green "$closing_log.failing" "$collected_total" 2>/dev/null; then
+348
echo "CLOSING CHECK CANNOT FAIL: it accepted a log that reported failures" >&2
+349
exit 1; fi
+350
echo "closing check armed: refuses a truncated log, an empty run, a partial pass and a reported failure; accepts a green summary"
+351
+352
set +e
+353
setsid -w "$sigil" -L src -L test test "$suite" >"$closing_log" 2>&1
+354
closing_status=$?
+355
set -e
+356
sed -e 's/\x1b\[[0-9;]*m//g' "$closing_log" >"$closing_log.clean"
+357
tail -8 "$closing_log.clean"
+358
test "$closing_status" -eq 0 || {
+359
echo "CLOSING RUN EXITED $closing_status on RESTORED sources -- red->green is NOT demonstrated" >&2
+360
tail -40 "$closing_log.clean" >&2; exit 1; }
+361
assert_suite_green "$closing_log.clean" "$collected_total" || {
+362
echo "CLOSING RUN did not demonstrate red->green on restored sources" >&2
+363
tail -40 "$closing_log.clean" >&2; exit 1; }
+364
+365
echo "SD_P1_SABOTAGE_COMPLETE 8 legs, closing suite green over $collected_total collected tests"
test/sabotage-sd-p4.shmodified
@@ -62,7 +62,7 @@ PY
62
expect_suite_red() {
63
local label="$1" target="$2" log="$3"
64
set +e
−65
setsid "$sigil" -L src -L test test "$target" >"$log" 2>&1
+65
setsid -w "$sigil" -L src -L test test "$target" >"$log" 2>&1
66
local status=$?
67
set -e
68
echo "--- $label: exit=$status"
@@ -85,7 +85,7 @@ apply "$inspect" \
85
"expression: (if (and (not command?) (eq? head 'expr)) (begin (eval (syntax->datum (caddr node))) (caddr node)) #f)"
86
witness_log="$work/leg1-witness.log"
87
set +e
−88
setsid "$sigil" -L src -L test eval -f test/sd-p4-witness-probe.sgl >"$witness_log" 2>&1
+88
setsid -w "$sigil" -L src -L test eval -f test/sd-p4-witness-probe.sgl >"$witness_log" 2>&1
89
set -e
90
if rg -q 'SDP4-WITNESS-' "$witness_log"; then
91
echo "LEG 1 RED (correct): author markers appeared during passive inspection:"