Commitffcf7a54Recorded1 Sep 2026Repositorykiln

lang: act on the adversarial review -- five defects, two of them mine

Message

An adversarial reviewer in an isolated checkout at fbe4746 returned 12 confirmed findings. Five needed code; the rest are corrected comments, or reported and deliberately not fixed here.

TWO DEFECTS THIS BRANCH INTRODUCED, ONE ROOT.

allow-failure: excused a gate-born errored node, and I had just made that status reachable. Consequences, both measured: a group whose every node was born errored by an undecidable gate rolled up "passed" -- green badge, group-terminal? true, nothing ran; and predecessor-verdict declined to gate the dependents of the very node kiln refused to decide, so everything downstream was promoted to pending. Permissive in both directions at once, which is the shape this whole evaluator exists to remove.

Fixed at the source rather than in each consumer: allow-failure: is a promise about the JOB'S WORK failing, not about kiln being unable to work out whether the job should run, and an author cannot have meant it that way because they believe their clause is decidable. An undecidable gate now clears the flag in the stored plan form.

THE REST.

  • A node born TERMINAL was never stamped finished_at. db-create-node-run! writes a status column and nothing else, and db-update-run-status! is the only thing in the tree that stamps the timestamp, so writing the terminal status at creation walked past it. That falsified queue.sgl:57-62 by name and rendered a gated-off node on a different time basis from its cascade-skipped siblings.
  • A matrix dimension could shadow $ref/$sha/$trigger, and the two resolvers disagreed about precedence: the gate read the binding, the ${...} interpolation beside it read the run context. when: (branch? $ref) under matrix: #{ ref: ... } silently answered #f and skipped the job. Now refused at lint AND the evaluator's precedence is aligned with the interpolator's, because plans are stored and replayed and the linter holding is not a defence for a form already in the database.
  Scoped to the three names something actually resolves, not all of
  run-context-vars: refusing the whole list broke run-lang-checks.sgl's
  "dimensions named after kiln's own keys still expand", which names a
  dimension `steps` on purpose because over-refusal is a known risk on
  this surface. That control did its job.
  • compile-node-script's optional 4th argument was unvalidated: a wrong value raised "dict-ref: expected dict" from record internals if the node happened to have a gated step, and was accepted in silence if it did not; a 5th was ignored. Validated at the boundary now.
  • emit-step's refusal rendered a string head "shell", a keyword head shell: and the supported symbol shell identically, and a number head as the empty string. It writes the head now. Also removed an unreachable else arm that looked like a live backstop.

Comments corrected rather than code: derive-fork's soundness argument holds on the webhook path and NOT on /api/plan-runs, which reads trigger and trusted as independent caller-supplied fields (route is behind with-auth, so caller-integrity rather than anonymous); the gated? accumulator is an optimisation with no observable behaviour, which the reviewer proved by removing it and watching 206 checks stay green; advance-group!'s docstring claimed a skipped-yet-allow-failure promotion that predecessor-verdict has never implemented.

Reported, NOT fixed here: a gated-off node posts no forge commit status, re-opening the hole runner.sgl:626-633 documents -- it needs report-commit-status!, whose signature is being changed by other work (t-4af08d).

test/run-when-checks.sgl: 95 -> 119 checks. Twelve raises? arms had no message assertion and would have passed on any raise at all, including one from a typo in their own fixture; they now name a substring only the intended refusal produces. Two fork? arms did not test what their names claimed and were rewritten. Sabotage re-run at ten legs, every plant hash-verified, every sabotaged tree confirmed to still compile, every leg red.

Corrects an overstatement in fbe4746's message: jobs/sigil-release.kiln does not compile at that SHA for SIX reasons (worker image:, a packaged action head, secrets:, use:), not only the secrets: refusal, so the publish job was not reachable to run on every push. The ordering argument is unaffected -- when: being permissive is a defect however few files currently reach it, and any gated publish job would run ungated the moment secret injection lands.

Changed
 src/kiln/lang/interp.sgl              |  28 +++++++++++++++-
 src/kiln/lang/refs.sgl                |  24 ++++++++++++++
 src/kiln/lang/when-eval.sgl           |  38 ++++++++++++++++++---
 src/kiln/nodeexec.sgl                 |  84 ++++++++++++++++++++++++++++++++++++++++-------
 src/kiln/scheduler.sgl                |  48 +++++++++++++++++++++++----
 test-jobs/when-badpred-allowfail.kiln |  21 ++++++++++++
 test/run-when-checks.sgl              | 277 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------
 7 files changed, 462 insertions(+), 58 deletions(-)
Diff
src/kiln/lang/interp.sglmodified
@@ -21,7 +21,8 @@
21
(sigil io)
22
(kiln lang records)
23
(kiln lang errors)
−24
(kiln lang predicates))
+24
(kiln lang predicates)
+25
(kiln lang refs))
26
27
(export analyze-forms parse-pipeline
28
dict-form? dict-form-pairs kw-ref)
@@ -252,6 +253,31 @@
253
(diag-error! sink
254
(string-append what
255
" names beginning KILN_ are controller-reserved")
+256
line form))
+257
;; A binding may not SHADOW a resolved run-context variable.
+258
;;
+259
;; With a dimension called `ref`, `$ref` in a gate and `${ref}` in
+260
;; the command beside it go through DIFFERENT tables -- the
+261
;; `when:` evaluator's and nodeexec's interpolator -- and need not
+262
;; agree, so the two can denote different values inside a single
+263
;; step. The concrete damage is a silent skip:
+264
;; `when: (branch? $ref)` reading a matrix value rather than the
+265
;; run's ref, answering #f, and gating the job off with no
+266
;; diagnostic anywhere.
+267
;;
+268
;; Scoped to `resolved-run-context-vars`, NOT to all of
+269
;; `run-context-vars`. Most of that list names things kiln does
+270
;; not record, and a name nothing resolves cannot be shadowed.
+271
;; Refusing the whole list broke run-lang-checks.sgl's
+272
;; "dimensions named after kiln's own keys still expand", which
+273
;; names a dimension `steps` deliberately because over-refusal is
+274
;; a known risk on this surface. That control did its job.
+275
(when (memq (string->symbol name) resolved-run-context-vars)
+276
(diag-error! sink
+277
(string-append what " '" name "' shadows the run-context "
+278
"variable $" name ": a gate and a ${" name
+279
"} interpolation in the same step would then "
+280
"mean different things -- rename the binding")
281
line form))))
282
283
;; exclude:/include: -> an #[array] of matrix cells, each a #{dict}.
src/kiln/lang/refs.sglmodified
@@ -21,6 +21,7 @@
21
(export ref-symbol? ref-name ref-head
22
string-interpolations
23
run-context-vars
+24
resolved-run-context-vars
25
collect-refs collect-string-refs
26
ref-resolvable?)
27
@@ -29,10 +30,33 @@
30
;; The fixed run-context namespace available to every node. Matrix
31
;; dimension names and `needs`/`steps` output namespaces are added
32
;; per-node by the plan compiler.
+33
;;
+34
;; This list is what LINT ACCEPTS as a reference head. Most of it is
+35
;; aspirational: `$actor`, `$repo`, `$run-id` and the rest name things
+36
;; kiln does not record, so they pass ref-checking and resolve to
+37
;; nothing at either consumer.
38
(define run-context-vars
39
'(ref sha trigger branch tag event actor repo
40
run-id run-number workspace needs steps))
41
+42
;; The subset that something actually RESOLVES today -- `${ref}` and
+43
;; `${sha}` through nodeexec's interpolator, and $ref/$sha/$trigger
+44
;; through the `when:` evaluator.
+45
;;
+46
;; It exists because these are the only names a MATRIX DIMENSION can
+47
;; ambiguously shadow: with a dimension called `ref`, a gate reading
+48
;; `$ref` and a command reading `${ref}` go through different tables
+49
;; and need not agree. `check-binding-name` refuses exactly these.
+50
;;
+51
;; DELIBERATELY NOT the whole of `run-context-vars`, and the
+52
;; difference was not a judgement call -- refusing the full list
+53
;; broke `run-lang-checks.sgl`'s "dimensions named after kiln's own
+54
;; keys still expand", which names a dimension `steps` on purpose
+55
;; because over-refusal is a live risk in this area. A name nothing
+56
;; resolves cannot be shadowed. When a variable here gains a real
+57
;; value, move it into this list in the same change.
+58
(define resolved-run-context-vars '(ref sha trigger))
+59
60
;; Is `x` a $reference symbol?
61
(define (ref-symbol? x)
62
(and (symbol? x)
src/kiln/lang/when-eval.sglmodified
@@ -79,9 +79,25 @@
79
80
;;; Derive `fork?' from what a run row actually records.
81
;;;
−82
;;; Kiln stores no is-fork flag; it stores `trusted', which
−83
;;; (kiln trust) `classify-trust' computed FROM is-fork? for exactly
−84
;;; one trigger. So the derivation is exact where it is defined:
+82
;;; Kiln stores no is-fork flag; it stores `trusted'. On the WEBHOOK
+83
;;; path that value came from (kiln trust) `classify-trust', which
+84
;;; computed it FROM is-fork? for exactly one trigger, so the
+85
;;; inversion below recovers what was thrown away.
+86
;;;
+87
;;; THAT PROVENANCE IS NOT UNIVERSAL, and the limit is worth knowing
+88
;;; before relying on `(fork?)`. `plan-run-handler` (web.sgl:1624)
+89
;;; reads `trigger` and `trusted` as INDEPENDENT caller-supplied JSON
+90
;;; fields and hands both to `schedule-plan-group!`. A caller posting
+91
;;; {"trigger":"pull-request","trusted":true} to /api/plan-runs
+92
;;; produces a run on which `(fork?)` answers #f having been told
+93
;;; nothing whatever about forkness -- a fabricated answer in the one
+94
;;; place this module otherwise promises to raise. That route is
+95
;;; behind `with-auth` (web.sgl:262), so it is a caller-integrity
+96
;;; question and not an anonymous one, and closing it means making
+97
;;; the API derive trust rather than accept it. Until then, treat
+98
;;; `(fork?)` as trustworthy on webhook-scheduled runs only.
+99
;;;
+100
;;; Where it is defined, the derivation is exact:
101
;;;
102
;;; pull-request -> trusted? is literally (not is-fork?), invert it
103
;;; push/tag/manual -> #f; all three require write access to this
@@ -216,12 +232,24 @@
232
"before the work it gates, so dotted references ($needs.*, "
233
"$steps.*) have no value yet")
234
form))
−219
((binding-of head (when-context-bindings ctx))
−220
=> (lambda (cell) (cdr cell)))
+235
;; RUN CONTEXT FIRST, matrix bindings second, and the ORDER IS
+236
;; LOAD-BEARING: `compile-node-script` resolves `${name}` from
+237
;; `(append (run-context-bindings ref sha) (pf-bindings pf))`,
+238
;; which is run-context-first. When this cond checked bindings
+239
;; first, a job with a matrix dimension literally named `ref`
+240
;; made `$ref` and `${ref}` mean different things INSIDE ONE
+241
;; STEP -- the gate read the matrix value, the command read the
+242
;; run's ref. `when: (branch? $ref)` then silently answered #f
+243
;; and skipped the job. Lint now refuses such a dimension
+244
;; outright, but a plan_form written before that rule still
+245
;; replays through here, so the two resolvers agree by
+246
;; construction rather than by the linter holding.
247
((eq? head 'ref) (need-fact (when-context-ref ctx) "$ref" form))
248
((eq? head 'sha) (need-fact (when-context-sha ctx) "$sha" form))
249
((eq? head 'trigger)
250
(name-of (need-fact (when-context-trigger ctx) "$trigger" form)))
+251
((binding-of head (when-context-bindings ctx))
+252
=> (lambda (cell) (cdr cell)))
253
(else
254
(when-error
255
(string-append
src/kiln/nodeexec.sglmodified
@@ -47,12 +47,32 @@
47
;; it exactly (strings, symbols, booleans, nested lists), and it is
48
;; deliberately NOT keyword-tagged so the reader treats every step
49
;; form as opaque data.
−50
(define (build-node-plan-form node)
+50
;;
+51
;; `allow-failure-applies?` (optional, default #t) exists for ONE
+52
;; caller: a node the scheduler is about to create already-terminal
+53
;; because its `when:` clause could not be decided.
+54
;;
+55
;; `allow-failure:` is a promise the author makes about the JOB'S
+56
;; WORK -- "this target is flaky, do not sink the pipeline on it". It
+57
;; is not a promise about kiln being unable to work out whether the
+58
;; job should run at all, and the author cannot have meant it that
+59
;; way, because they believe their clause is decidable. Letting it
+60
;; excuse an undecidable gate produced two live defects at once:
+61
;; `update-group-aggregate!` suppressed `bad?` and rolled a group in
+62
;; which NOTHING RAN up to "passed", and `predecessor-verdict`
+63
;; declined to gate the dependents of the very node kiln refused to
+64
;; decide. Both are the silent-permissive shape this whole evaluator
+65
;; exists to remove, so the flag is cleared at the source rather than
+66
;; special-cased in each consumer.
+67
(define (build-node-plan-form node . opts)
68
(let ((datum
69
(list
70
(cons "id" (plan-node-id node))
71
(cons "job" (symbol->string (plan-node-job node)))
−55
(cons "allow-failure" (if (plan-node-allow-failure node) #t #f))
+72
(cons "allow-failure"
+73
(if (and (plan-node-allow-failure node)
+74
(if (pair? opts) (car opts) #t))
+75
#t #f))
76
(cons "bindings"
77
(map (lambda (p) (cons (car p) (cdr p)))
78
(plan-node-bindings node)))
@@ -157,10 +177,7 @@
177
;; and never a silent run.
178
(define (compile-node-script pf ref sha . opts)
179
(let ((ctx (append (run-context-bindings ref sha) (pf-bindings pf)))
−160
(wctx (if (pair? opts)
−161
(car opts)
−162
(when-context ref: ref sha: sha
−163
bindings: (pf-bindings pf))))
+180
(wctx (resolve-when-context opts ref sha pf))
181
(out (open-output-string)))
182
(define (emit line) (write-string line out) (write-char #\newline out))
183
(emit "#!/bin/sh")
@@ -170,6 +187,36 @@
187
(for-each (lambda (step) (emit-step step ctx emit wctx)) (pf-steps pf))
188
(get-output-string out)))
189
+190
;; Validate the optional 4th argument AT THE BOUNDARY, before any
+191
;; step is looked at.
+192
;;
+193
;; Deferring it meant a wrong value failed in three different ways
+194
;; depending on the node: with a gated step, an incomprehensible
+195
;; "dict-ref: expected dict" from record internals naming no clause;
+196
;; with no gated step, NOTHING AT ALL, so a caller passing rubbish
+197
;; was told it was fine right up until someone added a `when:`; and a
+198
;; 5th argument was ignored outright. A predicate machine whose whole
+199
;; contract is "refuse rather than guess" cannot accept a context it
+200
;; has not looked at.
+201
(define (resolve-when-context opts ref sha pf)
+202
(cond
+203
((null? opts)
+204
;; No context: decide what ref/sha/bindings can decide, and let
+205
;; trigger- and trust-reading predicates RAISE. Never default.
+206
(when-context ref: ref sha: sha bindings: (pf-bindings pf)))
+207
((not (null? (cdr opts)))
+208
(error (string-append
+209
"compile-node-script takes at most one when-context after "
+210
"(pf ref sha); got " (number->string (length opts))
+211
" extra argument(s)")))
+212
((not (when-context? (car opts)))
+213
(error (string-append
+214
"compile-node-script's 4th argument must be a when-context "
+215
"(build one with run-when-context or node-when-context); got: "
+216
(call-with-output-string
+217
(lambda (p) (write (car opts) p))))))
+218
(else (car opts))))
+219
220
;;; The `when:` evaluation context for a node run, in the shape the
221
;;; runner has its data: a parsed plan form plus the run row's
222
;;; coordinates. The context itself is built by `run-when-context',
@@ -189,19 +236,32 @@
236
;; form, and letting the gate swallow it would make the refusal
237
;; depend on a predicate's value -- i.e. hide it exactly when a
238
;; false gate is also present.
+239
;; `write` the head, do not `symbol->string*` it. That coercion
+240
;; renders the string head "shell" and the keyword head shell:
+241
;; identically to the supported symbol `shell`, and a number head
+242
;; as the empty string -- so the refusal named a head that looks
+243
;; supported, on the message an operator has to act on. Harmless
+244
;; while this arm was last; it is now the first thing every
+245
;; stored form hits.
246
((not (memq (car step) '(shell restore gather artifact)))
247
(error (string-append "refusing unsupported stored plan step: "
−194
(symbol->string* (car step)))))
+248
(call-with-output-string
+249
(lambda (p) (write (car step) p))))))
250
;; when: FALSE -- emit nothing at all. Not a comment, not a
251
;; no-op section marker: a gated-off step must leave no trace in
252
;; the program, so "did not run" cannot be confused with "ran and
253
;; did nothing" by anyone reading the log.
254
((not (step-gate-open? step wctx)) #f)
−200
((eq? (car step) 'shell) (emit-shell step ctx emit))
−201
((eq? (car step) 'restore) (emit-restore step ctx emit))
−202
((eq? (car step) 'gather) (emit-gather step ctx emit))
−203
((eq? (car step) 'artifact) (emit-artifact step ctx emit))
−204
(else #f)))
+255
;; The memq arm above raises for every head outside this set, so
+256
;; these four are exhaustive and there is no `else`. An `else`
+257
;; here would be unreachable, and unreachable code that LOOKS
+258
;; like a backstop is worse than none: it invites the next reader
+259
;; to believe an unsupported head can reach it and quietly do
+260
;; nothing, which is precisely what the memq arm exists to stop.
+261
((eq? (car step) 'shell) (emit-shell step ctx emit))
+262
((eq? (car step) 'restore) (emit-restore step ctx emit))
+263
((eq? (car step) 'gather) (emit-gather step ctx emit))
+264
((eq? (car step) 'artifact) (emit-artifact step ctx emit))))
265
266
;; #t when the step has no `when:`, or its clause decides true.
267
;; Raises (via the evaluator) when the clause cannot be decided.
src/kiln/scheduler.sglmodified
@@ -177,8 +177,19 @@
177
;; was born terminal sits 'waiting' with nothing left to
178
;; run that would ever call advance-group!. One pass now
179
;; (it is a fixpoint, and idempotent) settles the cascade
−180
;; and the aggregate. Skipped only when a gate actually
−181
;; fired, so an ungated group's scheduling is unchanged.
+180
;; and the aggregate.
+181
;;
+182
;; `gated?` IS AN OPTIMISATION, NOT A CORRECTNESS GUARD,
+183
;; and saying so is the point of this paragraph. An
+184
;; adversarial reviewer replaced it with an unconditional
+185
;; call and all 206 out-of-CI checks stayed green, which
+186
;; is the honest result: db-create-group! already inserts
+187
;; status 'pending', and a group with any pending node
+188
;; aggregates to "pending", so on an ungated group the
+189
;; extra pass writes back what is already there. What it
+190
;; buys is the query, not the answer. Do not add a check
+191
;; that pretends to cover this branch -- there is no
+192
;; behaviour to observe.
193
(when gated? (advance-group! db group-id))
194
(cons group-id (reverse ids)))
195
(let* ((node (car ns))
@@ -196,10 +207,29 @@
207
(else "pending")))
208
(number (db-next-run-number db job-name))
209
(bindings-str (serialize-datum (plan-node-bindings node)))
−199
(node-form (build-node-plan-form node))
+210
;; An undecidable gate clears `allow-failure` in the
+211
;; stored form -- see build-node-plan-form. Without
+212
;; that, a gated job carrying `allow-failure: #t`
+213
;; both rolled its group up GREEN with nothing run
+214
;; and let its dependents proceed.
+215
(node-form (build-node-plan-form node (not (pair? gate))))
216
(id (db-create-node-run! db group-id node-id job-name number
217
ref sha status trusted? trigger-type
218
bindings-str node-form message)))
+219
;; Stamp finished_at on a node born TERMINAL.
+220
;;
+221
;; db-create-node-run! writes a status column and nothing
+222
;; else; db-update-run-status! is the only thing in the
+223
;; tree that stamps finished_at, and writing the terminal
+224
;; status at creation walks straight past it. The result
+225
;; was a `skipped` run that queue.sgl:57-62 documents as
+226
;; carrying finished_at and did not, rendering with a
+227
;; different time basis from its cascade-skipped siblings
+228
;; two lines down the same page. Re-asserting the status
+229
;; through the normal path is a no-op on the status
+230
;; column and stamps the timestamp.
+231
(when (not (eq? gate 'open))
+232
(db-update-run-status! db id status))
233
;; An undecidable gate must not look like an intended
234
;; skip, so it gets a status of its own AND the reason
235
;; written where an operator reads it.
@@ -256,10 +286,16 @@
286
287
;;; Advance a group after one of its node runs reaches a terminal
288
;;; status. For each still-'waiting' node, inspect its predecessors:
−259
;;; - all satisfying (passed, or failed-but-allow-failure, or a
−260
;;; skipped-yet-allow-failure) -> promote to 'pending'
−261
;;; - any hard-failed / skipped predecessor -> mark 'skipped'
+289
;;; - all satisfying (passed, or failed/errored-but-allow-failure)
+290
;;; -> promote to 'pending'
+291
;;; - any hard-failed / skipped / cancelled predecessor -> 'skipped'
292
;;; - otherwise (a predecessor still non-terminal) -> stay 'waiting'
+293
;;;
+294
;;; `skipped` gates REGARDLESS of allow-failure, and always has --
+295
;;; this docstring used to claim a "skipped-yet-allow-failure"
+296
;;; promotion that `predecessor-verdict` has never implemented. The
+297
;;; wrong sentence mattered little while skipped predecessors were
+298
;;; rare; `when:` gating makes this the common path.
299
;;; Then recompute + persist the group's aggregate status. Idempotent:
300
;;; every transition is guarded on the current status, so concurrent
301
;;; advances converge.
test-jobs/when-badpred-allowfail.kilnadded
@@ -0,0 +1,21 @@
+1
;;; when-badpred-allowfail.kiln -- an undecidable gate on a job that ALSO
+2
;;; carries allow-failure: #t, plus a dependent of it.
+3
;;;
+4
;;; This is the shape that turned a group in which nothing ran GREEN.
+5
;;; `allow-failure:` is a promise about the job's WORK failing; it must
+6
;;; not excuse kiln being unable to decide whether the job should run at
+7
;;; all. If it does, `bad?` is suppressed in the group rollup AND
+8
;;; `predecessor-verdict` declines to gate the dependents of the very
+9
;;; node kiln refused to decide -- permissive in both directions at once.
+10
+11
(pipeline when-badpred-allowfail
+12
on: #[ (push) ]
+13
+14
(job ebad
+15
when: (equal? $actor "somebody")
+16
allow-failure: #t
+17
steps: #[ (shell "echo EBAD-RAN" name: ebadstep) ])
+18
+19
(job edep
+20
needs: #[ ebad ]
+21
steps: #[ (shell "echo EDEP-RAN" name: edepstep) ]))
test/run-when-checks.sglmodified
@@ -86,6 +86,17 @@
86
(thunk)
87
#f))
88
+89
;; #t only if the thunk raises AND the message says the expected thing.
+90
;;
+91
;; `raises?` alone is a weak assertion and an adversarial review said so:
+92
;; it passes on ANY raise, including one from a typo in the arm's own
+93
;; fixture, an unbound variable, or an evaluator that has been broken
+94
;; into raising on everything. Every fail-closed arm below therefore
+95
;; names a substring only the intended refusal produces.
+96
(define (raises-with? thunk expected)
+97
(let ((m (raise-message thunk)))
+98
(and m (contains? m expected))))
+99
100
;; Contexts. `full` is what a real push carries.
101
(define full (run-when-context '() "refs/heads/main" "abc123" 'push #t))
102
(define tagctx (run-when-context '() "refs/tags/v1.0.0" "abc123" 'tag #t))
@@ -215,33 +226,60 @@
226
(lambda () (evaluate-when '(changed? "src/") full)))))
227
(and m (contains? m "does not collect"))))
228
−218
(check "(trusted?) with no trust known raises"
−219
(raises? (lambda () (evaluate-when '(trusted?) bare))))
−220
(check "(fork?) with no trigger known raises"
−221
(raises? (lambda () (evaluate-when '(fork?) bare))))
−222
(check "(manual?) with no trigger known raises"
−223
(raises? (lambda () (evaluate-when '(manual?) bare))))
−224
(check "(fork?) on a CHAINED run raises rather than inventing #f"
−225
(raises? (lambda ()
−226
(evaluate-when '(fork?)
−227
(run-when-context '() "refs/heads/main" "s" 'chained #t)))))
−228
(check "tag? on a non-string operand raises"
−229
(raises? (lambda () (evaluate-when '(tag? 7) full))))
−230
(check "an unknown predicate head raises (stored-form backstop)"
−231
(raises? (lambda () (evaluate-when '(frobnicate? $ref) full))))
−232
(check "a non-form clause raises"
−233
(raises? (lambda () (evaluate-when 'notaform full))))
+229
(check "(trusted?) with no trust known raises, saying so"
+230
(raises-with? (lambda () (evaluate-when '(trusted?) bare))
+231
"(trusted?) cannot be decided"))
+232
(check "(fork?) with neither trust nor trigger known raises, saying so"
+233
(raises-with? (lambda () (evaluate-when '(fork?) bare))
+234
"(fork?) cannot be decided"))
+235
(check "(manual?) with no trigger known raises, naming the trigger"
+236
(raises-with? (lambda () (evaluate-when '(manual?) bare))
+237
"no trigger type"))
+238
+239
;; `(fork?)` on a chained run and `(fork?)` on an empty context raise the
+240
;; SAME message, so a raise alone cannot show that derive-fork was
+241
;; consulted rather than the record default surviving. Assert the derived
+242
;; field itself, then the raise.
+243
(check "a CHAINED run's context carries fork: 'unknown (derive-fork ran)"
+244
(eq? 'unknown
+245
(when-context-fork
+246
(run-when-context '() "refs/heads/main" "s" 'chained #t))))
+247
(check "...and (fork?) on it raises rather than inventing #f"
+248
(raises-with? (lambda ()
+249
(evaluate-when '(fork?)
+250
(run-when-context '() "refs/heads/main" "s" 'chained #t)))
+251
"(fork?) cannot be decided"))
+252
;; Positive control for the pair above: on a trigger derive-fork CAN
+253
;; decide, the same field is a real boolean and (fork?) answers.
+254
(check "positive control: a push context carries fork: #f, not 'unknown"
+255
(eq? #f (when-context-fork full)))
+256
+257
(check "tag? on a non-string operand raises, naming what it wanted"
+258
(raises-with? (lambda () (evaluate-when '(tag? 7) full))
+259
"needs a ref string"))
+260
(check "an unknown predicate head raises (stored-form backstop), naming it"
+261
(raises-with? (lambda () (evaluate-when '(frobnicate? $ref) full))
+262
"unknown predicate 'frobnicate?'"))
+263
(check "a non-form clause raises, saying it is not a predicate form"
+264
(raises-with? (lambda () (evaluate-when 'notaform full))
+265
"not a predicate form"))
266
267
;; NO SHORT-CIRCUIT, and this is the arm that proves it. If `and` stopped
268
;; at its first false conjunct, the undecidable second one would never be
269
;; reached and the whole clause would come back a quiet #f -- putting the
270
;; caller straight back into the defect, through the connective.
+271
;;
+272
;; The message assertion is load-bearing here: it pins the raise to the
+273
;; SECOND operand ($actor), so the arm cannot pass on a raise coming from
+274
;; the first one or from the connective itself.
275
(check "and does NOT short-circuit past an undecidable conjunct"
−240
(raises? (lambda ()
−241
(evaluate-when '(and (tag? $ref) (equal? $actor "x")) full))))
+276
(raises-with? (lambda ()
+277
(evaluate-when '(and (tag? $ref) (equal? $actor "x")) full))
+278
"$actor"))
279
(check "or does NOT short-circuit past an undecidable disjunct"
−243
(raises? (lambda ()
−244
(evaluate-when '(or (branch? $ref) (equal? $actor "x")) full))))
+280
(raises-with? (lambda ()
+281
(evaluate-when '(or (branch? $ref) (equal? $actor "x")) full))
+282
"$actor"))
283
284
;; Positive control for the two arms above: the SAME clause shapes, with
285
;; a decidable second operand, must return a plain boolean. Without this
@@ -307,17 +345,67 @@
345
;; BEFORE any gate -- an unsupported head must not become invisible just
346
;; because it happens to sit behind a false predicate.
347
(check "an unsupported stored step still refuses (no gate)"
−310
(raises? (lambda ()
−311
(compile-node-script
−312
(list (cons "bindings" '()) (cons "steps" '((sc/package-ci))))
−313
"refs/heads/main" "abc123"))))
+348
(raises-with? (lambda ()
+349
(compile-node-script
+350
(list (cons "bindings" '())
+351
(cons "steps" '((sc/package-ci))))
+352
"refs/heads/main" "abc123"))
+353
"refusing unsupported stored plan step: sc/package-ci"))
354
(check "an unsupported stored step refuses even when gated FALSE"
−315
(raises? (lambda ()
−316
(compile-node-script
+355
(raises-with? (lambda ()
+356
(compile-node-script
+357
(list (cons "bindings" '())
+358
(cons "steps"
+359
'((sc/package-ci when: (tag? $ref)))))
+360
"refs/heads/main" "abc123"))
+361
"refusing unsupported stored plan step: sc/package-ci"))
+362
;; The refusal must NAME the head it saw, distinguishably. `symbol->string*`
+363
;; rendered a string head "shell", a keyword head shell: and the supported
+364
;; symbol `shell` identically, and a number head as nothing at all -- so
+365
;; the message an operator acts on named a head that looks supported.
+366
(check "a STRING head is refused and quoted as a string, not as `shell`"
+367
(raises-with? (lambda ()
+368
(compile-node-script
+369
(list (cons "bindings" '())
+370
(cons "steps" '(("shell" "echo x"))))
+371
"refs/heads/main" "abc123"))
+372
"\"shell\""))
+373
(check "a NUMBER head is refused and named, not rendered as empty"
+374
(raises-with? (lambda ()
+375
(compile-node-script
+376
(list (cons "bindings" '())
+377
(cons "steps" '((7 "echo x"))))
+378
"refs/heads/main" "abc123"))
+379
"step: 7"))
+380
;; Positive control for the four arms above: the same call shape with a
+381
;; SUPPORTED head compiles rather than refusing, so these are not passing
+382
;; against a compile-node-script that refuses everything.
+383
(check "positive control: a supported head in the same shape compiles"
+384
(contains? (compile-node-script
385
(list (cons "bindings" '())
−318
(cons "steps"
−319
'((sc/package-ci when: (tag? $ref)))))
−320
"refs/heads/main" "abc123"))))
+386
(cons "steps" '((shell "echo SUPPORTED" name: s))))
+387
"refs/heads/main" "abc123")
+388
"echo SUPPORTED"))
+389
+390
;; The optional 4th argument is validated at the boundary. A caller that
+391
;; passes the wrong thing used to get either an incomprehensible error
+392
;; from record internals or, if the node had no gated step, NOTHING.
+393
(check "a non-when-context 4th argument is refused, whatever the steps are"
+394
(raises-with? (lambda ()
+395
(compile-node-script
+396
(list (cons "bindings" '())
+397
(cons "steps" '((shell "echo x"))))
+398
"refs/heads/main" "abc123" 42))
+399
"must be a when-context"))
+400
(check "a 5th argument is refused rather than ignored"
+401
(raises-with? (lambda ()
+402
(compile-node-script
+403
(list (cons "bindings" '())
+404
(cons "steps" '((shell "echo x"))))
+405
"refs/heads/main" "abc123"
+406
(run-when-context '() "r" "s" 'push #t)
+407
'extra))
+408
"at most one when-context"))
409
410
;; ============================================================
411
;; 4. JOB-level gating, and BOTH directions of it
@@ -424,6 +512,87 @@
512
(logs (db-get-logs edb (assoc-ref 'id r))))
513
(and (pair? logs)
514
(contains? (assoc-ref 'line (car logs)) "$actor"))))
+515
;; The advice doubts the gate: this arm catches a defect in the evaluator
+516
;; as well as an undecidable clause, and the operator cannot tell which.
+517
(check "...and the message does not pin the blame on the pipeline file"
+518
(let* ((r (db-group-node-run edb egid "jobbad"))
+519
(logs (db-get-logs edb (assoc-ref 'id r))))
+520
(and (pair? logs)
+521
(contains? (assoc-ref 'line (car logs)) "kiln bug"))))
+522
+523
;; A node born TERMINAL is stamped finished_at like every other terminal
+524
;; run. db-create-node-run! writes only a status column, and
+525
;; db-update-run-status! is the sole thing in the tree that stamps the
+526
;; timestamp -- writing the terminal status at creation walked past it,
+527
;; leaving a `skipped` run that queue.sgl:57-62 documents as carrying
+528
;; finished_at and did not.
+529
(check "a gate-born skipped node is stamped finished_at"
+530
(let ((r (db-group-node-run bdb bgid "nevertag")))
+531
(and (assoc-ref 'finished_at r) #t)))
+532
(check "a gate-born errored node is stamped finished_at"
+533
(let ((r (db-group-node-run edb egid "jobbad")))
+534
(and (assoc-ref 'finished_at r) #t)))
+535
;; Positive control: the cascade-skipped node reached its status through
+536
;; db-update-run-status! and was always stamped, so this pair is not
+537
;; passing because every run happens to carry the column.
+538
(check "positive control: a cascade-skipped node is stamped too"
+539
(let ((r (db-group-node-run bdb bgid "afterskip")))
+540
(and (assoc-ref 'finished_at r) #t)))
+541
(check "positive control: a node that has NOT finished carries no stamp"
+542
(let ((r (db-group-node-run bdb bgid "always")))
+543
(not (assoc-ref 'finished_at r))))
+544
+545
;; ---- allow-failure must NOT excuse an undecidable gate ----
+546
;;
+547
;; `allow-failure:` is a promise about the JOB'S WORK failing, not about
+548
;; kiln being unable to decide whether the job should run. Letting it
+549
;; excuse a gate-born error produced two defects at once: a group in
+550
;; which NOTHING RAN rolled up "passed", and the dependents of the very
+551
;; node kiln refused to decide were promoted to pending.
+552
+553
(display "\nallow-failure does not excuse an undecidable gate\n")
+554
+555
(define fdb (fresh-db "/tmp/when-check-allowfail.db"))
+556
(define fplan (compile-first-plan "test-jobs/when-badpred-allowfail.kiln"))
+557
(define fgid (car (schedule-plan-group! fdb fplan "local/test/test"
+558
"refs/heads/main" "abc123" 'push #t)))
+559
+560
(check "an allow-failure job with an undecidable gate is still errored"
+561
(equal? "errored" (node-status fdb fgid "ebad")))
+562
(check "its dependent is SKIPPED, not promoted to pending"
+563
(equal? "skipped" (node-status fdb fgid "edep")))
+564
(check "the stored plan_form has allow-failure cleared for that node"
+565
(not (pf-allow-failure
+566
(parse-node-plan-form
+567
(assoc-ref 'plan_form (db-group-node-run fdb fgid "ebad"))))))
+568
(check "a group in which NOTHING ran does not roll up 'passed'"
+569
(not (equal? "passed" (group-status fdb fgid))))
+570
(check "...it rolls up 'failed', because kiln could not decide"
+571
(equal? "failed" (group-status fdb fgid)))
+572
;; Positive control: allow-failure still does its actual job -- excusing
+573
;; a node whose WORK failed. Without this the change above would be
+574
;; indistinguishable from having broken allow-failure outright.
+575
;; Fail ONLY the allow-failure node; its dependent runs and passes. The
+576
;; group must still be green, because allow-failure is doing exactly what
+577
;; it is for. (Failing every node instead would correctly produce
+578
;; "failed" and prove nothing -- `build` carries no allow-failure.)
+579
(check "positive control: allow-failure still excuses a node that RAN and failed"
+580
(let* ((db (fresh-db "/tmp/when-check-allowfail-pos.db"))
+581
(p (compile-first-plan "test-jobs/k2b-allowfail.kiln"))
+582
(gid (car (schedule-plan-group! db p "local/test/test"
+583
"refs/heads/main" "abc123" 'push #t)))
+584
(boot (db-group-node-run db gid "bootstrap")))
+585
(db-update-run-status! db (assoc-ref 'id boot) "failed")
+586
(advance-group! db gid)
+587
(and
+588
;; allow-failure did not gate the dependent...
+589
(equal? "pending" (node-status db gid "build"))
+590
(begin
+591
(db-update-run-status!
+592
db (assoc-ref 'id (db-group-node-run db gid "build")) "passed")
+593
(advance-group! db gid)
+594
;; ...and it did not sink the group either.
+595
(equal? "passed" (group-status db gid))))))
596
;; The group is still "pending" here because `stepbad' has not run -- an
597
;; errored node does not make a group terminal on its own. Drive the rest
598
;; of the group to a terminal state and THEN read the rollup.
@@ -508,11 +677,12 @@
677
(has-diag?
678
(sink-of "(pipeline p (job a when: (changed? \"src/\") steps: #[ (shell \"x\") ]))")
679
"$ref"))
−511
;; Positive control: lint is not refusing every when: clause.
−512
(check "positive control: a supported predicate still lints clean"
−513
(not (has-diag?
−514
(sink-of "(pipeline p (job a when: (and (tag? $ref) (trusted?)) steps: #[ (shell \"x\") ]))")
−515
"when:")))
+680
;; Positive control: lint is not refusing every when: clause. Assert the
+681
;; sink is EMPTY rather than that its text lacks "when:" -- the weaker
+682
;; form would also pass on some unrelated diagnostic.
+683
(check "positive control: a supported predicate lints with NO diagnostics"
+684
(not (diags-have-errors?
+685
(sink-of "(pipeline p (job a when: (and (tag? $ref) (trusted?)) steps: #[ (shell \"x\") ]))"))))
686
(check "an unknown predicate is still refused as unknown, not unimplemented"
687
(has-diag?
688
(sink-of "(pipeline p (job a when: (supported? $ref) steps: #[ (shell \"x\") ]))")
@@ -540,6 +710,45 @@
710
(sink-of "(pipeline p (job a when: (and (tag? $ref)) steps: #[ (shell \"x\") ]))")
711
"at least one predicate")))
712
+713
;; ---- a matrix dimension may not SHADOW a run-context variable ----
+714
;;
+715
;; With a dimension named `ref`, `$ref` in a gate and `${ref}` in the
+716
;; command beside it resolved through DIFFERENT tables and could name
+717
;; different values inside one step. `when: (branch? $ref)` then read the
+718
;; matrix value, answered #f, and gated the job off silently.
+719
+720
(check "a matrix dimension named `ref` is refused"
+721
(has-diag?
+722
(sink-of "(pipeline p (job a matrix: #{ ref: #[ \"x\" ] } steps: #[ (shell \"y\") ]))")
+723
"shadows the run-context variable $ref"))
+724
(check "a matrix dimension named `sha` is refused"
+725
(has-diag?
+726
(sink-of "(pipeline p (job a matrix: #{ sha: #[ \"x\" ] } steps: #[ (shell \"y\") ]))")
+727
"shadows the run-context variable $sha"))
+728
(check "a matrix dimension named `trigger` is refused"
+729
(has-diag?
+730
(sink-of "(pipeline p (job a matrix: #{ trigger: #[ \"x\" ] } steps: #[ (shell \"y\") ]))")
+731
"shadows the run-context variable $trigger"))
+732
;; The same guard covers include:/exclude: cell keys, or the restriction
+733
;; is evadable by moving the name into a cell -- the t-79e4ab lesson.
+734
(check "an include: cell key named `ref` is refused too"
+735
(has-diag?
+736
(sink-of "(pipeline p (job a matrix: #{ t: #[ \"x\" ] } include: #[ #{ ref: \"z\" } ] steps: #[ (shell \"y\") ]))")
+737
"shadows the run-context variable $ref"))
+738
;; Positive control: an ordinary dimension name is still fine.
+739
(check "positive control: a dimension named `target` lints clean"
+740
(not (has-diag?
+741
(sink-of "(pipeline p (job a matrix: #{ target: #[ \"x\" ] } steps: #[ (shell \"y\") ]))")
+742
"shadows")))
+743
+744
;; And the evaluator agrees with the interpolator on precedence even for
+745
;; a stored plan_form written before that lint rule existed -- plans are
+746
;; stored and replayed, so the linter holding is not a defence.
+747
(check "run context wins over a shadowing binding in the evaluator"
+748
(evaluate-when '(branch? $ref)
+749
(run-when-context '((ref . "SHADOWVAL"))
+750
"refs/heads/main" "abc123" 'push #t)))
+751
752
;; ---- summary ----
753
(display (string-append "\n" (number->string passes) " passed, "
754
(number->string fails) " failed\n"))