Commit81ba8a6eRecorded21 Jul 2026Repositorysigil-wire

Use native bytevector-ieee-double accessors for the float path

Message

Swap the float encode/decode from the pure-arithmetic IEEE-754 bit derivation to sigil-stdlib's native bytevector-ieee-double-ref/set! (added in 0.17.18 alongside the bignum bitwise-correctness fix):

  • encode-float: bytevector-ieee-double-set! ... 'little; decode-float: bytevector-ieee-double-ref (cur-take! c 8) 0 'little. Deletes ~85 lines (double->bits/bits->double/pack-bits/POW2/WIRE-INF/NAN).
  • -0.0 now round-trips bit-exactly (the accessor preserves the sign bit), superseding the earlier collapse-to-+0.0 limitation.
  • NaN-box safety on decode: -ref canonicalizes any hostile bit pattern to a safe quiet-NaN flonum, so a malicious float body can never yield a type-confused value. Added decode tests (tag-zone + all-ones bits).
  • Varint path kept as pure arithmetic on purpose: correct on any sigil (incl. older/wasm builds), the bitwise fix notwithstanding.

Verified 126/126 against a dev sigil (0.17.18-dev+0bb453df) via dev-redirects.sgl. README + docs/wire.md updated.

Changed
 README.md          |  11 +++++++----
 docs/wire.md       |  15 ++++++++-------
 src/sigil/wire.sgl | 124 +++++++++++++++++++++++-----------------------------------------------------------------------------------------------------
 test/test-wire.sgl |  24 +++++++++++++++++++++++-
 4 files changed, 61 insertions(+), 113 deletions(-)
Diff
README.mdmodified
@@ -110,10 +110,13 @@ signed to unsigned so small negatives stay short: `n >= 0 → 2n`, `n < 0 →
110
111
### Floats
112
−113
Doubles are stored as their raw 64-bit IEEE-754 bit pattern, little-endian.
−114
`+inf`, `-inf` and `NaN` round-trip bit-exactly. Note that Sigil cannot
−115
distinguish `-0.0` from `+0.0` (they compare equal under every predicate and
−116
print identically), so a negative zero encodes as `+0.0`.
+113
Doubles are stored as their raw 64-bit IEEE-754 bit pattern, little-endian, via
+114
the native `bytevector-ieee-double-{ref,set!}` accessors. Every double — normal,
+115
subnormal, `±0.0`, `±inf`, `NaN` — round-trips **bit-exactly**, including the sign
+116
bit of `-0.0` (even though Sigil predicates cannot themselves distinguish `-0.0`
+117
from `+0.0`). On decode, the accessor canonicalizes any hostile bit pattern to a
+118
safe quiet-NaN flonum, so a malicious float body can never produce a
+119
type-confused value.
120
121
## Security
122
docs/wire.mdmodified
@@ -104,13 +104,14 @@ Eight bytes holding the raw IEEE-754 binary64 bit pattern, little-endian. Sign i
104
the top bit of the last (most significant) byte.
105
106
- `+inf` = `7FF0000000000000`, `-inf` = `FFF0000000000000`.
−107
- `NaN` is written canonically as `7FF8000000000000` (any incoming NaN
−108
normalizes to this on encode); on decode any payload with exponent field all-1
−109
and non-zero fraction is a NaN.
−110
- `-0.0` is not represented distinctly by the reference implementation (Sigil
−111
cannot distinguish it from `+0.0`); it encodes as `+0.0` = `0000000000000000`.
−112
A decoder that receives `8000000000000000` should still produce `-0.0` where
−113
the host supports it.
+107
- `NaN`: any payload with the exponent field all-ones and a non-zero fraction is
+108
a NaN. The reference implementation writes whatever bits the host double
+109
carries and, on decode, canonicalizes a bit pattern that would collide with the
+110
runtime's tagged-value space to a safe quiet NaN (see below).
+111
- `-0.0` = `8000000000000000` is represented distinctly and round-trips
+112
bit-exactly (the reference implementation uses the host's native IEEE accessor,
+113
which preserves the sign bit — even though the language's numeric predicates
+114
cannot distinguish `-0.0` from `+0.0`).
115
116
Example: `3.14 → 03 1F 85 EB 51 B8 1E 09 40` (bytes are `0x40091EB851EB851F` LE).
117
src/sigil/wire.sglmodified
@@ -150,12 +150,13 @@
150
151
;; ========== Varint (unsigned LEB128) ==========
152
−153
;; NOTE: the varint and float-bit paths use only arithmetic (quotient /
−154
;; remainder / * / +), never bitwise-and / bitwise-ior / arithmetic-shift.
−155
;; Sigil's bitwise/shift primitives mis-handle values straddling the
−156
;; fixnum<->bignum boundary (e.g. round-tripping 2^62 or 7^500 through
−157
;; shift+ior corrupts the value), whereas plain integer arithmetic is
−158
;; correct for arbitrary-precision integers. See the wire-format topic note.
+153
;; NOTE: the varint path uses only arithmetic (quotient / remainder / * / +),
+154
;; never bitwise-and / bitwise-ior / arithmetic-shift. Older Sigil bitwise/
+155
;; shift primitives mis-handle values straddling the fixnum<->bignum boundary
+156
;; (round-tripping 2^62 or 7^500 through shift+ior corrupted the value); that
+157
;; VM bug is fixed as of 0.17.18, but the arithmetic form is kept because it
+158
;; is correct on ANY sigil (incl. older and wasm builds) at no real cost.
+159
;; See the wire-format topic note.
160
161
;; Write a non-negative integer as unsigned LEB128 to a port.
162
(define (write-uvarint u port)
@@ -180,90 +181,16 @@
181
(quotient u 2)
182
(- (- (quotient u 2)) 1)))
183
−183
;; ========== Float <-> IEEE-754 bits ==========
+184
;; ========== Float <-> IEEE-754 bytes ==========
185
;;
−185
;; Sigil's numeric tower is exact integers (incl. bignums) and inexact
−186
;; doubles — there are NO exact rationals — so `(exact <float>)` errors on a
−187
;; non-integer. We therefore derive the 64-bit IEEE-754 representation with
−188
;; float arithmetic + exact-integer ops, never touching a rational. Every
−189
;; scaling factor stays within the representable double range (|exponent| <=
−190
;; 1022 per step) so no intermediate overflows to infinity.
−191
−192
(define POW2-52 (expt 2 52)) ; 4503599627370496
−193
(define POW2-63 (expt 2 63)) ; sign-bit place value
−194
−195
;; Sigil has no infinity/NaN literals, but doubles CAN be inf/NaN (e.g. from
−196
;; overflow). Build them once, by arithmetic, to reconstruct on decode.
−197
(define WIRE-POS-INF (expt 2.0 2000)) ; overflows to +inf
−198
(define WIRE-NEG-INF (- (expt 2.0 2000))) ; -inf
−199
(define WIRE-NAN (- WIRE-POS-INF WIRE-POS-INF)) ; inf - inf = NaN
−200
−201
;; Assemble sign/biased-exponent/fraction into a 64-bit unsigned integer.
−202
(define (pack-bits sign biased frac)
−203
(+ (* sign POW2-63) (* biased POW2-52) frac))
−204
−205
;; Encode a double as its unsigned 64-bit IEEE-754 bit pattern.
−206
;; NaN is checked BEFORE zero: Sigil's `(= nan 0.0)` returns #t, so a
−207
;; zero-first test would mis-encode NaN as +0.0.
−208
(define (double->bits x)
−209
(cond
−210
;; NaN: canonical quiet NaN (0x7FF8000000000000); fraction = 2^51.
−211
((nan? x) (pack-bits 0 2047 (quotient POW2-52 2)))
−212
;; +/- infinity.
−213
((not (finite? x))
−214
(if (< x 0.0) (pack-bits 1 2047 0) (pack-bits 0 2047 0)))
−215
;; Zero (positive and negative zero both encode as +0.0; Sigil does not
−216
;; distinguish them under `=`, `eqv?`, `equal?`, or `number->string`).
−217
((= x 0.0) 0)
−218
(else
−219
(let* ((neg (< x 0.0))
−220
(sign (if neg 1 0))
−221
(ax (abs x)))
−222
(if (< ax (expt 2.0 -1022))
−223
;; Subnormal: significand = round(ax * 2^1074), scaled in two
−224
;; representable steps (2^1074 itself overflows a double).
−225
(let ((frac (exact (round (* (* ax (expt 2.0 1022))
−226
(expt 2.0 52))))))
−227
(pack-bits sign 0 frac))
−228
;; Normal: find unbiased exponent e with 2^e <= ax < 2^(e+1).
−229
(let loop ((e (exact (floor (/ (log ax) (log 2.0))))))
−230
(cond
−231
((<= (expt 2.0 (+ e 1)) ax) (loop (+ e 1)))
−232
((> (expt 2.0 e) ax) (loop (- e 1)))
−233
(else
−234
;; ax/2^e is exactly a double in [1,2); *2^52 is an exact
−235
;; integer significand in [2^52, 2^53).
−236
(let* ((sig (exact (round (* (/ ax (expt 2.0 e))
−237
(expt 2.0 52)))))
−238
(frac (- sig POW2-52))
−239
(biased (+ e 1023)))
−240
(pack-bits sign biased frac))))))))))
−241
−242
;; Decode an unsigned 64-bit IEEE-754 bit pattern to a double.
−243
(define (bits->double bits)
−244
(let* ((sign (quotient bits POW2-63))
−245
(rest (remainder bits POW2-63))
−246
(biased (quotient rest POW2-52))
−247
(frac (remainder rest POW2-52)))
−248
(cond
−249
;; Inf / NaN carry their own sign; return directly.
−250
((= biased 2047)
−251
(cond
−252
((not (= frac 0)) WIRE-NAN)
−253
((= sign 1) WIRE-NEG-INF)
−254
(else WIRE-POS-INF)))
−255
(else
−256
(let ((mag (cond
−257
;; Zero / subnormal.
−258
((= biased 0)
−259
(if (= frac 0)
−260
0.0
−261
(* frac (expt 2.0 -1074))))
−262
;; Normal.
−263
(else
−264
(* (+ frac POW2-52)
−265
(expt 2.0 (- biased 1075)))))))
−266
(if (= sign 1) (- mag) mag))))))
+186
;; The wire float is the raw 8-byte IEEE-754 double, LITTLE-ENDIAN (fixed by
+187
;; the format, independent of host byte order). Sigil-stdlib's native
+188
;; `bytevector-ieee-double-{ref,set!}` do the exact conversion, so every
+189
;; double — normal, subnormal, +/-0.0, +/-inf, NaN — round-trips bit-exactly.
+190
;; Crucially for the trust boundary, `-ref` is NaN-box-safe: it canonicalizes
+191
;; any hostile bit pattern (including ones that would collide with the VM's
+192
;; immediate tag space) to a genuine quiet-NaN flonum, never a type-confused
+193
;; Value. It also bounds-checks the offset/length itself.
194
195
;; ========== Encoding ==========
196
@@ -346,12 +273,10 @@
273
(error "wire-encode: value is not representable on the wire" value))))
274
275
(define (encode-float value port)
−349
(let ((bits (double->bits value)))
−350
;; 8 bytes, little-endian (arithmetic, not bitwise).
−351
(let loop ((i 0))
−352
(when (< i 8)
−353
(write-u8 (remainder (quotient bits (expt 256 i)) 256) port)
−354
(loop (+ i 1))))))
+276
;; Raw 8-byte IEEE-754 double, little-endian (exact bits, incl. -0.0).
+277
(let ((bv (make-bytevector 8 0)))
+278
(bytevector-ieee-double-set! bv 0 value 'little)
+279
(write-bytevector bv port)))
280
281
(define (encode-list value port)
282
;; Walk once: verify the list is proper and count length, collecting the
@@ -530,12 +455,9 @@
455
(cur-take! c n)))
456
457
(define (decode-float c)
−533
(cur-need! c 8)
−534
(let loop ((i 0) (bits 0))
−535
(if (= i 8)
−536
(bits->double bits)
−537
(loop (+ i 1)
−538
(+ bits (* (cur-u8! c) (expt 256 i)))))))
+458
;; cur-take! bounds-checks the 8 bytes; bytevector-ieee-double-ref
+459
;; canonicalizes any hostile bit pattern to a safe flonum.
+460
(bytevector-ieee-double-ref (cur-take! c 8) 0 'little))
461
462
(define (decode-char c)
463
(let ((cp (read-uvarint c LEN-VARINT-MAX-BYTES)))
test/test-wire.sglmodified
@@ -77,7 +77,29 @@
77
(test "+inf stays infinite" (assert-true (infinite? (rt (expt 2.0 2000)))))
78
(test "-inf stays negative" (assert-true (< (rt (- (expt 2.0 2000))) 0.0)))
79
(test "NaN stays NaN"
−80
(assert-true (nan? (rt (- (expt 2.0 2000) (expt 2.0 2000)))))))
+80
(assert-true (nan? (rt (- (expt 2.0 2000) (expt 2.0 2000))))))
+81
;; The native IEEE accessor preserves the sign bit, so -0.0 now round-trips
+82
;; bit-exactly. equal? cannot distinguish +/-0.0, so compare the wire bytes:
+83
;; -0.0 and +0.0 must encode to DIFFERENT frames (sign bit set only for -0.0).
+84
(test "-0.0 round-trips (decodes to a zero)"
+85
(assert-equal 0.0 (rt (- 0.0))))
+86
(test "-0.0 preserves its sign bit on the wire (distinct from +0.0)"
+87
(assert-false (equal? (wire-encode (- 0.0)) (wire-encode 0.0)))))
+88
+89
(test-group "decode - hostile float bit patterns (NaN-box safety)"
+90
;; A float body may carry any 8 bytes. bytevector-ieee-double-ref canonicalizes
+91
;; bit patterns that would collide with the VM's immediate tag space to a safe
+92
;; quiet-NaN flonum — never a type-confused Value, crash, or OOB read.
+93
(test "tag-zone NaN 0x7FFC... reads as a safe NaN flonum"
+94
(let ((r (guard (e (#t 'clean-error))
+95
(wire-decode (frame #x03 0 0 0 0 0 0 #xFC #x7F)))))
+96
(assert-true (or (eq? r 'clean-error) (and (number? r) (nan? r))))))
+97
(test "all-ones float bits read as a safe NaN flonum"
+98
(let ((r (guard (e (#t 'clean-error))
+99
(wire-decode (frame #x03 #xFF #xFF #xFF #xFF #xFF #xFF #xFF #xFF)))))
+100
(assert-true (or (eq? r 'clean-error) (and (number? r) (nan? r))))))
+101
(test "float body truncated to 4 bytes errors cleanly"
+102
(assert-true (raises? (lambda () (wire-decode (frame #x03 0 0 0 0)))))))
103
104
(test-group "round-trip - strings"
105
(test "empty" (assert-equal "" (rt "")))