Add (press feeds): RSS 2.0 and JSON Feed 1.1 for matched content
A pipeline stage, with-feeds, that writes a feed of a site's dated content as two synthetic assets with their own producers (the shape generate-docs-css uses, so attach-asset-producers keeps them and execute-changed rebuilds them on every incremental build): RSS 2.0 with the Atom self link, dc:creator and RFC 822 dates, and JSON Feed 1.1 with authors, RFC 3339 dates and the summary: front matter when a post has one. The entries are the matched items that carry a date:, newest first, at most limit: (20); each has its absolute URL (the site's base-url: plus its canonical path, /index.html dropped) and its rendered body with the site's transforms applied, so a Markdown link resolves to its output path, and a leading top-level heading left out (it is the title). The RFC 822 weekday is Sakamoto's calendar arithmetic, not a timestamp: the stdlib's time-weekday reads the system zone, which moves a midnight date a day west of Greenwich. Re-exported from (press).
The pieces (feed-items, feed-item-url, feed-item-html, rss-string, json-feed-string, rfc-822-date, rfc-3339-date) are exported for tests and custom feeds; test/test-feeds.sgl covers the dates, the selection and order, the URLs, the transforms, both documents' shape and escaping, the stage's assets and the files they write (13 tests; the gate's legacy count moves from 45 to 58). README, docs/publish.md and the CHANGELOG describe the stage. Proven end to end on a scratch site through a dev-redirects.sgl pointing at this checkout.
CHANGELOG.md | 6 ++++
README.md | 29 ++++++++++++++++
docs/publish.md | 35 +++++++++++++++++++
src/press.sgl | 4 +++
src/press/feeds.sgl | 316 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
test/gate-d4b.sh | 6 ++--
test/test-feeds.sgl | 150 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
7 files changed, 543 insertions(+), 3 deletions(-)CHANGELOG.mdmodified
The format is based on [Keep a Changelog](https://keepachangelog.com/),and this project adheres to [Semantic Versioning](https://semver.org/).## [0.1.6] - 2026-09-22### Added- Added `(press feeds)` and its `with-feeds` stage: RSS 2.0 (with the Atom self link) and JSON Feed 1.1 for matched content, newest first by `date:`, absolute URLs from `base-url:`, rendered bodies with the site's transforms applied, RFC 822 and RFC 3339 dates.## [0.6.0] - 2026-02-22### AddedREADME.mdmodified
| `attach-asset-producers` | Set up asset copying || `rewrite-links` | Resolve internal links || `syntax-highlight` | Apply syntax highlighting to code blocks || `with-feeds` | Write RSS 2.0 and JSON Feed 1.1 for matched content |## Feeds`with-feeds` writes an RSS 2.0 feed and a JSON Feed 1.1 for the content itmatches, newest first by the `date:` front matter, with absolute URLs builtfrom the site's `base-url:` and each entry's rendered body:```scheme(import (press) (press generate))(run-publish (site title: "My Blog" base-url: "https://example.com" content: "posts/**/*.md" pipeline: (lambda (stages) (append stages (list (with-feeds match: (matches-path? "/posts/") title: "My Blog" description: "Notes on what I build" rss: "/posts/feed.xml" json: "/posts/feed.json" author: "Me"))))))```Pass `rss: #f` or `json: #f` to write one format only; `limit:` caps theentries (20 by default). A post's `summary:` front matter becomes the JSONFeed entry's summary.## API documentationdocs/publish.mdmodified
| `(press stages)` | Filtering, navigation, indexing || `(press render)` | Content/asset producers, themes || `(press generate)` | Collection pages, SXML pages || `(press feeds)` | RSS 2.0 and JSON Feed 1.1 || `(press transform)`| Syntax highlighting, link resolution || `(press dev-server)`| Development server with live reload |(run-publish docs-site)```### Blog with Feeds`with-feeds` adds an RSS 2.0 feed and a JSON Feed 1.1 as synthetic assets.The entries are the matched items that carry a `date:`, newest first;each has the item's absolute URL (the site's `base-url:` plus itscanonical path) and its rendered body with the site's transforms applied(a leading top-level heading, the title, is left out of the body).```scheme(import (press) (press generate))(define blog (site title: "Blog" base-url: "https://example.com" content: "posts/**/*.md" pipeline: (lambda (stages) (append stages (list (with-feeds match: (matches-path? "/posts/") description: "New posts" rss: "/posts/feed.xml" json: "/posts/feed.json" author: "Me" limit: 20))))))(run-publish blog)````title:` defaults to the site's title and `base-url:` to the site's;`rss: #f` or `json: #f` leaves that format out; a post's `summary:`front matter becomes the JSON Feed entry's summary. `(press feeds)` alsoexports the pieces (`feed-items`, `rss-string`, `json-feed-string`,`rfc-822-date`, `rfc-3339-date`) for a custom feed.### Documentation Site with Syntax Highlighting```schemesrc/press.sglmodified
(press stages) (press render) (press generate) (press feeds) (press transform) (press dev-server) (press docs)) collection-page sxml-page ;; Feeds (re-export from feeds) with-feeds ;; Theme support (re-export) default-theme sxml->htmlsrc/press/feeds.sgladded
;;; (press feeds) - RSS and JSON Feed generation;;;;;; A pipeline stage that writes a feed of a site's dated content: RSS 2.0;;; (with the Atom self link) and JSON Feed 1.1, either or both. The;;; matched items are sorted newest first by their `date:` front matter,;;; and each entry carries the item's absolute URL and its rendered body,;;; with the site's transforms applied so internal links resolve.;;;;;; ```scheme;;; (with-feeds match: (matches-path? "/posts/");;; title: "My Blog";;; description: "Notes on things I build";;; rss: "/posts/feed.xml";;; json: "/posts/feed.json");;; ```;;;;;; The absolute URLs come from the site's `base-url:` (or the stage's own;;; `base-url:` keyword). The feeds are synthetic assets with their own;;; producers, so they are written with the rest of the output and rebuilt;;; on every incremental build.;;;;;; The pure builders `rss-string` and `json-feed-string` take a config, a;;; state and a list of items and answer the feed text, so a feed can be;;; tested, or written some other way, without the stage.(define-library (press feeds) (import (sigil core) (sigil list) (sigil math) (sigil string) (sigil io) (sigil fs) (sigil path) (sigil time) (sigil sxml) (sigil json) (press core) (press scan) (press model)) (export ;; The pipeline stage with-feeds ;; The pieces, for tests and custom feeds feed-items feed-item-url feed-item-html rss-string json-feed-string rfc-822-date rfc-3339-date) (begin ;; ============================================================ ;; The Stage ;; ============================================================ ;;; Create a pipeline stage that adds an RSS feed and/or a JSON Feed. ;;; ;;; Keywords: ;;; - `match:` a predicate on items (see `has-tag?`, `matches-path?` in ;;; (press generate)); every item when omitted ;;; - `title:` the feed's title (the site's title when omitted) ;;; - `description:` one line about the feed ;;; - `base-url:` the site's absolute URL, e.g. "https://example.com" ;;; (the config's `base-url:` when omitted) ;;; - `rss:` the RSS output path, e.g. "/feed.xml" (#f for none) ;;; - `json:` the JSON Feed output path, e.g. "/feed.json" (#f for none) ;;; - `author:` the author's name, carried by every entry ;;; - `language:` the RSS channel language (default "en") ;;; - `limit:` at most this many entries, newest first (default 20) (define (with-feeds (keys: (match #f) (title #f) (description "") (base-url #f) (rss "/feed.xml") (json "/feed.json") (author #f) (language "en") (limit 20))) (lambda (config state) (let* ((output-dir (dict-ref config output-dir: "public")) (options #{title: title description: description base-url: base-url author: author language: language rss-path: rss json-path: json}) (asset (lambda (path mime-type make-text) #{source-path: #f output-path: path mime-type: mime-type producer: (lambda () (let* ((items (feed-items state match limit)) (full-path (string-append output-dir path))) (ensure-directory (path-dirname full-path)) (write-file-string full-path (make-text config state items options)) path))})) (assets (append (if rss (list (asset rss "application/rss+xml" rss-string)) '()) (if json (list (asset json "application/feed+json" json-feed-string)) '())))) (add-assets state assets)))) ;; ============================================================ ;; Selecting Entries ;; ============================================================ ;;; The items a feed lists: those the predicate accepts (all when #f) ;;; that carry a `date:`, newest first, at most `limit` of them. (define (feed-items state match limit) (let* ((items (state-items state)) (matching (filter (lambda (item) (and (or (not match) (match item)) (item-date item))) items)) (sorted (list-sort (lambda (a b) (string>? (item-date a) (item-date b))) matching))) (take-at-most sorted limit))) (define (take-at-most items n) (if (or (null? items) (and n (<= n 0))) '() (cons (car items) (take-at-most (cdr items) (and n (- n 1)))))) ;; The item's date as the string its front matter gave, or #f. (define (item-date item) (let ((date (dict-ref (item-metadata item) date: #f))) (and date (format "~a" date)))) ;; ============================================================ ;; URLs and Bodies ;; ============================================================ ;;; The item's absolute URL: the base URL and the item's canonical path ;;; (an `index.html` output is its directory). (define (feed-item-url base-url item) (string-append (trim-slash base-url) (output-path->url (item-output-path item)))) (define (trim-slash url) (if (string-ends-with? url "/") (substring url 0 (- (string-length url) 1)) url)) (define (output-path->url path) (if (string-ends-with? path "/index.html") (substring path 0 (- (string-length path) 10)) path)) ;;; The item's body as HTML, the config's transforms applied (so a ;;; Markdown link to another page becomes its output path). A leading ;;; top-level heading is left out: it is the entry's title, which a ;;; reader shows on its own. (define (feed-item-html config state item) (let* ((raw (item-body item)) (all (if (and (struct? raw) (document? raw)) (document-children raw) raw)) (children (if (and (pair? all) (pair? (car all)) (eq? (caar all) 'h1)) (cdr all) all)) (transforms (dict-ref config transforms: '())) (transformed (map (lambda (node) (apply-transforms node transforms config state item)) children))) (apply string-append (map sxml->html transformed)))) ;; Every transform over every node of the tree, children first (the ;; shape (press render) uses for a page). (define (apply-transforms node transforms config state item) (fold-left (lambda (n transform) (walk n transform config state item)) node transforms)) (define (walk node transform config state item) (if (pair? node) (let* ((tag (car node)) (rest (cdr node)) (has-attrs? (and (pair? rest) (pair? (car rest)) (eq? (caar rest) '@))) (attrs (if has-attrs? (cdar rest) '())) (children (if has-attrs? (cdr rest) rest)) (walked (map (lambda (c) (walk c transform config state item)) children)) (rebuilt (if has-attrs? `(,tag (@ ,@attrs) ,@walked) `(,tag ,@walked)))) (transform config state item rebuilt)) node)) ;; ============================================================ ;; Dates ;; ============================================================ ;; A front-matter date ("2025-01-15", or with a time) as (y m d h mi s). (define (date-fields text) (parse-iso-date text)) (define WEEKDAYS (vector "Sun" "Mon" "Tue" "Wed" "Thu" "Fri" "Sat")) (define MONTHS (vector "Jan" "Feb" "Mar" "Apr" "May" "Jun" "Jul" "Aug" "Sep" "Oct" "Nov" "Dec")) (define (pad2 n) (if (< n 10) (string-append "0" (number->string n)) (number->string n))) ;; The day of the week (0 = Sunday) of a civil date, by Sakamoto's ;; method: arithmetic on the calendar, no timestamp and no time zone ;; (the stdlib's time-weekday reads a timestamp in the system's zone, ;; which would move a midnight date a day west of Greenwich). (define SAKAMOTO (vector 0 3 2 5 0 3 5 1 4 6 2 4)) (define (weekday y m d) (let ((y (if (< m 3) (- y 1) y))) (modulo (+ y (quotient y 4) (- (quotient y 100)) (quotient y 400) (vector-ref SAKAMOTO (- m 1)) d) 7))) ;;; The RFC 822 form RSS wants: "Wed, 15 Jan 2025 00:00:00 +0000". (define (rfc-822-date text) (let* ((f (date-fields text)) (y (list-ref f 0)) (m (list-ref f 1)) (d (list-ref f 2)) (h (list-ref f 3)) (mi (list-ref f 4)) (s (list-ref f 5)) (weekday (weekday y m d))) (string-append (vector-ref WEEKDAYS weekday) ", " (pad2 d) " " (vector-ref MONTHS (- m 1)) " " (number->string y) " " (pad2 h) ":" (pad2 mi) ":" (pad2 s) " +0000"))) ;;; The RFC 3339 form JSON Feed wants: "2025-01-15T00:00:00Z". (define (rfc-3339-date text) (format-iso-date (date-fields text))) ;; ============================================================ ;; RSS 2.0 ;; ============================================================ (define (option options key config-key config default) (or (dict-ref options key #f) (and config-key (dict-ref config config-key #f)) default)) ;;; The RSS 2.0 document for the items, as a string. (define (rss-string config state items options) (let* ((base-url (option options base-url: base-url: config "/")) (title (option options title: title: config "Feed")) (description (option options description: #f config "")) (language (option options language: #f config "en")) (author (dict-ref options author: #f)) (self-path (dict-ref options rss-path: #f)) (entry (lambda (item) (let ((url (feed-item-url base-url item))) `(item (title ,(item-title item)) (link ,url) (guid (@ (isPermaLink "true")) ,url) (pubDate ,(rfc-822-date (item-date item))) ,@(if author `((dc:creator ,author)) '()) (description ,(feed-item-html config state item)))))) (doc `(rss (@ (version "2.0") (xmlns:atom "http://www.w3.org/2005/Atom") (xmlns:dc "http://purl.org/dc/elements/1.1/")) (channel (title ,title) (link ,(string-append (trim-slash base-url) "/")) (description ,description) (language ,language) ,@(if self-path `((atom:link (@ (href ,(string-append (trim-slash base-url) self-path)) (rel "self") (type "application/rss+xml")))) '()) ,@(if (pair? items) `((lastBuildDate ,(rfc-822-date (item-date (car items))))) '()) ,@(map entry items))))) (string-append "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" (sxml->xml doc) "\n"))) ;; ============================================================ ;; JSON Feed 1.1 ;; ============================================================ ;;; The JSON Feed 1.1 document for the items, as a string. (define (json-feed-string config state items options) (let* ((base-url (option options base-url: base-url: config "/")) (title (option options title: title: config "Feed")) (description (option options description: #f config "")) (language (option options language: #f config "en")) (author (dict-ref options author: #f)) (self-path (dict-ref options json-path: #f)) (authors (if author (list #{name: author}) '())) (entry (lambda (item) (let* ((url (feed-item-url base-url item)) (summary (dict-ref (item-metadata item) summary: #f)) (base #{id: url url: url title: (item-title item) date_published: (rfc-3339-date (item-date item)) content_html: (feed-item-html config state item)})) (if summary (dict-set base summary: (format "~a" summary)) base)))) (feed (let ((head #{version: "https://jsonfeed.org/version/1.1" title: title home_page_url: (string-append (trim-slash base-url) "/") description: description language: language})) (let* ((with-self (if self-path (dict-set head feed_url: (string-append (trim-slash base-url) self-path)) head)) (with-authors (if (pair? authors) (dict-set with-self authors: authors) with-self))) (dict-set with-authors items: (map entry items)))))) (string-append (json-encode feed indent: #t) "\n")))))test/gate-d4b.shmodified
echo D4B_GATE_00_FRESH_OUTPUT_AND_CACHE"$sigil" "${legacy_load_paths[@]}" -L test test \ test/test-basic.sgl test/test-docs.sgl test/test-functional.sgl --report \ test/test-basic.sgl test/test-docs.sgl test/test-functional.sgl test/test-feeds.sgl --report \ | tee "$cache_dir/legacy.log"rg -q '45 passed' "$cache_dir/legacy.log"echo D4B_GATE_00B_LEGACY_PRESS_45_45rg -q '58 passed' "$cache_dir/legacy.log"echo D4B_GATE_00B_LEGACY_PRESS_58_58"$sigil" "${load_paths[@]}" -L test test test/test-sigil-doc.sgl --report \ | tee "$cache_dir/adapter.log"test/test-feeds.sgladded
;;; test-feeds.sgl - Tests for (press feeds);;;;;; The feed stage over an in-memory state: the entries it selects and;;; their order, the two documents it writes (RSS 2.0 and JSON Feed 1.1),;;; the dates in each format, the escaping, the absolute URLs, and the;;; transforms applied to a body.(import (sigil core) (sigil string) (sigil fs) (sigil io) (sigil json) (sigil test) (press core) (press scan) (press model) (press generate) (press feeds))(define (post title path date body . rest) (let ((summary (if (pair? rest) (car rest) #f))) #{source-path: (string-append "content" path ".md") output-path: (string-append path "/index.html") title: title metadata: (if summary #{date: date summary: summary} #{date: date}) format: 'markdown body-loader: (lambda () (document metadata: #{} children: body))}))(define config #{title: "Site" base-url: "https://example.com/" output-dir: "/tmp/press-feeds-test"})(define posts (list (post "Second & last" "/posts/second" "2025-02-10" '((p "Two " (b "bold") " & more"))) (post "First" "/posts/first" "2025-01-15" '((p "One") (p (a (@ (href "../about.md")) "about"))) "the first one") #{source-path: "content/about.md" output-path: "/about/index.html" title: "About" metadata: #{} format: 'markdown body-loader: (lambda () (document metadata: #{} children: '((p "no date, never listed"))))} (post "Elsewhere" "/notes/x" "2025-03-01" '((p "not a post")))))(define state (add-items (initial-state) posts))(define options #{title: "Posts" description: "the posts" base-url: #f author: "Ann" language: "en" rss-path: "/posts/feed.xml" json-path: "/posts/feed.json"})(test-group "feeds" (test-group "dates" (test "RFC 822 from a date, the weekday by the calendar" (assert-equal "Wed, 15 Jan 2025 00:00:00 +0000" (rfc-822-date "2025-01-15")) (assert-equal "Sun, 01 Mar 2026 00:00:00 +0000" (rfc-822-date "2026-03-01")) (assert-equal "Sat, 29 Feb 2020 13:05:09 +0000" (rfc-822-date "2020-02-29T13:05:09Z"))) (test "RFC 3339 from a date" (assert-equal "2025-01-15T00:00:00Z" (rfc-3339-date "2025-01-15")) (assert-equal "2020-02-29T13:05:09Z" (rfc-3339-date "2020-02-29T13:05:09Z")))) (test-group "entries" (test "the matched, dated items, newest first, at most the limit" (let ((items (feed-items state (matches-path? "/posts/") 20))) (assert-equal '("Second & last" "First") (map item-title items))) (let ((items (feed-items state #f 2))) (assert-equal '("Elsewhere" "Second & last") (map item-title items))) (assert-equal '() (feed-items state (matches-path? "/nowhere/") 20))) (test "the absolute URL drops index.html and doubles no slash" (assert-equal "https://example.com/posts/first/" (feed-item-url "https://example.com/" (cadr posts))) (assert-equal "https://example.com/posts/first/" (feed-item-url "https://example.com" (cadr posts)))) (test "the body is HTML with the config's transforms applied" (let* ((cfg (dict-set config transforms: (list (lambda (c s item node) (if (and (pair? node) (eq? (car node) 'a)) `(a (@ (href "/about/")) ,@(cddr node)) node))))) (html (feed-item-html cfg state (cadr posts)))) (assert-equal "<p>One</p><p><a href=\"/about/\">about</a></p>" html))) (test "a leading top-level heading (the title) is left out of the body" (let ((item (post "Titled" "/posts/t" "2025-04-01" '((h1 "Titled") (p "body") (h1 "Not first"))))) (assert-equal "<p>body</p><h1>Not first</h1>" (feed-item-html config state item))))) (test-group "rss" (let* ((items (feed-items state (matches-path? "/posts/") 20)) (text (rss-string config state items options))) (test "a 2.0 document with the channel, the self link and the entries" (assert-true (string-starts-with? text "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<rss version=\"2.0\"")) (assert-true (string-contains? text "<channel><title>Posts</title><link>https://example.com/</link><description>the posts</description><language>en</language>")) (assert-true (string-contains? text "<atom:link href=\"https://example.com/posts/feed.xml\" rel=\"self\" type=\"application/rss+xml\"")) (assert-true (string-contains? text "<lastBuildDate>Mon, 10 Feb 2025 00:00:00 +0000</lastBuildDate>")) (assert-true (string-contains? text "<item><title>Second & last</title><link>https://example.com/posts/second/</link><guid isPermaLink=\"true\">https://example.com/posts/second/</guid><pubDate>Mon, 10 Feb 2025 00:00:00 +0000</pubDate><dc:creator>Ann</dc:creator>")) (assert-true (string-contains? text "<item><title>First</title>"))) (test "the body is escaped text, not markup" (assert-true (string-contains? text "<description><p>Two <b>bold</b> &amp; more</p></description>")) (assert-false (string-contains? text "<description><p>"))) (test "the newest entry comes first" (assert-true (< (string-find text "<title>Second & last</title>") (string-find text "<title>First</title>")))))) (test-group "json-feed" (let* ((items (feed-items state (matches-path? "/posts/") 20)) (text (json-feed-string config state items options)) (feed (json-decode text))) (test "a 1.1 document with the head and the entries" (assert-equal "https://jsonfeed.org/version/1.1" (dict-ref feed version:)) (assert-equal "Posts" (dict-ref feed title:)) (assert-equal "https://example.com/" (dict-ref feed home_page_url:)) (assert-equal "https://example.com/posts/feed.json" (dict-ref feed feed_url:)) (assert-equal "the posts" (dict-ref feed description:)) (assert-equal "Ann" (dict-ref (car (array->list (dict-ref feed authors:))) name:)) (assert-equal 2 (length (array->list (dict-ref feed items:))))) (test "each entry: id and url, title, the RFC 3339 date, the HTML body, the summary when given" (let ((second (car (array->list (dict-ref feed items:)))) (first (cadr (array->list (dict-ref feed items:))))) (assert-equal "https://example.com/posts/second/" (dict-ref second id:)) (assert-equal "https://example.com/posts/second/" (dict-ref second url:)) (assert-equal "Second & last" (dict-ref second title:)) (assert-equal "2025-02-10T00:00:00Z" (dict-ref second date_published:)) (assert-equal "<p>Two <b>bold</b> & more</p>" (dict-ref second content_html:)) (assert-false (dict-ref second summary: #f)) (assert-equal "the first one" (dict-ref first summary:)))))) (test-group "stage" (test "with-feeds adds the two assets and their producers write the files" (let* ((dir "/tmp/press-feeds-test") (stage (with-feeds match: (matches-path? "/posts/") title: "Posts" description: "the posts" rss: "/posts/feed.xml" json: "/posts/feed.json" author: "Ann")) (result (stage config state)) (assets (state-assets result))) (assert-equal 2 (length assets)) (assert-equal '("/posts/feed.xml" "/posts/feed.json") (map (lambda (a) (dict-ref a output-path:)) assets)) (assert-equal '("application/rss+xml" "application/feed+json") (map (lambda (a) (dict-ref a mime-type:)) assets)) (for-each (lambda (asset) ((dict-ref asset producer:))) assets) (let ((rss (read-file-string (string-append dir "/posts/feed.xml"))) (json (json-decode (read-file-string (string-append dir "/posts/feed.json"))))) (assert-true (string-contains? rss "<rss version=\"2.0\"")) (assert-true (string-contains? rss "https://example.com/posts/second/")) (assert-equal 2 (length (array->list (dict-ref json items:)))) (assert-equal "https://example.com/posts/first/" (dict-ref (cadr (array->list (dict-ref json items:))) url:))))) (test "rss: #f or json: #f leaves that feed out; the site's title and base-url are the defaults" (let* ((stage (with-feeds json: #f)) (assets (state-assets (stage config state)))) (assert-equal '("/feed.xml") (map (lambda (a) (dict-ref a output-path:)) assets))) (let* ((text (rss-string config state (feed-items state #f 20) #{title: #f description: "" base-url: #f author: #f language: "en" rss-path: "/feed.xml" json-path: #f}))) (assert-true (string-contains? text "<title>Site</title><link>https://example.com/</link>")) (assert-true (string-contains? text "<item><title>Elsewhere</title>"))))))