Cabalmail

Host your own email and enhance your privacy

View the Project on GitHub cabalmail/cabal-infra

RSS Reader — Implementation Plan

Context

This plan implements the RSS reader for Cabalmail 1.x based on the decisions recorded in rss-requirements.md. It is the companion build plan to that requirements pass; whenever this document says “per Dx,” it refers to a decision in that file. The RSS API is purely additive, which docs/compatibility.md classifies as a minor release, so the work ships as 1.x minor releases (this directory), each phase under its own release.

Progress

This table and the **Status:** line under each phase are updated in the same PR as the work, per the docs convention. The Apple sections (“Apple-side design”, “Per-feed cookie scoping › Apple clients”, and the phase 5 status) are the shipped record as of 1.18.1 and take dated errata; the Android phase and phases 7 to 10 are still forward-looking and are revised in place. The 2026-09-13 pass that split the two is summarized under “Revisions (2026-09-13)”.

Phase Work item Status
1 DynamoDB tables + supporting infra Shipped 1.12.2 (2026-09-09)
2 Scheduler + fetcher Lambdas Shipped 1.12.3 (2026-09-09); www fallback 1.15.1; image enclosures 1.16.0
3 Subscription + reader API Shipped 1.13.0 (2026-09-10); additive: state_since 1.17.0, default_remote_content 1.17.0, default_filter 1.18.0
4 OPML import/export (API) Shipped 1.14.0 (2026-09-10)
5 Apple clients (offline + FTS + cookie scoping) Shipped 1.14.0 (5b) / 1.15.0 (5c, 5d); UAT refinements 1.16.0 to 1.18.1, see “Post-5c UAT refinements”
6 Android client (offline + FTS + profile scoping) 6a to 6c merged to stage (2026-09-13); 6d (polish) in review (2026-09-13)
7 Image proxy + cache Not started
8 Push notification integration Not started
9 Credentialed feeds Not started
10 Adaptive cadence + health surface polish Health badge shipped 1.16.0 (2026-09-11); rest not started

Revisions (2026-09-09)

The plan was first written in May 2026 and revised in place on 2026-09-09 to reflect what shipped in the 0.11.x and 1.x releases and to fix design defects found on re-read. This is an in-flight plan, so the corrections are folded into the text rather than recorded as errata; this section is the summary of what moved and why.

Revisions (2026-09-13)

The operator’s acceptance testing of the Apple reader (1.16.0 through 1.18.1, 2026-09-10 to 2026-09-13) produced changes that the phase 5 design note did not anticipate. This pass records them so the Android phase is built against what Apple actually does, not against the 2026-09-10 design. Three kinds of edit were made:

Three decisions shape the architecture more than the rest:

Per open Q3 the fetcher is Lambda-based, not an ECS service (as a scheduler plus a queue-fed worker; see “Data flow”). Per open Q5, user-level customization (display preferences, ordering) is stored server-side and applied client-side, so the shared canonical-feed records hold no per-user data and the multi-tenancy boundary is enforced by storage layout, not by access logic.

The plan is structured as ten independently-shippable phases. Each phase ends in a state that can be deployed to prod without the next phase being present.

Goals

Non-goals (v1, all per the requirements doc)

Architecture overview

Component diagram

                          +----------------------------+
                          |  DynamoDB tables           |
                          |    cabal-rss-feed          |
                          |    cabal-rss-item  --------+--- stream (INSERT) --+
                          |    cabal-rss-subscription  |                      |
                          |    cabal-rss-folder        |                      |
                          |    cabal-rss-user-item-    |                      |
                          |      state                 |                      |
                          +-------------+--------------+                      |
                                        ^                                     v
   +-----------+   Scheduler   +--------+-------+   SQS    +----------+  +----+-----+
   | EventBridge| every 5 min  | rss_schedule   | -------> | rss_fetch|  | rss_     |
   | Scheduler  | -----------> | (claims due    | feed ids | (one feed|  | notify   |
   +-----------+               |  feeds)        |          |  per msg)|  | (fan-out |
                               +----------------+          +----+-----+  |  to      |
                                                                |        |  users)  |
                                                 HTTPS + CGET   |        +----+-----+
                                                 via NAT EIPs   v             |
                                                           +---------+        | cabal-push-
                                                           |  feeds  |        | queue (SQS)
                                                           |  (any)  |        v
                                                           +---------+  +-----------+
                                                                        |  push_    |
                                                                        |  dispatch |
                                                                        | (0.11.x)  |
                                                                        +-----+-----+
                                                                              |
                                                                          APNs / FCM
                                                                              v
                    +-----------------+                  +-----------------+
                    |  iOS / macOS    |                  |  Android        |
                    |  (CabalmailKit) |                  |  (kit)          |
                    |  reader UI      |                  |  reader UI      |
                    |  WKWebView per  |                  |  WebView profile|
                    |  feed (WKWDS    |                  |  per feed       |
                    |  per identifier)|                  |                 |
                    |  +-----------+  |                  |  +-----------+  |
                    |  | ItemCache |  |                  |  | Room +    |  |
                    |  | SQLite    |  |                  |  | FTS       |  |
                    |  | + FTS5    |  |                  |  | offline + |  |
                    |  | offline + |  |                  |  | per-feed  |  |
                    |  | per-feed  |  |                  |  | search    |  |
                    |  | search    |  |                  |  +-----------+  |
                    |  +-----------+  |                  +--------+--------+
                    +--------+--------+                           |
                             +--------------+---------------------+
                                            |
                                            v
                              +-------------+---------+     +-------------------+
                              |  API Gateway          | --> |  rss_* Lambdas    |
                              |  (Cognito auth, flat  |     |  (lambda/api/,    |
                              |   path_part per fn)   |     |   VPC-attached)   |
                              +-----------------------+     +---------+---------+
                                                                      |
                                                                      v
                                                            +---------+---------+
                                                            |  rss_image Lambda |
                                                            |  + S3 cache (7d)  |
                                                            |  -> presigned URL |
                                                            +-------------------+

Data flow

Fetch path. EventBridge Scheduler fires rss_schedule every 5 minutes. It queries cabal-rss-feed’s by_due GSI (PK = "active", SK <= now), claims each due feed with a conditional update that advances next_fetch_at (so a slow or duplicated tick cannot fetch the same feed twice), and enqueues one message per feed id on a new cabal-rss-fetch-queue SQS queue (with a dead-letter queue, like cabal-push-queue). rss_fetch consumes that queue one feed per invocation with a small reserved concurrency (start at 5) so outbound traffic stays polite and bounded. For each feed it issues a conditional GET against the publisher with the prior ETag/Last-Modified, parses the response, upserts items via the by_guid GSI lookup + PutItem, updates the feed’s health fields and observed_items_per_day, and sets a new next_fetch_at from the adaptive-cadence formula. A worker failure lands the message back on the queue (visibility timeout) and eventually in the DLQ, which is the operator’s signal that a feed is misbehaving in a way the health fields did not capture.

The split replaces the single “loop over every due feed” Lambda of the original plan. It removes the 15-minute Lambda ceiling as a failure mode, gives per-feed retries for free, mirrors the push_dispatch queue-consumer pattern the project already runs, and costs nothing extra at hobby scale.

Credentialed feeds run on the same worker in a per-user track: the cabal-rss-feed row has is_shared = false and owner_user set to the subscriber. The worker pulls credentials from SSM at /cabal/rss/credentials/<user>/<feed_id>.

Notification path. cabal-rss-item has a DynamoDB Stream (NEW_IMAGE); rss_notify is its Lambda trigger, filtered to INSERT events so item updates (re-publishes) never notify. For each new item it queries the sparse GSI subscription.by_feed_notify for subscribers with notifications on and enqueues one wake signal per (user, item) onto the existing cabal-push-queue with kind = "rss". This replaces the original pending_notification table, the TransactWriteItems coupling in the fetcher, the 60-second cron, and its TTL sweep: the stream is the durable pending queue, Lambda stream triggers have no read charge, and retries come from the stream retry policy. push_dispatch gains a kind = "rss" branch (the existing mail signal is {user, folder, uid, msg_id} with no kind; absence of the field means mail). See “Push notification integration” for what changes inside push_dispatch and the clients.

Read path. Clients call API Gateway endpoints. Following the existing gateway shape — one Lambda per flat path_part, JSON bodies, no path parameters — the endpoints are rss_* functions under lambda/api/ (e.g. /rss_subscribe, /rss_list_items, /rss_set_item_state), not a REST-style /rss/items/{id} tree. This keeps them on the existing build (build-api-one.sh), lint, CI filter (lambda/api/**), and Terraform (modules/app/modules/call) paths with no new plumbing. Lambdas execute DynamoDB Query/BatchGet calls directly. The endpoint that returns an item body rewrites <img src="..."> to image-proxy URLs (phase 7).

For folder-spanning item lists, the API runs N parallel Queries against the item table (one per feed in the folder), merge-sorts the streams by published_at in the Lambda, and overlays per-user state from user-item-state. See “Data model” for how unread and favorite filtering work.

Native sync path. When online, a client polls /rss_list_items with since=<fetched_at cursor> per subscribed feed to pull new items into its local cache. The cursor is the server’s fetched_at (ingest time), not published_at: publishers backdate and re-date items, and a cursor over published_at silently misses them. Foreground sync and background refresh handle the rest; the push path assists on Apple through an App Group handoff (not by the NSE writing the cache directly — see “Apple-side item cache”).

Shared vs. per-user data

Concept Shared / per-user
cabal-rss-feed Shared (public feeds) or per-user
  (credentialed feeds, is_shared = false)
cabal-rss-item Shared with the feed it belongs to
subscription Per-user (links user to feed_id)
folder Per-user
user-item-state Per-user (read, favorite, read_at)
Credentials Per-user SSM parameters; credentialed feeds
  bypass sharing
User settings Per-user rss_* keys in the existing
  cabal-user-preferences app map
Display preferences Per-user attrs on subscription, applied
  client-side (per open Q5)

Credentialed feeds get a per-user cabal-rss-feed row (same canonical URL, different owner_user). This trades schema purity for explicit isolation — there is no code path where a credentialed fetch lands content into a row that another user can read.

Data model (DynamoDB)

All tables follow terraform/infra/modules/table/main.tf: on-demand capacity, server-side encryption with the AWS-owned key (the standing #tfsec:ignore:aws-dynamodb-table-customer-key directive; there is no project KMS key for tables and the plan no longer introduces one), point-in-time recovery enabled, and deletion protection on every table that holds user data. The per-user key is the Cognito username (the cognito:username claim, as extracted by _shared/helper.py and used by every other per-user table), named user below. The schema:

cabal-rss-feed
  PK: feed_id (UUID v4 string)
  attrs: canonical_url, is_shared (bool), owner_user (null when shared),
         feed_type ('rss'|'atom'|'json'), display_name, description,
         site_url, next_fetch_at_iso, last_fetched_at_iso,
         last_etag, last_modified, last_status_code, last_error,
         consecutive_failure_count, cadence_minutes,
         observed_items_per_day, item_count, subscriber_count,
         created_at
  GSI by_canonical:  PK = canonical_url
                     SK = owner_user_or_NULL_SENTINEL
                     (dedup on subscribe; sentinel string for nulls
                      since DynamoDB GSIs reject null SK)
  GSI by_due (sparse): PK = due_shard ("active" while fetchable;
                            attribute removed when dead-lettered; the
                            whole row is deleted when subscriber_count
                            reaches 0)
                       SK = next_fetch_at_iso

cabal-rss-item
  PK: feed_id
  SK: published_at_iso#item_id  (sortable + unique; published_at is
                                 fixed at first sight — a re-publish
                                 updates attrs, never the SK)
  attrs: item_id (UUID), guid, title, author, url,
         summary_html, content_html | content_s3_key,
         published_at, updated_at, fetched_at
  GSI by_guid:    PK = feed_id, SK = guid
                  (upsert dedup lookup on fetch; guid falls back to
                   the item link, then to a hash of title+content,
                   when the feed omits one)
  GSI by_fetched: PK = feed_id, SK = fetched_at_iso#item_id
                  (the since-cursor for client sync)
  Stream: NEW_IMAGE (drives rss_notify)

cabal-rss-subscription
  PK: user
  SK: subscription_id (UUID)
  attrs: feed_id, folder_id, custom_title, ordering_mode,
         default_open_mode, default_styling,
         default_remote_content ('inherit'|'show'|'hide', 1.17.0),
         default_filter ('all'|'unread'|'favorite', default 'unread',
                         1.18.0),
         notifications_enabled, credentials_scheme (null | 'basic' |
         'url_key' | 'cookie'), read_watermark_iso,
         data_store_uuid, created_at
  GSI by_user_folder: PK = user
                      SK = folder_id#subscription_id
                      (list subscriptions in folder)
  GSI by_feed_notify (sparse): PK = feed_id
                               SK = user
                               (present only when
                                notifications_enabled = true)

cabal-rss-folder
  PK: user
  SK: folder_id (UUID)
  attrs: parent_folder_id (null = root), name, display_order,
         default_filter ('all'|'unread'|'favorite', default 'unread',
                         1.18.0)

cabal-rss-user-item-state
  PK: user#feed_id
  SK: published_at_iso#item_id    (matches item SK shape)
  attrs: item_id, is_read, is_favorite, read_at, updated_at,
         updated_key (updated_at_iso#item_id, set on every write,
                      1.17.0)
  GSI favorite_by_feed (sparse): PK = user#feed_id
                                 SK = published_at_iso#item_id
                                 (present when is_favorite = true)
  GSI by_updated:  PK = user#feed_id
                   SK = updated_key
                   (the state_since cursor for cross-device state
                    sync, 1.17.0; see "Sync loop" step 3's erratum)

Wire-level additions that ride on these rows, all additive under docs/compatibility.md: items and state rows carry is_read_explicit (whether is_read is the user’s own mark rather than the watermark rule), /rss_list_items reports has_more on both its list and its sync forms, and /rss_list_items has a third form keyed on state_since that returns state rows instead of items. docs/rss.md is the reference for all three.

Five tables, down from eight. What went, and why:

A few notes on the model:

Apple-side design (phase 5, as decided 2026-09-10)

This section is the phase 5 design note: what CabalmailKit and the two app targets gain, in the order the PRs land. It supersedes the May sketch below it in spirit; the May schema is kept as the record of where the design started. The one server-side change phase 5 needs is the rss_mark_as_read key in set_preferences’s APP_ALLOWED (and the Linux xtask drift test’s mirror of it).

Storage: a thin actor over the system SQLite

CabalmailKit has no third-party Swift dependencies and keeps it that way (operator decision 2026-09-10): RssStore is an actor over the SQLite3 module that ships with every Apple OS, a few hundred lines wrapping sqlite3_prepare_v2 / bind / step for the dozen statements this cache needs. WAL mode, one file per account scope (Application Support/<scope>/rss.sqlite, the same account hashing Preferences.scopeIdentifier uses), forward-only migrations by PRAGMA user_version; downgrade is delete-and-repopulate from the server. The store mirrors the whole RSS catalog, not just items, so the sidebar renders offline:

folders        (folder_id PK, parent_folder_id, name, display_order)
subscriptions  (subscription_id PK, feed_id, folder_id, custom_title,
                ordering_mode, default_open_mode, default_styling,
                notifications_enabled, read_watermark, data_store_uuid,
                feed_title, feed_site_url, feed_type, feed_health_json)
items          (feed_id, sort_key, item_id, guid, title, author, url,
                published_at, fetched_key, summary_html, content_html,
                is_read, is_favorite, state_is_explicit,   -- server state
                cached_at,  PRIMARY KEY (feed_id, sort_key))
items_fts      fts5(title, body_text, content='items', content_rowid=id,
                    tokenize='unicode61')  -- kept by triggers; no stemmer,
                                           -- so typed prefixes match
feed_sync      (feed_id PK, since_cursor, oldest_sort_key, last_synced_at)
pending        (id PK, kind, feed_id, sort_key, value, created_at)
               -- kind: read | favorite | mark_all_read(subscription)

Erratum (2026-09-13): the shipped store (RssStore.swift, at PRAGMA user_version = 4 as of 1.18.0) differs from this sketch in ways Android should mirror. subscriptions carries the feed summary as one feed_json column rather than the four feed_* columns, plus credentials_scheme, created_at, default_remote_content (inherit | show | hide, v3) and default_filter (all | unread | favorite, v4); folders also carries default_filter (v4). items adds updated_at, fetched_at, body_text (the stripped text the FTS table is built from) and state_is_explicit is the local name of the wire’s is_read_explicit; feed_sync replaces oldest_sort_key with older_cursor + older_exhausted (what gates “Load older items”) and adds state_cursor (v2, the state_since cursor; empty on an upgraded store so the next sync pulls the whole state partition); pending adds subscription_id so a mark-all-read row can be replayed in order as a fence. The FTS tokenizer is unicode61 remove_diacritics 2. The read-state rule as shipped is RssStore.readExpression: an explicit row wins, else is_read = 1 OR published_at <= watermark.

body_text for the FTS table is the stripped HTML of summary + content; stripping reuses the Kit’s existing HTMLText plain-text path (the mail snippet code), not NSAttributedString’s WebKit-backed parser. Read state is computed locally by the same rule the server uses (an explicit row wins, else published_at <= read_watermark), so the list, the unread counts (SELECT COUNT per subscription/folder), and the filters never need a network round trip.

Kit API surface

Sync loop

  1. Catalog refresh (/rss_list_subscriptions) on sign-in, on foreground, on pull-to-refresh, and after every management mutation: folders and subscriptions upserted, departed ones deleted along with their items and their WKWebsiteDataStore.
  2. Per-feed item sync via /rss_list_items?since= from the stored cursor, pages of 100 until has_more is false (bounded per run). A new subscription is populated instead from /rss_list_items?order=newest&limit=100 and its cursor set to the largest fetched_key seen; “Load older” pages the same call with its cursor and appends. Subscribed feeds sync with the same concurrency cap the mail sidebar uses for folder counts (4).

  3. Mutation queue: mark read/unread, favorite, and mark-all-read apply to the store immediately and enqueue; the engine drains the queue in /rss_set_item_state batches (≤100) and /rss_mark_all_read calls whenever online, on reconnect (Reachability), and before each item sync. Last write wins; a server row that disagrees after a drain is taken as truth.

    Erratum (2026-09-11): as shipped, nothing carried a server row back to a device that had not written it. The since-sync in step 2 is keyed on fetched_key, which a state change never touches, so a mark made on one device was invisible to the others until they re-listed the item; the wire item had no explicit-state marker, so even a re-listed explicit unread older than the watermark read as read locally; and the drain sent every item mark before any mark-all-read regardless of the order the user made them. Fixed by a state-sync form of /rss_list_items (state_since, backed by a by_updated index), is_read_explicit on the wire, and an order-preserving drain; see docs/rss.md.

  4. Triggers: selecting a feed or folder syncs the visible feeds; foreground syncs everything subscribed; iOS BGAppRefreshTask (system-scheduled, at least hourly requested) and a 15-minute timer on macOS while the app runs; the push-assisted path arrives with phase 8 through the App Group handoff described there.

    Erratum (2026-09-13): no BGAppRefreshTask shipped. The fifteen-minute refresh (AppState+Feeds.swift, mirroring the inbox badge poller) runs on every platform while the app is in the foreground and posts to FeedStateBus; iOS background refresh is deferred to phase 8, where the push wake makes it worth registering. Two triggers were added that this list lacks: the reader’s optimistic local writes (per-feed defaults and the sticky pill) go store-first and then to the server, and the catalog is re-read after every refetch so health marks stay current.

Screens

Delivery order

Four PRs, each green and shippable on its own: 5a Kit (models, client, store, sync engine, tests) plus the APP_ALLOWED key; 5b the read path on iOS and macOS (sidebar section and tab, list, reader, article view); 5c management (subscribe, settings, folders, OPML, the Settings category, macOS commands); 5d search, offline indicators, and polish from dogfooding. The Feeds section and tab appear with 5b.

Erratum (2026-09-13): the order was 5a, 5b, 5d, 5c (5d was taken first because the read path was what was being dogfooded), and per-feed search shipped in 5b rather than 5d. 5b shipped in 1.14.0 and 5c + 5d in 1.15.0. The polish “from dogfooding” then continued for another five releases; the “Post-5c UAT refinements” list under the phase 5 status is the record of it.

Apple-side item cache, FTS, and offline reading (May 2026 sketch)

Kept as the planning record; the design above is the one being built.

This lives entirely in CabalmailKit and the iOS/macOS apps; no server-side change beyond the since cursor on /rss_list_items (phase 3).

Storage

SQLite with FTS5 (Apple’s bundled SQLite ships with FTS5). Library choice is a phase-5 decision to make deliberately: CabalmailKit currently has no third-party Swift package dependencies (its Package.swift declares none; the only vendored code is the marked/turndown JS, materialized by scripts/sync-vendored.sh and pinned via react/admin/package.json). GRDB.swift is the obvious choice and would be the Kit’s first SPM dependency, with the supply-chain and pinning obligations that implies under the supply-chain hardening work. The alternative is a thin actor over the system sqlite3 C API — a few hundred lines for the handful of statements this cache needs. Either is fine; the point is to choose, not to drift into a dependency. The cache schema:

CREATE TABLE items_cache (
  id                  INTEGER PRIMARY KEY AUTOINCREMENT,
  feed_id             TEXT NOT NULL,
  item_id             TEXT NOT NULL,       -- server-side item_id
  guid                TEXT NOT NULL,
  title               TEXT,
  author              TEXT,
  url                 TEXT,
  summary_html        TEXT,
  content_html        TEXT,
  published_at        INTEGER,             -- unix epoch
  fetched_at          INTEGER,             -- server ingest time (sync cursor)
  is_read             INTEGER DEFAULT 0,   -- mirrors server state
  is_favorite         INTEGER DEFAULT 0,
  fetched_locally_at  INTEGER,
  UNIQUE (feed_id, item_id)
);
CREATE INDEX items_cache_feed_pub
  ON items_cache (feed_id, published_at DESC);

CREATE VIRTUAL TABLE items_fts USING fts5(
  title,
  body_text,                                -- stripped HTML
  content='items_cache',
  content_rowid='id',
  tokenize='porter unicode61'
);
-- triggers keep items_fts in sync with items_cache

CREATE TABLE feed_sync_state (
  feed_id                          TEXT PRIMARY KEY,
  last_synced_at                   INTEGER,
  since_cursor                     TEXT,     -- server fetched_at#item_id
  oldest_cached_published_at       INTEGER,
  read_watermark                   INTEGER
);

CREATE TABLE pending_mutations (
  id           INTEGER PRIMARY KEY AUTOINCREMENT,
  item_id      TEXT NOT NULL,
  feed_id      TEXT NOT NULL,
  mutation     TEXT NOT NULL,    -- 'mark_read' | 'mark_unread'
                                 -- | 'favorite' | 'unfavorite'
                                 -- | 'mark_all_read'
  created_at   INTEGER NOT NULL
);

items_fts mirrors stripped-HTML text of summary_html and content_html for indexing. HTML stripping uses Apple’s NSAttributedString(data:options:documentAttributes:) with .html document type — adequate for plaintext extraction, ships with the OS, no third-party dependency. (The Kit’s MIME layer already has HTML-to-text plumbing for mail snippets; reuse it if it fits.)

pending_mutations is the offline-write queue. When the user marks an item read or favorites it while offline, the cache is updated immediately and a row lands in pending_mutations; the next time the client has connectivity, a sync job drains the queue against the server with last-write-wins semantics.

Sync strategy

Three sync paths cooperate:

Initial population. On first launch (or first subscription), the client eagerly fetches the most recent ~100 items per subscribed feed in the background. This makes search useful from day one without hammering the server. A “load older” UI affordance on each feed triggers a one-shot pull of older items into the cache (and re-indexes FTS).

Cache retention

Default: keep items from the last 365 days per feed, plus all favorites unconditionally. User-configurable per-feed override. The eviction job runs on app launch (low priority, batched). The server keeps items forever (per D4), so re-fetching evicted items via “load older” always works.

Storage estimate at default retention: 50 active feeds × ~365 items/year × ~5 KB/item ≈ 90 MB. Comfortable on any current device.

Offline reading

The client’s reader UI reads exclusively from the local cache. “Online vs. offline” is transparent to the user — what’s cached is what’s shown. Articles opened via the web view still require network (per D6); the UI signals this distinction with a small “article requires connection” indicator when offline.

Images inside cached in-feed content rely on whatever the web view’s URLCache happened to fetch on previous online viewing. Explicit pre-cache of image bytes is deferred (could be a v1.x addition).

In-feed content renders through the existing message-body renderer on each platform, including its reader-mode toggle. The two platforms currently implement reader mode differently (Android strips author CSS; Apple overrides it) as a deliberate side-by-side comparison the operator is still running; RSS inherits whichever approach each client has and does not pick a winner.

Per-feed FTS

Search is a feed-scoped FTS5 query against items_fts joined to items_cache:

SELECT items_cache.*
FROM items_cache
JOIN items_fts ON items_fts.rowid = items_cache.id
WHERE items_fts MATCH ? AND items_cache.feed_id = ?
ORDER BY rank;

BM25 ranking is built into FTS5. The UI surfaces a per-feed search field; results show what’s in the cache. A “search older items” affordance triggers a deeper pull from the server before re-running the query.

Cross-device inconsistency

Different devices maintain independent caches; search results scale with each cache. This is acceptable for per-feed search (you search where you are) and is consistent with how every other multi-device RSS reader behaves.

Android-side item cache, FTS, and offline reading

The Android kit module already has a Room database (RoomEnvelopeCache, storing envelopes as wire JSON keyed by folder and uid, exercised on-device rather than under Robolectric). The RSS cache is a sibling in the same module:

Networking

Every API Lambda has been VPC-attached since the private-IMAP replumb (0.11.x): modules/app/modules/call sets vpc_config for all of them, the VPC has free gateway endpoints for S3 and DynamoDB, and the remaining AWS-service calls (SSM, SQS, Cognito, Logs) ride the NAT path by deliberate choice (interface endpoints cost per AZ-hour and those flows are thin). The RSS Lambdas follow the same pattern, for three reasons:

Consequences worth naming:

The May plan’s “Lambda networking policy” (in the VPC only if the function needs an internal-only resource) is superseded: the working rule is that Lambdas in this account are VPC-attached, and a function that wants to be outside must argue for it.

Apple clients

WKWebView supports per-instance website data stores since iOS 17/macOS 14 via WKWebsiteDataStore(forIdentifier: UUID). The Kit floor is iOS 18/macOS 15, so this is available unconditionally. The identifier is the subscription’s data_store_uuid, generated server-side on subscribe and returned with the subscription, so every Apple device of the same user shares one identifier per subscription (the stores themselves are still per-device). When the user opens an article in a web view, the view’s configuration uses the per-feed data store; cookies and localStorage set on the publisher’s site survive across launches and stay isolated from other feeds’ data stores.

Removal: when the user unsubscribes from a feed, the client calls WKWebsiteDataStore.remove(forIdentifier:) to delete all stored cookies and localStorage for that feed.

Erratum (2026-09-13): not as shipped. On macOS 26 WKWebsiteDataStore.remove(forIdentifier:) and allDataStoreIdentifiers segfault in a process that has not yet stood up a web view, which is exactly the state after unsubscribing a feed whose article was never opened (reproduced standalone; WebKit’s defect, not ours). FeedWebStorage (1.15.0) instead opens the store for the identifier and clears it through the instance removeData(ofTypes:modifiedSince:) API. The Android analog (ProfileStore.deleteProfile) has no such known trap, but the Android phase should still clear through the profile’s own cookie manager and storage first, so a delete that fails leaves nothing behind.

Android client

The androidx.webkit multi-profile API (ProfileStore / Profile, applied with WebViewCompat.setProfile) gives each WebView its own cookie jar, storage, and cache, keyed by a profile name — the direct analog of WKWebsiteDataStore(forIdentifier:). It is gated on the installed WebView build via WebViewFeature.isFeatureSupported(MULTI_PROFILE); on API 31+ devices with a current system WebView it is present. Where it is not, the client falls back to the default profile and surfaces the same “sessions are shared across feeds on this device” notice the React analysis once described. Profile name is the subscription’s data_store_uuid; unsubscribing deletes the profile.

What the React analysis established

The May plan spent a section on why a browser cannot partition cookies per app-controlled identity (CHIPS partitions by top-level site; service workers cannot touch Cookie headers; <iframe sandbox> has no cookie flag). With no React RSS UI that analysis is moot for v1; it remains correct if a browser client is ever revisited, and the remedies it listed (open in a new tab; hide credentialed feeds; a server-side cookie-rewriting proxy, not recommended) are still the menu.

Phased implementation

Each phase is independently shippable. Phase 1 is foundational; from phase 3 onward, work can parallelize across the OPML, Apple, Android, image-proxy, and push tracks. Phases are cut along verification boundaries — what can be exercised end to end on stage (by script, by the daily client tester, or by the operator dogfooding) before the next phase lands — rather than by authoring effort, which is no longer the constraint on this project.

Phase 1: DynamoDB tables + supporting infra

Status: Shipped in 1.12.2 (2026-09-09, PR #1488). The per-function IAM grants and the quiesce hook moved to phase 2, where their consumers exist (see the notes in the work list).

Goal. All DynamoDB tables, the item-table stream, the fetch queue and its DLQ, and the rss-cache S3 bucket in place in all three environments. No application code. (The SSM credential hierarchy needs no Terraform until phase 9 writes to it; parameters are created at runtime.)

Work.

The one-shot table-verification Lambda in the May plan is dropped: terraform apply failing is the verification, and the IaC gates (Checkov, tflint, tfsec) already run on every plan.

Rollback. Destroy the resources. Nothing references them.

Phase 2: Scheduler + fetcher Lambdas

Status: On stage (2026-09-09, PR #1489), validated with three seeded public feeds (Atom, RSS, JSON Feed): first fetch stored 78 items, the forced second cycle returned 304 where the publisher honours validators. That soak found one publisher (Cloudflare-fronted) that never 304s when sent the weak ETag Cloudflare substitutes on compressed responses, or any stale If-Modified-Since; the client now sends the strong form of the ETag alone, and the date only when no ETag is known. rss_schedule and rss_fetch under lambda/api/, the shared rss_url / rss_http / rss_parse / rss_cadence modules with unit tests, modules/app/rss_fetcher.tf (roles, log groups, SSM cadence bounds, the five-minute schedule gated on var.quiesced, the queue event source mapping), the front-door/feedbot.html bot page, and the docs/quiesce.md row. As built, two details differ from the text below: the User-Agent is the constant Cabalmail-Feedbot/1 (+https://www.<control-domain>/feedbot.html) rather than carrying the release version (the fetcher zip is built without knowledge of the release), and a permanent redirect whose target already belongs to another feed row is recorded (redirect_conflict_url) rather than merged, because merging needs to re-point every subscriber and that is API-side work for phase 3. Shipped in 1.12.3 (2026-09-09). Two parser and fetch changes followed from real feeds: the www. fallback (1.15.1, described under the phase 5 status) and, in 1.16.0, image enclosures folded into item bodies. When an entry attaches a picture as an image <enclosure>, a media:content of medium image, or a media:thumbnail, and its body has no <img>, rss_parse prepends the picture to the body (NASA’s Image of the Day carries prose alone in the description). Item identity is computed from the body as published, before the fold, so guid-less feeds keep stable ids; the content hash does change, so already-stored items pick the picture up on the next fetch. Clients therefore never need to render enclosures themselves.

Goal. Public feeds are fetched on their cadence, RSS/Atom/JSON parsed, items upserted, health tracked. Validate by seeding a few cabal-rss-feed rows and inspecting cabal-rss-item after a tick.

Work.

Rollback. Disable the Scheduler schedule; data is append-only, no destructive change.

Phase 3: Subscription + reader API

Status: In review (2026-09-09). Eleven rss_* endpoints under lambda/api/ with _shared/rss_api.py (envelope, keys, computed read state, serialization) and _shared/rss_discover.py (autodiscovery); reference in docs/rss.md. As built, versus the text below: health rides on each subscription’s feed summary in /rss_list_subscriptions instead of a separate health endpoint; folders and subscriptions come back from that one call; item state is set in batches through /rss_set_item_state; the first fetch of a new feed is not done inline but by handing the feed to the worker queue immediately (one ingest code path, items within seconds); the day-grouped orderings are applied client-side; unread counts are not served (clients count from their cache); and the shared-feed owner sentinel is ~shared. Stage validation (2026-09-09) found two defects, both fixed in a follow-up: the RSS endpoints lacked dynamodb:BatchWriteItem (unsubscribe’s purge), so they now carry their own IAM statement instead of riding the mail endpoints’ uniform one; and the normalizer’s trailing-slash rule turned a real feed URL (/feeds/json) into a 404 (/feeds/json/). The operator then ruled that / is canonical only right after the host, and elsewhere the server decides (D1, revised): the normalizer neither adds nor removes trailing slashes, and the feed lookup matches the canonical URL exactly. A publisher that treats /feed and /feed/ as one object says so with a 301, which the subscribe probe follows and canonicalizes before looking up, so both forms still land on one shared row without Cabalmail guessing. The subscribe path itself moved into _shared/rss_subscribe_core.py so phase 4’s OPML import shares it, and unsubscribe now deletes state before the subscription row (a failure between the two had stranded state that resurfaced on re-subscribe). Shipped in 1.13.0 (2026-09-10). Three additive changes followed from the Apple acceptance testing, all in docs/rss.md: the state_since form of /rss_list_items with the by_updated index, updated_key on every state write, and is_read_explicit on items and state rows (1.17.0, #1539); default_remote_content on the subscription row, settable through /rss_update_subscription (1.17.0); and default_filter on subscription and folder rows, settable through /rss_update_subscription and /rss_update_folder, plus the filter: prefix arm in set_preferences (1.18.0). None of these changes an existing field, so a client built against 1.13.0 keeps working; a client that wants cross-device read state must consume all three parts of the 1.17.0 change together.

Goal. Authenticated clients can subscribe to a feed, organize feeds into folders, list items with filtering, mark items read/ favorite, and pull incremental updates via the since cursor that the native caches rely on. No image proxy yet (image <img> tags pass through verbatim).

Work.

Rollback. Remove the API routes. Data rows are inert without the fetcher (still running) and the routes (removed).

Phase 4: OPML import/export (API)

Status: In review (2026-09-09). /rss_opml_import and /rss_opml_export with _shared/rss_opml.py. As built: the body carries the OPML text in JSON ({"opml": ...}) rather than a multipart upload, matching every other endpoint; import creates unknown feeds without the interactive probe (from the OPML’s own title) and hands them to the worker, so a large export fits one request and a dead entry surfaces as feed health rather than an import error; folders are reused by name under the same parent and an optional folder_id roots the import; the fixtures are Feedly-, NetNewsWire-, and Reeder-shaped documents (with lower-case attribute variants) in the unit tests rather than real exports, which would carry the operator’s feed list.

Goal. A user can upload an OPML file and have its feeds and folders imported; a user can download an OPML file of their current state. This phase ships the API; the native UIs land with their client phases. It comes before any client phase so the operator’s own feed list can be loaded on stage and dogfooded through the API before a UI exists.

Work.

Rollback. Remove the routes. Existing imported data stays.

Phase 5: Apple clients (with offline + FTS)

Status: 5a (Kit) in review (2026-09-10): RssClient + the URLSessionApiClient conformance, wire models, SQLiteDatabase (the thin sqlite3 actor’s backing type), RssStore with FTS5, RssSyncEngine, the rssMarkAsRead preference (gated like flag_palette), CabalmailClient wiring, 25 Kit tests, the APP_ALLOWED key server-side, and the Linux drift test’s exemption. Two as-built notes: the FTS index uses the plain unicode61 tokenizer, not porter, because stemming stored tokens breaks the typed-prefix matching a search field needs; and sign-out’s clearLocalData() now clears the RSS store too. 5b (read path) on stage (2026-09-10): FeedSidebarViewModel / FeedItemListViewModel / FeedItemDetailViewModel, the Feeds section in the mail sidebar (macOS, iPad-regular) driving the split view’s content and detail columns, a Feeds tab with its own collapsing split on iPhone and visionOS, FeedItemListView (filters, orderings, swipes, per-feed FTS search, load older), FeedItemDetailView on the existing HTMLBodyView, and ArticleWebView on the subscription’s WKWebsiteDataStore(forIdentifier:) with the vendored Readability.js reader toggle (@mozilla/readability pinned in react/admin/package.json, materialized by sync-vendored.sh into RSS/ReaderAssets, credited in Acknowledgements). Session start and foreground trigger syncAll. Not in 5b: subscribe / folder / settings UI, OPML, the Settings category, macOS menu commands (5c), and the list virtualization and offline indicators (5d). Rows in multi-feed lists show the item URL’s host as the feed label until 5d resolves titles. Driven end to end on stage (macOS, 2026-09-10, XCUITest under guidrive): sidebar tree and rolled-up badges, All Feeds and single-feed scopes, filters, per-feed search, the reader’s styling toggle, the article web view and its Readability pass, favorite from the reader. Two follow-ups shipped from that run: the item list, reader, and sidebar now share a FeedStateBus (main-actor, weak subscribers) so a read or favorite change in one patches the others without a reload, and the swipe tint and favorite star read ColorTokens.flagged* (the colour-audit test had failed on stage, which also skipped the TestFlight uploads). Noted for 5d: the disabled mail toolbar items still show while a feed is selected with no item open, compose leaves the toolbar in feed scope, and item rows want accessibility identifiers for the tester. 5d (polish) on stage (2026-09-10), taken before 5c because the read path is what is being dogfooded: rows in multi-feed scopes and the reader header name the feed (subscription title, host as fallback); a fifteen-minute foreground refresh in AppState+Feeds.swift mirrors the inbox badge poller and posts to FeedStateBus so badges and first-page lists follow (iOS background fetch waits for phase 8); the article button reads “needs a connection” while unreachable and ArticleWebView replaces WebKit’s error page with a notice and Retry; “Search older items” appears when a feed search finds nothing cached; the macOS empty feed pane reserves the feed toolbar’s six slots (EmptyFeedDetailToolbar, order shared with the live toolbar through FeedReaderAction) instead of the mail reader’s eleven; New Message stays in the toolbar in feed scope; item rows carry feed.item.<id> identifiers. Deliberately not done: the mail list’s index-addressed virtualization. The feed list reads pages of 100 from SQLite and appends on demand, and SwiftUI’s List is already lazy per row; the mail pattern exists because envelopes arrive from the server by index window, which the local store makes unnecessary. Revisit only if a feed with thousands of cached items scrolls badly. 5c (management) on stage (2026-09-10): FeedManagementViewModel over the Kit’s RssClient + RssStore (server first, then the store, then FeedStateBus, which gained a catalog channel the sidebar re-reads on); SubscribeFeedSheet, FeedFolderSheet (create / rename / move), FeedSubscriptionSettingsSheet (title, folder, ordering, open mode, styling, fetcher health, Unsubscribe); the sidebar’s + menu and row context menus (FeedAddMenu, FeedSidebarContextMenu), hosted by FeedManagementActions + the .feedManagementSheets modifier so the mail sidebar, the Feeds tab, and the item list’s settings button share one presenter; OPML import / export through the system open and save panels (FeedOpml.swift); the Feeds menu (FeedsMenuCommands, ⌥⌘N subscribe) dispatching through AppState.requestFeedCommand; Settings › Feeds with the rss_mark_as_read picker and the OPML actions. Forms are owned by the presenter, not the sheet (#889). Found while driving it: on macOS 26 WKWebsiteDataStore.remove(forIdentifier:) (and allDataStoreIdentifiers) segfaults in a process that has not yet stood up a web view, which is the state after unsubscribing a feed whose article was never opened; FeedWebStorage now opens the store for the identifier and clears it through the instance removeData API instead (reproduced standalone, so it is WebKit’s, not ours). Not done: drag-to-reorder into folders (the settings sheet and “Rename or Move” cover moves) and the notifications toggle (waits for phase 8’s push). Post-5c fix (2026-09-10): the first real OPML imports showed two feeds with no items because the apex host does not serve them: one 404s the apex path, the other redirects every apex path to its front page, and only www. works. D1’s “apex is canonical” is kept as the preference, verified: rss_url.www_variant names the www. form of an apex canonical, the fetcher’s www_fallback and the probe’s fetch_probe try it once when the apex does not yield a feed and adopt it as canonical (conflict-aware via the same path permanent redirects use), redirect_target keeps www. when a redirect points from the apex form to the www. form of the same URL, and the subscribe lookup also checks the www. row so a second subscriber typing the apex form lands on the existing feed.

Post-5c UAT refinements (2026-09-10 to 2026-09-13, releases 1.16.0 to 1.18.1). The operator’s acceptance testing on all four Apple platforms drove the following, each shipped with its own changelog fragment. This list is the Android parity checklist’s source; the erratum blocks in the “Apple-side design” section carry the design consequences.

Still open against the Apple reader at the time of writing: #1548 (the iPhone All Feeds row lacks its siblings’ accent icon and badge) and #1507 (compiler-warning sites from 5b). Neither changes the design.

Goal. The iOS, iPadOS, visionOS, and macOS clients have a reader UI with per-feed WKWebsiteDataStore isolation, offline reading, and per-feed FTS. This is the first user-facing surface.

Work.

Rollback. Hide the RSS tab behind a build flag. ItemCache schema migrations are forward-only; downgrade strategy is “delete and re- populate from server.”

Phase 6: Android client (with offline + FTS)

Status: 6d (polish) in review (2026-09-13); 6a to 6c merged to stage the same day. 6d as built: the article view’s reader toggle on the vendored Readability.js (ReaderAssets, read from assets/reader/; materialized by android/scripts/sync-vendored.sh from react/admin/node_modules, gitignored, run by android.yml and lint.yml before gradle; a build without it offers no toggle), with the Apple state machine (extract once the live page has loaded, cache the document, a link followed from the reader leaves reader mode) and the extraction restyled through readerModeHtml; the scroll-anchor decision: the native fraction. HtmlBody reports the WebView’s own scroll position as a fraction of the scrollable height and restores to one after layout, no JavaScript involved, and FeedReadingPositions (per install, bounded to 200 items, one JSON file) stores it in the Apple reader’s f<fraction> fallback form, so a position captured here restores there once the resume-session plan’s Phase C carries it. The element anchor (i<path>|<delta>) stays Apple-only unless the mail reader’s JavaScript posture changes, which is not a decision for this plan. Feed rows carry the Apple rows’ combined accessibility label (“Unread, title, date”). Not in 6d: the local session record and launch restore, which are the resume-session plan’s Phase B and belong with its mail half. Driven on the Pixel 8 API 35 emulator (2026-09-13): a Daring Fireball item opened into the article view with reader mode on by the feed’s default, and the page came back as the extracted story in the app’s styling; a long AWS post scrolled, left, and reopened at the same place, the position file holding f0.111 under the item’s feedId#sortKey; no crashes.

6c as built, in ui/feeds/: FeedManagementViewModel (the Apple FeedManagementViewModel’s effect order: subscribe and update write the server’s row back and post the catalog change, a new feed’s first page pulled best-effort; unsubscribe, folders, and import re-pull the catalog through the engine, whose diff drops the departed subscriptions’ web profiles), FeedFormRules (the https:// prefix rule, the folder picker walk that excludes a subtree, the settings and folder diffs that keep Save disabled when nothing changed), FeedHealthText and FeedOpmlSummary in the Apple wording, FeedManagementSheets hosting the subscribe, folder, and settings bottom sheets, the unsubscribe / delete-folder / mark-all-read confirmations, the OPML document picker (the file read on IO, the text sent to the model) and the export share through the FileProvider’s attachments path, and the one-shot notices; the tree’s + menu and long-press row menus (folder: subscribe here, new folder inside, rename or move, mark all read, delete; feed: settings, mark all read, open site, unsubscribe; All Feeds: mark all read), the empty state’s subscribe button, a Feed Settings entry in the item list’s menu, and Settings › Feeds’ Subscriptions rows for OPML. Tests: FeedFormsTest and FeedManagementViewModelTest against a server fake that keeps a catalog, so a refresh sees each write. Not in 6c, as on Apple: drag-to-reorder and the notifications toggle (phase 8). Driven on the Pixel 8 API 35 emulator against the claude stage account (2026-09-13): a folder created from the + menu, a feed subscribed into it from the folder’s long-press menu (a site address, autodiscovered by the server, the list opening on the new feed), its title changed in the settings sheet and reflected in the tree, unsubscribed through the confirmation and gone from the tree, the folder renamed and then deleted through its confirmation, and the OPML export offered through the share sheet; no crashes, and the account left as found.

6b as built, in android/app/.../ui/feeds/: a FEEDS top-level destination (a vendored rss_feed glyph, since the core icon set has none) with the feed tree (FeedTree, the Apple FeedSidebarRows rules: folders by display order then name, child folders before feeds, root feeds last, roll-ups over the unfiltered tree, collapse persisted in the local preferences), health marks from FeedHealth (3 / 20 thresholds), and an All Feeds row; FeedItemListViewModel resolving the scope’s row from the store at creation so the stored pill and ordering apply on first open, sticky pills through FeedListFilterPolicy (row for a feed or folder, feedsAllFilter for All Feeds), the four orderings in the overflow menu written back to the feed’s row, swipe read / favorite with the mail rows’ once-per-gesture latch, per-feed search, the older-items footer gated on olderExhausted, a confirmed mark-all-read, and the health headline; FeedItemDetailViewModel reading the item and its subscription from the store, FeedDetailPolicy for the first render, the three toggles writing one field each (no rollback on a failed write), and mark-read-on-open under rssMarkAsRead; the body through the mail reader’s HtmlBody (promoted to internal), both render modes now with the upgrade-insecure-requests meta from a new kit upgradeInsecureRequests, shared with mail; ArticleWebView on androidx.webkit (JavaScript on, the subscription’s data_store_uuid as the profile name where MULTI_PROFILE is supported, an error notice with Retry, system back through the page history), FeedWebProfiles.drop on the catalog diff; the wide layout FeedListDetailScreen with the tree as the leading pane; a Settings › Feeds category with the feed mark-as-read picker; and a fifteen-minute ForegroundPolling on the tree and lists. Routes carry RssItemScope.token and feedId + sortKey (FeedRoutes). Tests: FeedTreeTest, FeedHealthTest, FeedPoliciesTest, and FeedViewModelsTest against the kit’s InMemoryRssStore (first render honours the store row; a pill tap writes once; toggles write one field each; a failed write keeps the reader’s state; mark-read-on-open under the feed key only). Deferred to 6d: the Readability reader toggle in the article view (needs the vendored script materialized into app assets), and the reader’s scroll-anchor capture — Android’s HtmlBody runs with JavaScript off, so the Apple DOM-anchor script cannot run there; 6d must decide between a native scrollY fraction (the f<fraction> form only) and an app-installed bridge, which would mean enabling JavaScript in the mail reader and is not a change to make unasked. Driven on the Pixel 8 API 35 emulator against the claude stage account (2026-09-13): the tree with its roll-ups (All Feeds 97, Tech 15 = Rust Blog 10 + xkcd 5), a feed list opening on its stored Unread pill, a per-feed search hitting the FTS4 index by title and by body, the reader with the feed’s body in reader styling and the header links, the article web view loading the publisher’s page, a confirmed mark-all-read emptying the Unread view and recounting the tree (97 to 87, 15 to 5), and the article toggle sticking as the feed’s default so the next item opened into the article — the Room store’s catalog, items, read-state, FTS, and watermark SQL all exercised on device.

6a as built. com.cabalmail.kit.models.Rss (wire types, lenient enum decoding through RssWire.json, the RssItemScope token codec shared with Apple’s ResumeSession), RssClient implemented by ApiClient (all thirteen endpoints, the three forms of /rss_list_items, the error envelope’s code now carried on CabalmailException.ApiError), the RssStore contract with RssRules (the read-state rule, the four orderings, the search tokenizer, the indexed body text) shared by InMemoryRssStore (the JVM test double) and RoomRssStore (six entities mirroring the Apple schema at version 4, an @Fts4 content table over title and body text with unicode61, the Apple store’s SQL in @RawQuery and @Query strings), RssSyncEngine step for step with the Apple engine (initial population, bounded since- and state-sync loops, load older, the ordered drain with the mark-all-read fence, optimistic subscription and folder updates), and the two preferences (rssMarkAsRead, feedsAllFilter; null = never set, which keeps the key off the wire the way the Apple gates do). Tests mirror the Apple suites: models, API wire shapes, the store contract, and the engine against a scripted FakeRssClient. Two as-built notes: FTS4 has no rank, so search results are newest first; and the Room DAO is exercised on-device only, matching the envelope cache’s posture, so 6b’s first emulator run must include a search and a mark-all-read. Deferred (2026-09-10) before that. With phase 5 shipped, the operator chose to pause and refine the Apple implementation on real use before porting it: each week of dogfooding has been surfacing issues (the www-only publishers, the health badge below) that are cheaper to fix in one client than two. That refinement ran 1.16.0 through 1.18.1 (2026-09-10 to 2026-09-13) and is recorded under the phase 5 status as “Post-5c UAT refinements”; the work list below was rewritten on 2026-09-13 against it, so Android targets the reader as it is, not as the 2026-09-10 design note described it. Android starts when the operator calls the Apple reader settled.

Goal. The Android client reaches parity with phase 5 as shipped in 1.18.1: reader UI, offline reading, per-feed FTS, per-feed WebView profile scoping, cross-device read state, sticky per-feed reader defaults and filter pills, health badges, and the first-day fixes.

Work. Ordered like the Apple PRs (6a kit, 6b read path, 6c management, 6d polish), each shippable on its own; the Feeds destination appears with 6b.

Rollback. Feature flag hides the RSS destination.

Phase 7: Image proxy + cache

Status: Not started.

Goal. All <img> references in served item content are rewritten to the image-proxy URL, which fetches the publisher’s image on first request and caches in S3 for 7 days.

Work.

Rollback. Stop rewriting <img> tags in the read endpoints; pass-through resumes. The bucket can stay or be destroyed independently.

Phase 8: Push notification integration

Status: Not started.

Goal. New items in subscribed feeds with notifications enabled produce APNs and FCM notifications via the existing 0.11.x push path.

Work.

Rollback. Disable the stream event source mapping; the stream retains 24 hours of records, so re-enabling within a day replays missed notifications (or trims them, if that is preferable after an outage).

Phase 9: Credentialed feeds

Status: Not started.

Goal. Users can subscribe to private feeds using HTTP Basic, URL-key, or cookie auth. Credentials stored per-user, per-feed.

Work.

Rollback. Drop the credential endpoints; existing credentialed subscriptions sit dormant (fetcher returns 401, marks feed unhealthy).

Phase 10: Adaptive cadence + health surface polish

Status: Health badge pulled forward, shipped in 1.16.0 (2026-09-11): the first OPML import left two feeds silently empty, and a badge would have said why. FeedHealth (app layer) maps the fetcher’s summary to healthy / failing (3+ consecutive failures, warning tint) / stopped (20+ or dead-lettered, danger tint); the sidebar row shows the mark with the fetcher’s words as tooltip and accessibility label, and a single feed’s item list carries the same words as a header line. The rest of the phase is not started.

Goal. Tune the adaptive cadence formula based on observed production behavior, and surface feed health visibly enough that operator and users can spot problems.

Work.

Rollback. Revert the formula change; the health UI can stay or go independently.

Operational concerns

Cost shape

The new infrastructure budget per environment at hobby scale (one operator + a handful of beta users, ~100 feeds total, ~5000 items/ month):

Item Cost (USD/month)
DynamoDB on-demand (5 tables) ~$0.50 (read+write+storage)
GSI storage (sparse, small) negligible
DynamoDB Stream → Lambda free (trigger reads)
Image cache + spill S3 (7d TTL) <$2
Scheduler + fetch invocations <$1 (288 ticks + fetches/day)
API Lambda invocations scales with reader use, <$1
SQS (fetch queue; push reused) negligible
NAT egress for feed traffic $0 (instances) / <$0.50 (GW)
Total per environment ~$4/month

Dev is quiesced by default per the project’s standing practice and incurs negligible cost when off. Stage + prod together: under $10/month, down from ~$90/month under the original Aurora plan.

DynamoDB scales smoothly upward: at substantially higher use the per-million-request fees start to add up, but the data layer would still be in the low tens of dollars per month at, say, 100 users with typical reader activity.

Quiesce

docs/quiesce.md covers ECS + NAT + ASG. The RSS additions to the quiesce path:

Backup

DynamoDB PITR (35-day retention by default; we use 14 days non-prod, 35 prod) covers all RSS tables, and the tables are in the AWS Backup selection where backups are enabled. The rss-cache bucket is not in AWS Backup: img/ is regenerable on demand, and the items/ spill prefix, while authoritative for oversized bodies, is small and write-once; bucket versioning with a 7-day noncurrent retention is its recovery story for now. (AWS Backup for S3 needs its own service-role policy and continuous-backup configuration; revisit if the spill prefix grows.) SSM credential parameters are KMS-encrypted.

The device-side caches (Apple ItemCache, Android Room) are not backed up by Cabalmail — they are derived caches, rebuilt from the server on demand.

Multi-environment story

Branches/environments per the project’s existing model: development / stage / main. Nothing RSS-specific routes around the existing per-environment AWS account boundary. New infrastructure variables, if any, go in both the plan and apply tfvars steps of infra.yml.

Rollback per phase

Every phase has its own rollback note above. The dependency chain is 1 -> 2 -> 3 -> {4, 5, 6, 7, 8} -> 9 -> 10. Phase 8’s notification flow depends on phase 5/6 only for the on-device enrichment branches; the server side ships independently. Outside this plan, the resume-session plan’s Phase C follows phase 6 (see the sequencing note under phase 6).

Requirements challenges (raised and decided 2026-09-09)

These are places where the requirements doc, read against what had shipped since May, contradicted itself, rested on an assumption that no longer held, or needed a decision the operator had not been asked for. The operator ruled on them on 2026-09-09; each ruling is recorded as a Revised decision (2026-09-09) annotation in rss-requirements.md and applied above; two follow-up rulings closed the remaining items.

  1. D1 normalizer: which host form is canonical. The examples disagreed (www.example.com canonical in one, example.co.uk apex canonical in another). Decided: the apex form is canonical. The rule needs the Public Suffix List to identify the apex.
  2. D1 “http is not supported.” Decided: an http:// input is silently upgraded to https://; if the feed is not available over https, the user is told and nothing is fetched or stored.
  3. D3’s parenthetical about exposing IMAP externally is history: public IMAP and submission closed in 0.11.x. The decision (own API only) stands. No plan change.
  4. D6 “the embedded engine’s reader mode.” Neither WKWebView nor Android WebView exposes the browser’s reader mode to apps. Confirmed: a Readability.js-based reader view satisfies the requirement; the operator reserves the right to request tweaks after it ships.
  5. D15 reading-time estimate “from extracted text.” Under D6 = C there is no extracted text; for summary-only feeds an estimate would be wildly wrong. Decided: dropped from v1, kept as future work (client-side from cached content_html, hidden for summary-only items, if it returns).
  6. D15 auto-mark-read. Decided: the mail clients no longer offer a time-delayed mark-as-read, only manual or immediate-on-open, and feeds offer the same two options under a separate rss_mark_as_read preference rather than the mail mark_as_read key.
  7. D9 egress stability vs. Q3 “scheduled Lambda.” Resolved in the plan by running the fetcher in the VPC behind the NAT EIPs; noted so the choice is explicit. No requirement change.
  8. D10 health visibility with monitoring off. The plan uses CloudWatch metrics and a DLQ alarm because TF_VAR_MONITORING is false everywhere. No requirement change.
  9. D4 “forever” and feeds nobody subscribes to. Decided: when the last user unsubscribes from a feed, its content is deleted; a cold-storage option may be considered in a future release.
  10. Version label vs. semver. Decided: RSS is additive and ships as 1.x minor releases under docs/compatibility.md; the RSS docs moved from docs/2.x/ to docs/1.x/; the roadmap wiki row was updated to match.
  11. Client cut (Q6). The original “phased rollout is fine” answer predates Android shipping. Decided: Apple first, Android second; Linux waits until its mail client has caught up.

Open questions and risks

Open implementation-time items, scoped per phase:

  1. Pagination cursor format for folder-spanning Queries. An opaque base64 JSON blob carrying per-feed LastEvaluatedKey plus a merge position is workable. Design it deliberately in phase 3 so it is a stable contract clients can rely on under docs/compatibility.md.
  2. GSI hot-partition risk on by_due. The fixed PK = "active" sends all due-feed reads to a single partition. At hobby scale this is fine (a few hundred items in the index, queried twelve times an hour). If the feed count grows materially, shard by hash(feed_id) % N and Query in parallel across N constant PK values — which is why the attribute is named due_shard from the start.
  3. Unread computation cost. Items-minus-read-rows above a watermark is cheap per feed and fine for folder views at hobby scale, but “global unread count” touches every subscription. Cache it client-side and refresh on sync rather than asking the server on every badge redraw; if it ever matters server-side, a per- subscription counter maintained by rss_notify’s stream handler is the upgrade path.
  4. Sync strategy tuning on the native clients. The lazy/eager/push-assisted mix is a hypothesis; real-world battery and bandwidth behavior determines whether to bias more toward eager prefetch or lean harder on background-refresh. Tune in phases 5–6.
  5. Cache retention defaults. 365 days + favorites-exempt is a guess. Watch device storage usage in the phase 5/6 beta and adjust the default before GA.
  6. HTML stripping for FTS on Apple. NSAttributedString’s HTML parser is adequate for plaintext extraction but slow on large bodies (it spins up a full WebKit parser internally). If indexing throughput becomes an issue, swap to a lightweight tokenizer.
  7. Adaptive cadence pathologies. A feed that posts in bursts (weekday-only, say) will look slow on weekends and cadence will widen, then Monday’s burst is delayed by up to max_cadence. Time- of-day-aware cadence is out of scope for v1; phase 10 monitors whether it matters.
  8. OPML import edge cases. Real-world OPML files from Feedly, NetNewsWire, and Reeder encode folder hierarchy slightly differently. Phase 4’s fixtures pin down all three explicitly.
  9. Pending-mutation conflict resolution. Last-write-wins is the v1 strategy but breaks down if the user marks an item favorite on iPhone offline, then unfavorites on Mac online, then the iPhone reconnects. Acceptable in v1 (the iPhone wins because its mutation timestamp is later); revisit if it bites. Update (2026-09-13): the first cross-device defect was not this race but the absence of any path carrying a server state row to a device that had not written it (#1539, fixed in 1.17.0 by state_since). The replay order within one device’s queue also mattered: item marks were sent before mark-all-read regardless of the order made, so “mark all read, then one unread” ended read. The queue now replays in order with mark-all-read as a fence. The two-device race described here is still open and still last-write-wins.
  10. Android multi-profile WebView availability. The androidx.webkit multi-profile feature depends on the installed system WebView, not the OS version. Measure how often the fallback path is taken on the tester devices before deciding whether the fallback notice is enough.
  11. Feed-content trust. feedparser sanitizes item HTML, and both clients render bodies through the same sandboxed renderer they use for mail, so a hostile feed has the same (small) surface as a hostile email. Confirm in phase 5/6 that the RSS body view does not grant anything the mail body view withholds (scripts, remote content policy, link handling).
  12. Agent access. The tentative agent-mail-access-plan.md proposes scoped, token-based MCP access to a user’s mailbox. Feeds are an obvious second scope for the same grant model (“read my feeds, never mark them read”). Nothing in this plan should make that harder: keep read endpoints free of side effects and keep the per-user state writes in explicitly named endpoints.

Documentation

When the RSS feature ships, operator-facing documentation lives at docs/rss.md (top-level, per the docs convention) covering:

The docs/1.x/ directory keeps this plan and the requirements doc as the historical planning record.

Next steps

  1. Operator review of this revised plan — particularly the “Requirements challenges” section, the reduced data model, the in-VPC fetcher, and the Apple/Android phase ordering.
  2. If approved, phase 1 is the first PR: Terraform-only — tables, stream, fetch queue, bucket, IAM, and the quiesce hook. CI deploys it to stage.
  3. Each subsequent phase is a worktree branch merged to stage by PR and promoted to main with make promote, with a changelog fragment per change (Apple: / Android: prefixed where the client sources change) and this document’s Progress table updated in the same PR.