better cache

This commit is contained in:
2026-02-10 01:20:58 -05:00
parent c773affbc8
commit f056e67eae
39 changed files with 830 additions and 17 deletions

View File

@@ -0,0 +1,38 @@
## ADDED Requirements
### Requirement: Redis-backed cache service
The system MUST provide a Redis-backed cache service for use by ingestion and content processing flows.
The cache service MUST be runnable in local development via Docker Compose.
#### Scenario: Cache service available in Docker
- **WHEN** the Docker Compose stack is started
- **THEN** a Redis service is available to other services/scripts on the internal network
### Requirement: TTL-based invalidation
Cached entries MUST support TTL-based invalidation.
The system MUST define a default TTL and MUST allow overriding the TTL via environment/config.
#### Scenario: Default TTL applies
- **WHEN** a cached entry is written without an explicit TTL override
- **THEN** it expires after the configured default TTL
#### Scenario: TTL override applies
- **WHEN** a TTL override is configured via environment/config
- **THEN** new cached entries use that TTL for expiration
### Requirement: Cache key namespace
Cache keys MUST be namespaced by source and parameters so that different data requests do not collide.
#### Scenario: Two different sources do not collide
- **WHEN** the system caches a YouTube fetch and a WordPress fetch
- **THEN** they use different key namespaces and do not overwrite each other
### Requirement: Manual cache clear
The system MUST provide a script/command to manually clear the cache.
#### Scenario: Manual clear executed
- **WHEN** a developer runs the cache clear command
- **THEN** the cache is cleared and subsequent ingestion runs produce cache misses

View File

@@ -0,0 +1,23 @@
## MODIFIED Requirements
### Requirement: Refresh and caching
The system MUST cache the latest successful ingestion output and MUST serve the cached data to the site renderer.
The system MUST support periodic refresh on a schedule (at minimum daily) and MUST support a manual refresh trigger.
On ingestion failure, the system MUST continue serving the most recent cached data.
The ingestion pipeline MUST use the cache layer (when configured and reachable) to reduce repeated network and parsing work for external sources (for example, YouTube API/RSS and podcast RSS).
#### Scenario: Scheduled refresh fails
- **WHEN** a scheduled refresh run fails to fetch one or more sources
- **THEN** the site continues to use the most recent successfully cached dataset
#### Scenario: Manual refresh requested
- **WHEN** a manual refresh is triggered
- **THEN** the system attempts ingestion immediately and updates the cache if ingestion succeeds
#### Scenario: Cache hit avoids refetch
- **WHEN** a refresh run is executed within the cache TTL for a given source+parameters
- **THEN** the ingestion pipeline uses cached data for that source instead of refetching over the network

View File

@@ -0,0 +1,19 @@
## MODIFIED Requirements
### Requirement: Build-time caching
WordPress posts, pages, and categories MUST be written into the repo-local content cache used by the site build.
If the WordPress fetch fails, the system MUST NOT crash the entire build pipeline; it MUST either:
- keep the last-known-good cached WordPress content (if present), or
- store an empty WordPress dataset and allow the rest of the site to build.
When the cache layer is configured and reachable, the WordPress ingestion MUST cache `wp-json` responses (or normalized outputs) using a TTL so repeated ingestion runs avoid unnecessary network requests and parsing work.
#### Scenario: WordPress fetch fails
- **WHEN** a WordPress API request fails
- **THEN** the site build can still complete and the blog surface renders a graceful empty state
#### Scenario: Cache hit avoids wp-json refetch
- **WHEN** WordPress ingestion is executed within the configured cache TTL
- **THEN** it uses cached data instead of refetching from `wp-json`