Initial Commit

openspec/changes/p01-robust-onepager/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-02-12

openspec/changes/p01-robust-onepager/design.md

@@ -0,0 +1,111 @@
+## Context
+
+ClawFort needs a stunning one-page placeholder website that automatically aggregates and displays AI news hourly. The site must be containerized, use Perplexity API for news generation, and feature infinite scroll with 30-day retention.
+
+**Current State:** Greenfield project - no existing codebase.
+
+**Constraints:**
+- Must use Perplexity API (API key via environment variable)
+- Containerized deployment (Docker)
+- Lean JavaScript framework for frontend
+- 30-day news retention with archiving
+- Hourly automated updates
+
+## Goals / Non-Goals
+
+**Goals:**
+- Stunning one-page website with ClawFort branding
+- Hourly AI news aggregation via Perplexity API
+- Dynamic hero block with featured news and image
+- Infinite scroll news feed (10 initial items)
+- 30-day retention with automatic archiving
+- Source attribution for news and images
+- Fully containerized deployment
+- Responsive design
+
+**Non-Goals:**
+- User authentication/accounts
+- Manual news curation interface
+- Real-time updates (polling only)
+- Multi-language support
+- CMS integration
+- SEO optimization beyond basic meta tags
+
+## Decisions
+
+### Architecture: Monolithic Container
+**Decision:** Single container with frontend + backend + SQLite
+**Rationale:** Simplicity for a placeholder site, easy deployment, no external database dependencies
+**Alternative:** Microservices with separate DB container - rejected as overkill for this scope
+
+### Frontend Framework: Alpine.js + Tailwind CSS
+**Decision:** Alpine.js for lean reactivity, Tailwind for styling
+**Rationale:** Minimal bundle size (~15kb), no build step complexity, perfect for one-page sites
+**Alternative:** React/Vue - rejected as too heavy for simple infinite scroll and hero display
+
+### Backend: Python (FastAPI) + APScheduler
+**Decision:** FastAPI for REST API, APScheduler for cron-like jobs
+**Rationale:** Fast to develop, excellent async support, built-in OpenAPI docs, simple scheduling
+**Alternative:** Node.js/Express - rejected; Python better for data processing and Perplexity integration
+
+### Database: SQLite with SQLAlchemy
+**Decision:** SQLite for zero-config persistence
+**Rationale:** No separate DB container needed, sufficient for 30-day news retention (~1000-2000 records)
+**Alternative:** PostgreSQL - rejected as adds deployment complexity
+
+### News Aggregation Strategy
+**Decision:** Hourly cron job queries Perplexity for "latest AI news" with image generation
+**Rationale:** Simple, reliable, cost-effective
+**Implementation:** 
+- Perplexity API call: "What are the latest AI news from the last hour?"
+- Store: headline, summary, source URL, image URL, timestamp
+- Attribution: Display source name and image credit
+
+### Image Strategy
+**Decision:** Use Perplexity to suggest relevant images or generate via DALL-E if available, with local image optimization
+**Rationale:** Consistent with AI theme, no copyright concerns, plus configurable compression
+**Implementation:** 
+- Download and optimize images locally using Pillow
+- Configurable quality setting via `IMAGE_QUALITY` env var (1-100, default 85)
+- Store optimized images in `/app/static/images/`
+- Serve optimized versions, fallback to original URL if optimization fails
+**Alternative:** Unsplash API - rejected to keep dependencies minimal
+
+### Infinite Scroll Implementation
+**Decision:** Cursor-based pagination with Intersection Observer API
+**Rationale:** Efficient for large datasets, simple Alpine.js integration
+**Page Size:** 10 items per request
+
+### Archive Strategy
+**Decision:** Soft delete (archived flag) + nightly cleanup job
+**Rationale:** Easy to implement, data recoverable if needed
+**Cleanup:** Move items >30 days to archive table or delete
+
+## Risks / Trade-offs
+
+**[Risk] Perplexity API rate limits or downtime** → Mitigation: Implement exponential backoff, cache last successful fetch, display cached content with "last updated" timestamp, fallback to OpenRouter API if configured
+
+**[Risk] Container storage grows unbounded** → Mitigation: SQLite WAL mode, volume mounts for persistence, 30-day hard limit on retention
+
+**[Risk] News quality varies** → Mitigation: Basic filtering (require title + summary), manual blacklist capability in config
+
+**[Risk] Cold start performance** → Mitigation: SQLite connection pooling, frontend CDN-ready static assets
+
+**[Trade-off] SQLite vs PostgreSQL** → SQLite limits concurrent writes but acceptable for read-heavy news site
+
+**[Trade-off] Single container vs microservices** → Easier deployment but less scalable; acceptable for placeholder site
+
+## Migration Plan
+
+1. **Development:** Local Docker Compose setup
+2. **Environment:** Configure `PERPLEXITY_API_KEY` in `.env`
+3. **Build:** `docker build -t clawfort-site .`
+4. **Run:** `docker run -e PERPLEXITY_API_KEY=xxx -p 8000:8000 clawfort-site`
+5. **Data:** SQLite volume mount for persistence across restarts
+
+## Open Questions (Resolved)
+
+1. **Admin panel?** → Deferred to future
+2. **Image optimization?** → Yes, local optimization with Pillow, configurable quality via `IMAGE_QUALITY` env var
+3. **Analytics?** → Umami integration with `UMAMI_SCRIPT_URL` and `UMAMI_WEBSITE_ID` env vars, track page views, scroll events, and CTA clicks
+4. **API cost monitoring?** → Log Perplexity usage, fallback to OpenRouter API if `OPENROUTER_API_KEY` configured

openspec/changes/p01-robust-onepager/proposal.md

@@ -0,0 +1,54 @@
+## Why
+
+ClawFort needs a stunning one-page placeholder website that automatically generates and displays AI news hourly, creating a dynamic, always-fresh brand presence without manual content curation. The site will serve as a living showcase of AI capabilities while building brand recognition.
+
+## What Changes
+
+- **New Capabilities:**
+  - Automated AI news aggregation via Perplexity API (hourly updates)
+  - Dynamic hero section with featured news and images
+  - Infinite scroll news feed with 1-month retention
+  - Archive system for older news items
+  - Containerized deployment (Docker)
+  - Responsive single-page design with lean JavaScript framework
+
+- **Frontend:**
+  - One-page website with hero block
+  - Infinite scroll news feed (latest 10 on load)
+  - News attribution to sources
+  - Image credit display
+  - Responsive design
+
+- **Backend:**
+  - News aggregation service (hourly cron job)
+  - Perplexity API integration
+  - News storage with 30-day retention
+  - Archive management
+  - REST API for frontend
+
+- **Infrastructure:**
+  - Docker containerization
+  - Environment-based configuration
+  - Perplexity API key management
+
+## Capabilities
+
+### New Capabilities
+- `news-aggregator`: Automated AI news collection via Perplexity API with hourly scheduling
+- `news-storage`: Database storage with 30-day retention and archive management
+- `hero-display`: Dynamic hero block with featured news and image attribution
+- `infinite-scroll`: Frontend infinite scroll with lazy loading (10 initial, paginated)
+- `containerized-deployment`: Docker-based deployment with environment configuration
+- `responsive-frontend`: Single-page application with lean JavaScript framework
+
+### Modified Capabilities
+- None (new project)
+
+## Impact
+
+- **Code:** New full-stack application (frontend + backend)
+- **APIs:** Perplexity API integration required
+- **Dependencies:** Docker, Node.js/Python runtime, database (SQLite/PostgreSQL)
+- **Infrastructure:** Container orchestration support
+- **Environment:** `PERPLEXITY_API_KEY` required
+- **Data:** 30-day rolling news archive with automatic cleanup

openspec/changes/p01-robust-onepager/specs/containerized-deployment/spec.md

@@ -0,0 +1,45 @@
+## ADDED Requirements
+
+### Requirement: Containerized deployment
+The system SHALL run entirely within Docker containers with all dependencies included.
+
+#### Scenario: Single container build
+- **WHEN** building the Docker image
+- **THEN** the Dockerfile SHALL include Python runtime, Node.js (for Tailwind if needed), and all application code
+- **AND** expose port 8000 for web traffic
+
+#### Scenario: Environment configuration
+- **WHEN** running the container
+- **THEN** the system SHALL read PERPLEXITY_API_KEY from environment variables
+- **AND** fail to start if the key is missing or invalid
+- **AND** support optional configuration for retention days (default: 30)
+- **AND** support optional IMAGE_QUALITY for image compression (default: 85)
+- **AND** support optional OPENROUTER_API_KEY for fallback LLM provider
+- **AND** support optional UMAMI_SCRIPT_URL and UMAMI_WEBSITE_ID for analytics
+
+#### Scenario: Data persistence
+- **WHEN** the container restarts
+- **THEN** the SQLite database SHALL persist via Docker volume mount
+- **AND** news data SHALL remain intact across restarts
+
+## ADDED Requirements
+
+### Requirement: Responsive single-page design
+The system SHALL provide a stunning, responsive one-page website with ClawFort branding.
+
+#### Scenario: Brand consistency
+- **WHEN** viewing the website
+- **THEN** the design SHALL feature ClawFort branding (logo, colors, typography)
+- **AND** maintain visual consistency across all sections
+
+#### Scenario: Responsive layout
+- **WHEN** viewing on mobile, tablet, or desktop
+- **THEN** the layout SHALL adapt appropriately
+- **AND** the hero block SHALL resize proportionally
+- **AND** the news feed SHALL use appropriate column layouts
+
+#### Scenario: Performance
+- **WHEN** loading the page
+- **THEN** initial page load SHALL complete within 2 seconds
+- **AND** images SHALL lazy load outside viewport
+- **AND** JavaScript bundle SHALL be under 100KB gzipped

openspec/changes/p01-robust-onepager/specs/hero-display/spec.md

@@ -0,0 +1,55 @@
+## ADDED Requirements
+
+### Requirement: Hero block display
+The system SHALL display the most recent news item as a featured hero block with full attribution.
+
+#### Scenario: Hero rendering
+- **WHEN** the page loads
+- **THEN** the hero block SHALL display the latest news headline, summary, and featured image
+- **AND** show source attribution (e.g., "Via: TechCrunch")
+- **AND** show image credit (e.g., "Image: DALL-E")
+
+#### Scenario: Hero update
+- **WHEN** new news is fetched hourly
+- **THEN** the hero block SHALL automatically update to show the newest item
+- **AND** the previous hero item SHALL move to the news feed
+
+## ADDED Requirements
+
+### Requirement: Infinite scroll news feed
+The system SHALL display news items in reverse chronological order with infinite scroll pagination.
+
+#### Scenario: Initial load
+- **WHEN** the page first loads
+- **THEN** the system SHALL display the 10 most recent non-archived news items
+- **AND** exclude the hero item from the feed
+
+#### Scenario: Infinite scroll
+- **WHEN** the user scrolls to the bottom of the feed
+- **THEN** the system SHALL fetch the next 10 news items via API
+- **AND** append them to the feed without page reload
+- **AND** show a loading indicator during fetch
+
+#### Scenario: End of feed
+- **WHEN** all non-archived news items have been loaded
+- **THEN** the system SHALL display "No more news" message
+- **AND** disable further scroll triggers
+
+### Requirement: News attribution display
+The system SHALL clearly attribute all news content and images to their sources.
+
+#### Scenario: Source attribution
+- **WHEN** displaying any news item
+- **THEN** the system SHALL show the original source name and link
+- **AND** display image credit if available
+
+#### Scenario: Perplexity attribution
+- **WHEN** displaying aggregated content
+- **THEN** the system SHALL include "Powered by Perplexity" in the footer
+
+#### Scenario: Analytics tracking
+- **WHEN** Umami analytics is configured via `UMAMI_SCRIPT_URL` and `UMAMI_WEBSITE_ID`
+- **THEN** the system SHALL inject Umami tracking script into page head
+- **AND** track page view events on initial load
+- **AND** track scroll depth events (25%, 50%, 75%, 100%)
+- **AND** track CTA click events (news item clicks, source link clicks)

openspec/changes/p01-robust-onepager/specs/news-aggregator/spec.md

@@ -0,0 +1,56 @@
+## ADDED Requirements
+
+### Requirement: News aggregation via Perplexity API
+The system SHALL fetch AI news hourly from Perplexity API and store it with full attribution.
+
+#### Scenario: Hourly news fetch
+- **WHEN** the scheduled job runs every hour
+- **THEN** the system calls Perplexity API with query "latest AI news"
+- **AND** stores the response with headline, summary, source URL, and timestamp
+
+#### Scenario: API error handling
+- **WHEN** Perplexity API returns an error or timeout
+- **THEN** the system logs the error with cost tracking
+- **AND** retries with exponential backoff up to 3 times
+- **AND** falls back to OpenRouter API if `OPENROUTER_API_KEY` is configured
+- **AND** continues using cached content if all retries and fallback fail
+
+### Requirement: Featured image generation
+The system SHALL generate or fetch a relevant featured image for each news item.
+
+#### Scenario: Image acquisition
+- **WHEN** a new news item is fetched
+- **THEN** the system SHALL request a relevant image URL from Perplexity
+- **AND** download and optimize the image locally using Pillow
+- **AND** apply quality compression based on `IMAGE_QUALITY` env var (1-100, default 85)
+- **AND** store the optimized image path and original image credit/source information
+
+#### Scenario: Image optimization configuration
+- **WHEN** the system processes an image
+- **THEN** it SHALL read `IMAGE_QUALITY` from environment (default: 85)
+- **AND** apply JPEG compression at specified quality level
+- **AND** resize images exceeding 1200px width while maintaining aspect ratio
+- **AND** store optimized images in `/app/static/images/` directory
+
+#### Scenario: Image fallback
+- **WHEN** image generation fails or returns no result
+- **THEN** the system SHALL use a default ClawFort branded placeholder image
+
+## ADDED Requirements
+
+### Requirement: News data persistence with retention
+The system SHALL store news items for exactly 30 days with automatic archiving.
+
+#### Scenario: News storage
+- **WHEN** a news item is fetched from Perplexity
+- **THEN** the system SHALL store it in SQLite with fields: id, headline, summary, source_url, image_url, image_credit, published_at, created_at
+- **AND** set archived=false by default
+
+#### Scenario: Automatic archiving
+- **WHEN** a nightly cleanup job runs
+- **THEN** the system SHALL mark all news items older than 30 days as archived=true
+- **AND** delete archived items older than 60 days permanently
+
+#### Scenario: Duplicate prevention
+- **WHEN** fetching news that matches an existing headline (within 24 hours)
+- **THEN** the system SHALL skip insertion to prevent duplicates

openspec/changes/p01-robust-onepager/tasks.md

@@ -0,0 +1,82 @@
+## 1. Project Setup
+
+- [x] 1.1 Create project directory structure (backend/, frontend/, docker/)
+- [x] 1.2 Initialize Python project with pyproject.toml (FastAPI, SQLAlchemy, APScheduler, httpx)
+- [x] 1.3 Create requirements.txt for Docker build
+- [x] 1.4 Set up Tailwind CSS configuration
+- [x] 1.5 Create .env.example with all environment variables (PERPLEXITY_API_KEY, IMAGE_QUALITY, OPENROUTER_API_KEY, UMAMI_SCRIPT_URL, UMAMI_WEBSITE_ID)
+
+## 2. Database Layer
+
+- [x] 2.1 Create SQLAlchemy models (NewsItem with fields: id, headline, summary, source_url, image_url, image_credit, published_at, created_at, archived)
+- [x] 2.2 Create database initialization and migration scripts
+- [x] 2.3 Implement database connection management with SQLite
+- [x] 2.4 Create repository functions (create_news, get_recent_news, get_news_paginated, archive_old_news, delete_archived_news)
+
+## 3. News Aggregation Service
+
+- [x] 3.1 Implement Perplexity API client with httpx and cost logging
+- [x] 3.2 Create news fetch function with query "latest AI news"
+- [x] 3.3 Implement exponential backoff retry logic (3 attempts)
+- [x] 3.4 Add duplicate detection (headline match within 24h)
+- [x] 3.5 Create hourly scheduled job with APScheduler
+- [x] 3.6 Implement image URL fetching from Perplexity
+- [x] 3.7 Add image download and optimization with Pillow (configurable quality)
+- [x] 3.8 Implement OpenRouter API fallback for news fetching
+- [x] 3.9 Add default placeholder image fallback
+
+## 4. Backend API
+
+- [x] 4.1 Create FastAPI application structure
+- [x] 4.2 Implement GET /api/news endpoint with pagination (cursor-based)
+- [x] 4.3 Implement GET /api/news/latest endpoint for hero block
+- [x] 4.4 Add CORS middleware for frontend access
+- [x] 4.5 Create Pydantic schemas for API responses
+- [x] 4.6 Implement health check endpoint
+- [x] 4.7 Add API error handling and logging
+
+## 5. Frontend Implementation
+
+- [x] 5.1 Create HTML structure with ClawFort branding
+- [x] 5.2 Implement hero block with Alpine.js (latest news display)
+- [x] 5.3 Create news feed component with Alpine.js
+- [x] 5.4 Implement infinite scroll with Intersection Observer API
+- [x] 5.5 Add loading indicators and "No more news" message
+- [x] 5.6 Implement source attribution display
+- [x] 5.7 Add image lazy loading
+- [x] 5.8 Style with Tailwind CSS (responsive design)
+- [x] 5.9 Add "Powered by Perplexity" footer attribution
+- [x] 5.10 Implement Umami analytics integration (conditional on env vars)
+- [x] 5.11 Add analytics events: page view, scroll depth (25/50/75/100%), CTA clicks
+
+## 6. Archive Management
+
+- [x] 6.1 Implement nightly cleanup job (archive >30 days)
+- [x] 6.2 Create permanent deletion job (>60 days archived)
+- [x] 6.3 Add retention configuration (default 30 days)
+
+## 7. Docker Containerization
+
+- [x] 7.1 Create Dockerfile with multi-stage build (Python + static assets)
+- [x] 7.2 Create docker-compose.yml for local development
+- [x] 7.3 Add volume mount for SQLite persistence
+- [x] 7.4 Configure environment variable handling
+- [x] 7.5 Optimize image size (slim Python base)
+- [x] 7.6 Add .dockerignore file
+
+## 8. Testing & Validation
+
+- [x] 8.1 Test Perplexity API integration manually
+- [x] 8.2 Verify hourly news fetching works
+- [x] 8.3 Test infinite scroll pagination
+- [x] 8.4 Verify responsive design on mobile/desktop
+- [x] 8.5 Test container build and run
+- [x] 8.6 Verify data persistence across container restarts
+- [x] 8.7 Test archive cleanup functionality
+
+## 9. Documentation
+
+- [x] 9.1 Create README.md with setup instructions
+- [x] 9.2 Document environment variables
+- [x] 9.3 Add deployment instructions
+- [x] 9.4 Document API endpoints