Files

5.2 KiB

Context

ClawFort needs a stunning one-page placeholder website that automatically aggregates and displays AI news hourly. The site must be containerized, use Perplexity API for news generation, and feature infinite scroll with 30-day retention.

Current State: Greenfield project - no existing codebase.

Constraints:

  • Must use Perplexity API (API key via environment variable)
  • Containerized deployment (Docker)
  • Lean JavaScript framework for frontend
  • 30-day news retention with archiving
  • Hourly automated updates

Goals / Non-Goals

Goals:

  • Stunning one-page website with ClawFort branding
  • Hourly AI news aggregation via Perplexity API
  • Dynamic hero block with featured news and image
  • Infinite scroll news feed (10 initial items)
  • 30-day retention with automatic archiving
  • Source attribution for news and images
  • Fully containerized deployment
  • Responsive design

Non-Goals:

  • User authentication/accounts
  • Manual news curation interface
  • Real-time updates (polling only)
  • Multi-language support
  • CMS integration
  • SEO optimization beyond basic meta tags

Decisions

Architecture: Monolithic Container

Decision: Single container with frontend + backend + SQLite Rationale: Simplicity for a placeholder site, easy deployment, no external database dependencies Alternative: Microservices with separate DB container - rejected as overkill for this scope

Frontend Framework: Alpine.js + Tailwind CSS

Decision: Alpine.js for lean reactivity, Tailwind for styling Rationale: Minimal bundle size (~15kb), no build step complexity, perfect for one-page sites Alternative: React/Vue - rejected as too heavy for simple infinite scroll and hero display

Backend: Python (FastAPI) + APScheduler

Decision: FastAPI for REST API, APScheduler for cron-like jobs Rationale: Fast to develop, excellent async support, built-in OpenAPI docs, simple scheduling Alternative: Node.js/Express - rejected; Python better for data processing and Perplexity integration

Database: SQLite with SQLAlchemy

Decision: SQLite for zero-config persistence Rationale: No separate DB container needed, sufficient for 30-day news retention (~1000-2000 records) Alternative: PostgreSQL - rejected as adds deployment complexity

News Aggregation Strategy

Decision: Hourly cron job queries Perplexity for "latest AI news" with image generation Rationale: Simple, reliable, cost-effective Implementation:

  • Perplexity API call: "What are the latest AI news from the last hour?"
  • Store: headline, summary, source URL, image URL, timestamp
  • Attribution: Display source name and image credit

Image Strategy

Decision: Use Perplexity to suggest relevant images or generate via DALL-E if available, with local image optimization Rationale: Consistent with AI theme, no copyright concerns, plus configurable compression Implementation:

  • Download and optimize images locally using Pillow
  • Configurable quality setting via IMAGE_QUALITY env var (1-100, default 85)
  • Store optimized images in /app/static/images/
  • Serve optimized versions, fallback to original URL if optimization fails Alternative: Unsplash API - rejected to keep dependencies minimal

Infinite Scroll Implementation

Decision: Cursor-based pagination with Intersection Observer API Rationale: Efficient for large datasets, simple Alpine.js integration Page Size: 10 items per request

Archive Strategy

Decision: Soft delete (archived flag) + nightly cleanup job Rationale: Easy to implement, data recoverable if needed Cleanup: Move items >30 days to archive table or delete

Risks / Trade-offs

[Risk] Perplexity API rate limits or downtime → Mitigation: Implement exponential backoff, cache last successful fetch, display cached content with "last updated" timestamp, fallback to OpenRouter API if configured

[Risk] Container storage grows unbounded → Mitigation: SQLite WAL mode, volume mounts for persistence, 30-day hard limit on retention

[Risk] News quality varies → Mitigation: Basic filtering (require title + summary), manual blacklist capability in config

[Risk] Cold start performance → Mitigation: SQLite connection pooling, frontend CDN-ready static assets

[Trade-off] SQLite vs PostgreSQL → SQLite limits concurrent writes but acceptable for read-heavy news site

[Trade-off] Single container vs microservices → Easier deployment but less scalable; acceptable for placeholder site

Migration Plan

  1. Development: Local Docker Compose setup
  2. Environment: Configure PERPLEXITY_API_KEY in .env
  3. Build: docker build -t clawfort-site .
  4. Run: docker run -e PERPLEXITY_API_KEY=xxx -p 8000:8000 clawfort-site
  5. Data: SQLite volume mount for persistence across restarts

Open Questions (Resolved)

  1. Admin panel? → Deferred to future
  2. Image optimization? → Yes, local optimization with Pillow, configurable quality via IMAGE_QUALITY env var
  3. Analytics? → Umami integration with UMAMI_SCRIPT_URL and UMAMI_WEBSITE_ID env vars, track page views, scroll events, and CTA clicks
  4. API cost monitoring? → Log Perplexity usage, fallback to OpenRouter API if OPENROUTER_API_KEY configured