clawfort/README.md

ClawFort — AI News Aggregation One-Pager

A stunning single-page website that automatically aggregates and displays AI news hourly using the Perplexity API.

Quick Start

Docker (Recommended)

cp .env.example .env
# Edit .env and set PERPLEXITY_API_KEY

docker compose up --build

Open http://localhost:8000

Local Development

pip install -r requirements.txt
cp .env.example .env
# Edit .env and set PERPLEXITY_API_KEY

python -m uvicorn backend.main:app --reload --port 8000

Force Fetch Command

Use the force-fetch command to run one immediate news ingestion cycle outside the hourly scheduler.

python -m backend.cli force-fetch

Common use cases:

• Bootstrap: Populate initial content right after first deployment.
• Recovery: Re-run ingestion after a failed provider/API cycle.

Command behavior:

• Reuses existing retry, fallback, dedup, image optimization, and persistence logic.
• Prints success output with stored item count, for example: force-fetch succeeded: stored=3 elapsed=5.1s
• Prints actionable error output on fatal failures and exits non-zero.

Exit codes:

• 0: Command completed successfully (including runs that store zero new rows)
• 1: Fatal command failure (for example missing API keys or unrecoverable runtime error)

Multilingual Support

ClawFort supports English (en), Tamil (ta), and Malayalam (ml) content delivery.

• New articles are stored in English and translated to Tamil and Malayalam during ingestion.
• Translations are linked to the same base article and served by the existing news endpoints.
• If a requested translation is unavailable, the API falls back to English.

Language-aware API usage:

# Latest hero item in Tamil
curl "http://localhost:8000/api/news/latest?language=ta"

# Feed page in Malayalam
curl "http://localhost:8000/api/news?limit=10&language=ml"

Unsupported language codes default to English.

Frontend language selector behavior:

• Landing page includes a language selector (English, Tamil, Malayalam).
• Selected language is persisted in localStorage and mirrored in a client cookie.
• Returning users see content in their previously selected language.

Environment Variables

Variable	Required	Default	Description
PERPLEXITY_API_KEY	Yes	—	Perplexity API key for news fetching
IMAGE_QUALITY	No	85	JPEG compression quality (1-100)
OPENROUTER_API_KEY	No	—	Fallback LLM provider API key
RETENTION_DAYS	No	30	Days to keep news before archiving
UMAMI_SCRIPT_URL	No	—	Umami analytics script URL
UMAMI_WEBSITE_ID	No	—	Umami website tracking ID

Architecture

• Backend: Python (FastAPI) + SQLAlchemy + APScheduler
• Frontend: Alpine.js + Tailwind CSS (CDN, no build step)
• Database: SQLite with 30-day retention
• Container: Single Docker container

API Endpoints

Method	Path	Description
GET	/	Serve frontend
GET	/api/news/latest	Latest news item for hero block
GET	/api/news	Paginated news feed
GET	/api/health	Health check with news count
GET	/config	Frontend config (analytics)

GET /api/news

Query parameters:

• cursor (int, optional): Last item ID for pagination
• limit (int, default 10): Items per page (max 50)
• exclude_hero (int, optional): Hero item ID to exclude

Response:

{
  "items": [{ "id": 1, "headline": "...", "summary": "...", "source_url": "...", "image_url": "...", "image_credit": "...", "published_at": "...", "created_at": "..." }],
  "next_cursor": 5,
  "has_more": true
}

Deployment

docker build -t clawfort .
docker run -d \
  -e PERPLEXITY_API_KEY=pplx-xxx \
  -v clawfort-data:/app/data \
  -v clawfort-images:/app/backend/static/images \
  -p 8000:8000 \
  clawfort

Data persists across restarts via Docker volumes.

Scheduled Jobs

• Hourly: Fetch latest AI news from Perplexity API
• Nightly (3 AM): Archive news older than 30 days, delete archived items older than 60 days