Initial Commit

2026-02-12 16:50:29 -05:00
commit a1da041f14
74 changed files with 6140 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,135 @@
+# ClawFort — AI News Aggregation One-Pager
+
+A stunning single-page website that automatically aggregates and displays AI news hourly using the Perplexity API.
+
+## Quick Start
+
+### Docker (Recommended)
+
+```bash
+cp .env.example .env
+# Edit .env and set PERPLEXITY_API_KEY
+
+docker compose up --build
+```
+
+Open http://localhost:8000
+
+### Local Development
+
+```bash
+pip install -r requirements.txt
+cp .env.example .env
+# Edit .env and set PERPLEXITY_API_KEY
+
+python -m uvicorn backend.main:app --reload --port 8000
+```
+
+## Force Fetch Command
+
+Use the force-fetch command to run one immediate news ingestion cycle outside the hourly scheduler.
+
+```bash
+python -m backend.cli force-fetch
+```
+
+Common use cases:
+- **Bootstrap**: Populate initial content right after first deployment.
+- **Recovery**: Re-run ingestion after a failed provider/API cycle.
+
+Command behavior:
+- Reuses existing retry, fallback, dedup, image optimization, and persistence logic.
+- Prints success output with stored item count, for example: `force-fetch succeeded: stored=3 elapsed=5.1s`
+- Prints actionable error output on fatal failures and exits non-zero.
+
+Exit codes:
+- `0`: Command completed successfully (including runs that store zero new rows)
+- `1`: Fatal command failure (for example missing API keys or unrecoverable runtime error)
+
+## Multilingual Support
+
+ClawFort supports English (`en`), Tamil (`ta`), and Malayalam (`ml`) content delivery.
+
+- New articles are stored in English and translated to Tamil and Malayalam during ingestion.
+- Translations are linked to the same base article and served by the existing news endpoints.
+- If a requested translation is unavailable, the API falls back to English.
+
+Language-aware API usage:
+
+```bash
+# Latest hero item in Tamil
+curl "http://localhost:8000/api/news/latest?language=ta"
+
+# Feed page in Malayalam
+curl "http://localhost:8000/api/news?limit=10&language=ml"
+```
+
+Unsupported language codes default to English.
+
+Frontend language selector behavior:
+- Landing page includes a language selector (`English`, `Tamil`, `Malayalam`).
+- Selected language is persisted in `localStorage` and mirrored in a client cookie.
+- Returning users see content in their previously selected language.
+
+## Environment Variables
+
+| Variable | Required | Default | Description |
+|----------|----------|---------|-------------|
+| `PERPLEXITY_API_KEY` | Yes | — | Perplexity API key for news fetching |
+| `IMAGE_QUALITY` | No | `85` | JPEG compression quality (1-100) |
+| `OPENROUTER_API_KEY` | No | — | Fallback LLM provider API key |
+| `RETENTION_DAYS` | No | `30` | Days to keep news before archiving |
+| `UMAMI_SCRIPT_URL` | No | — | Umami analytics script URL |
+| `UMAMI_WEBSITE_ID` | No | — | Umami website tracking ID |
+
+## Architecture
+
+- **Backend**: Python (FastAPI) + SQLAlchemy + APScheduler
+- **Frontend**: Alpine.js + Tailwind CSS (CDN, no build step)
+- **Database**: SQLite with 30-day retention
+- **Container**: Single Docker container
+
+## API Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/` | Serve frontend |
+| `GET` | `/api/news/latest` | Latest news item for hero block |
+| `GET` | `/api/news` | Paginated news feed |
+| `GET` | `/api/health` | Health check with news count |
+| `GET` | `/config` | Frontend config (analytics) |
+
+### GET /api/news
+
+Query parameters:
+- `cursor` (int, optional): Last item ID for pagination
+- `limit` (int, default 10): Items per page (max 50)
+- `exclude_hero` (int, optional): Hero item ID to exclude
+
+Response:
+```json
+{
+  "items": [{ "id": 1, "headline": "...", "summary": "...", "source_url": "...", "image_url": "...", "image_credit": "...", "published_at": "...", "created_at": "..." }],
+  "next_cursor": 5,
+  "has_more": true
+}
+```
+
+## Deployment
+
+```bash
+docker build -t clawfort .
+docker run -d \
+  -e PERPLEXITY_API_KEY=pplx-xxx \
+  -v clawfort-data:/app/data \
+  -v clawfort-images:/app/backend/static/images \
+  -p 8000:8000 \
+  clawfort
+```
+
+Data persists across restarts via Docker volumes.
+
+## Scheduled Jobs
+
+- **Hourly**: Fetch latest AI news from Perplexity API
+- **Nightly (3 AM)**: Archive news older than 30 days, delete archived items older than 60 days