136 lines
4.1 KiB
Markdown
136 lines
4.1 KiB
Markdown
# ClawFort — AI News Aggregation One-Pager
|
|
|
|
A stunning single-page website that automatically aggregates and displays AI news hourly using the Perplexity API.
|
|
|
|
## Quick Start
|
|
|
|
### Docker (Recommended)
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env and set PERPLEXITY_API_KEY
|
|
|
|
docker compose up --build
|
|
```
|
|
|
|
Open http://localhost:8000
|
|
|
|
### Local Development
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
cp .env.example .env
|
|
# Edit .env and set PERPLEXITY_API_KEY
|
|
|
|
python -m uvicorn backend.main:app --reload --port 8000
|
|
```
|
|
|
|
## Force Fetch Command
|
|
|
|
Use the force-fetch command to run one immediate news ingestion cycle outside the hourly scheduler.
|
|
|
|
```bash
|
|
python -m backend.cli force-fetch
|
|
```
|
|
|
|
Common use cases:
|
|
- **Bootstrap**: Populate initial content right after first deployment.
|
|
- **Recovery**: Re-run ingestion after a failed provider/API cycle.
|
|
|
|
Command behavior:
|
|
- Reuses existing retry, fallback, dedup, image optimization, and persistence logic.
|
|
- Prints success output with stored item count, for example: `force-fetch succeeded: stored=3 elapsed=5.1s`
|
|
- Prints actionable error output on fatal failures and exits non-zero.
|
|
|
|
Exit codes:
|
|
- `0`: Command completed successfully (including runs that store zero new rows)
|
|
- `1`: Fatal command failure (for example missing API keys or unrecoverable runtime error)
|
|
|
|
## Multilingual Support
|
|
|
|
ClawFort supports English (`en`), Tamil (`ta`), and Malayalam (`ml`) content delivery.
|
|
|
|
- New articles are stored in English and translated to Tamil and Malayalam during ingestion.
|
|
- Translations are linked to the same base article and served by the existing news endpoints.
|
|
- If a requested translation is unavailable, the API falls back to English.
|
|
|
|
Language-aware API usage:
|
|
|
|
```bash
|
|
# Latest hero item in Tamil
|
|
curl "http://localhost:8000/api/news/latest?language=ta"
|
|
|
|
# Feed page in Malayalam
|
|
curl "http://localhost:8000/api/news?limit=10&language=ml"
|
|
```
|
|
|
|
Unsupported language codes default to English.
|
|
|
|
Frontend language selector behavior:
|
|
- Landing page includes a language selector (`English`, `Tamil`, `Malayalam`).
|
|
- Selected language is persisted in `localStorage` and mirrored in a client cookie.
|
|
- Returning users see content in their previously selected language.
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Required | Default | Description |
|
|
|----------|----------|---------|-------------|
|
|
| `PERPLEXITY_API_KEY` | Yes | — | Perplexity API key for news fetching |
|
|
| `IMAGE_QUALITY` | No | `85` | JPEG compression quality (1-100) |
|
|
| `OPENROUTER_API_KEY` | No | — | Fallback LLM provider API key |
|
|
| `RETENTION_DAYS` | No | `30` | Days to keep news before archiving |
|
|
| `UMAMI_SCRIPT_URL` | No | — | Umami analytics script URL |
|
|
| `UMAMI_WEBSITE_ID` | No | — | Umami website tracking ID |
|
|
|
|
## Architecture
|
|
|
|
- **Backend**: Python (FastAPI) + SQLAlchemy + APScheduler
|
|
- **Frontend**: Alpine.js + Tailwind CSS (CDN, no build step)
|
|
- **Database**: SQLite with 30-day retention
|
|
- **Container**: Single Docker container
|
|
|
|
## API Endpoints
|
|
|
|
| Method | Path | Description |
|
|
|--------|------|-------------|
|
|
| `GET` | `/` | Serve frontend |
|
|
| `GET` | `/api/news/latest` | Latest news item for hero block |
|
|
| `GET` | `/api/news` | Paginated news feed |
|
|
| `GET` | `/api/health` | Health check with news count |
|
|
| `GET` | `/config` | Frontend config (analytics) |
|
|
|
|
### GET /api/news
|
|
|
|
Query parameters:
|
|
- `cursor` (int, optional): Last item ID for pagination
|
|
- `limit` (int, default 10): Items per page (max 50)
|
|
- `exclude_hero` (int, optional): Hero item ID to exclude
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"items": [{ "id": 1, "headline": "...", "summary": "...", "source_url": "...", "image_url": "...", "image_credit": "...", "published_at": "...", "created_at": "..." }],
|
|
"next_cursor": 5,
|
|
"has_more": true
|
|
}
|
|
```
|
|
|
|
## Deployment
|
|
|
|
```bash
|
|
docker build -t clawfort .
|
|
docker run -d \
|
|
-e PERPLEXITY_API_KEY=pplx-xxx \
|
|
-v clawfort-data:/app/data \
|
|
-v clawfort-images:/app/backend/static/images \
|
|
-p 8000:8000 \
|
|
clawfort
|
|
```
|
|
|
|
Data persists across restarts via Docker volumes.
|
|
|
|
## Scheduled Jobs
|
|
|
|
- **Hourly**: Fetch latest AI news from Perplexity API
|
|
- **Nightly (3 AM)**: Archive news older than 30 days, delete archived items older than 60 days
|