Sprint 0-1: Python indexer, TS plugin scaffolding, and test suite

## What's new

**Python indexer (`python/obsidian_rag/`)** — full pipeline from scan to LanceDB:
- `config.py` — JSON config loader with cross-platform path resolution
- `security.py` — path traversal prevention, HTML stripping, sensitive content detection, dir allow/deny lists
- `chunker.py` — section-split for journal entries (date-named files), sliding-window for unstructured notes
- `embedder.py` — Ollama `/api/embeddings` client with batched requests and timeout/error handling
- `vector_store.py` — LanceDB schema, upsert (merge_insert), delete, search with filters, stats
- `indexer.py` — full/sync/reindex pipeline orchestrator with progress yields
- `cli.py` — `index | sync | reindex | status` CLI commands

**TypeScript plugin (`src/`)** — OpenClaw plugin scaffold:
- `utils/` — config loader, TypeScript types, response envelope factory, LanceDB client
- `services/` — health state machine (HEALTHY/DEGRADED/UNAVAILABLE), vault watcher with debounce/batching, indexer bridge (subprocess spawner)
- `tools/` — 4 tool stubs: search, index, status, memory_store (OpenClaw wiring pending)
- `index.ts` — plugin entry point with health probe + vault watcher startup

**Config** (`obsidian-rag/config.json`, `openclaw.plugin.json`):
- 627 files / 3764 chunks indexed in dev vault

**Tests: 76 passing**
- Python: 64 pytest tests (chunker, security, vector_store, config)
- TypeScript: 12 vitest tests (lancedb client, response envelope)

## Bugs fixed

- LanceDB `tags` column filter: `LIKE '%tag%'` → `list_contains(tags, 'tag')` (List<String> column)
- LanceDB JS `db.list_tables()` returns `ListTablesResponse` object, not plain array
- LanceDB JS result score field: `_score` → `_distance`
- TypeScript regex literal with unescaped `/` in path-resolve regex
- Python: `create_table_if_not_exists` identity check → name comparison

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-04-10 22:56:50 -04:00
parent 18ad47e100
commit 5c281165c7
40 changed files with 5814 additions and 59 deletions

44
src/tools/status.ts Normal file
View File

@@ -0,0 +1,44 @@
/** obsidian_rag_status tool implementation. */
import type { ObsidianRagConfig } from "../utils/config.js";
import type { ResponseEnvelope } from "../utils/types.js";
import { makeEnvelope } from "../utils/response.js";
import { readSyncResult } from "../services/indexer-bridge.js";
export async function statusTool(
config: ObsidianRagConfig,
): Promise<ResponseEnvelope<{
plugin_health: string;
total_docs: number;
total_chunks: number;
last_sync: string | null;
unindexed_files: number;
ollama_status: string;
active_job: null;
}>> {
const sync = readSyncResult(config);
const ollamaUp = await checkOllama(config);
return makeEnvelope(
sync ? "healthy" : "unavailable",
{
plugin_health: sync ? "healthy" : "unavailable",
total_docs: sync?.indexed_files ?? 0,
total_chunks: sync?.total_chunks ?? 0,
last_sync: sync?.timestamp ?? null,
unindexed_files: 0,
ollama_status: ollamaUp ? "up" : "down",
active_job: null,
},
null,
);
}
async function checkOllama(config: ObsidianRagConfig): Promise<boolean> {
try {
const res = await fetch(`${config.embedding.base_url}/api/tags`, { signal: AbortSignal.timeout(3000) });
return res.ok;
} catch {
return false;
}
}