From 2c976bb75b7a859c4c5519b3d20109e56e8a54c5 Mon Sep 17 00:00:00 2001 From: Santhosh Janardhanan Date: Fri, 10 Apr 2026 19:00:38 -0400 Subject: [PATCH] all in --- .claude/settings.local.json | 14 ++ .../content/architecture-overview.html | 103 +++++++++++ .../27-1775849590/content/data-layer.html | 150 ++++++++++++++++ .../27-1775849590/content/security-layer.html | 126 +++++++++++++ .../27-1775849590/content/session-layer.html | 115 ++++++++++++ .../content/testing-strategy.html | 147 +++++++++++++++ .../27-1775849590/content/tool-layer.html | 170 ++++++++++++++++++ .../content/transport-layer.html | 130 ++++++++++++++ .../27-1775849590/state/server-stopped | 1 + .../brainstorm/27-1775849590/state/server.pid | 1 + 10 files changed, 957 insertions(+) create mode 100644 .claude/settings.local.json create mode 100644 .superpowers/brainstorm/27-1775849590/content/architecture-overview.html create mode 100644 .superpowers/brainstorm/27-1775849590/content/data-layer.html create mode 100644 .superpowers/brainstorm/27-1775849590/content/security-layer.html create mode 100644 .superpowers/brainstorm/27-1775849590/content/session-layer.html create mode 100644 .superpowers/brainstorm/27-1775849590/content/testing-strategy.html create mode 100644 .superpowers/brainstorm/27-1775849590/content/tool-layer.html create mode 100644 .superpowers/brainstorm/27-1775849590/content/transport-layer.html create mode 100644 .superpowers/brainstorm/27-1775849590/state/server-stopped create mode 100644 .superpowers/brainstorm/27-1775849590/state/server.pid diff --git a/.claude/settings.local.json b/.claude/settings.local.json new file mode 100644 index 0000000..2858609 --- /dev/null +++ b/.claude/settings.local.json @@ -0,0 +1,14 @@ +{ + "permissions": { + "allow": [ + "Bash(bash:*)", + "Bash(cat /c/dev/obsidian-claw/.superpowers/brainstorm/*/state/server-info)", + "Bash(ls -d /c/dev/obsidian-claw/KnowledgeVault/Default/*/)", + "Bash(git init:*)", + "Bash(git add:*)", + "Bash(git commit -m ':*)", + "WebFetch(domain:www.ollama.com)", + "mcp__web-reader__webReader" + ] + } +} diff --git a/.superpowers/brainstorm/27-1775849590/content/architecture-overview.html b/.superpowers/brainstorm/27-1775849590/content/architecture-overview.html new file mode 100644 index 0000000..f1830b2 --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/content/architecture-overview.html @@ -0,0 +1,103 @@ +

Architecture Overview

+

Layered Protocol Model — Obsidian RAG Plugin for OpenClaw

+ +
+

System Boundary Diagram

+
+
Architecture: Obsidian RAG Plugin
+
+
+ +
+
+
USER LAYER
+
Natural Language Query
+
"How was my mental health in 2024?"
+
+
+ +
+
+
OPENCLAW AGENT
+
Tool Router
+
Selects plugin tools based on intent
+
Session Manager
+
Tracks plugin health state
+
+
+
+ +
↕ Tool Calls & Results
+ + +
+
PLUGIN LAYER (TypeScript)
+
+
+
TRANSPORT
+
Tool Registry
+
Error Normalizer
+
+
+
SESSION
+
Vault Watcher
+
Sync Scheduler
+
+
+
TOOL
+
search
+
index / status
+
memory_store
+
+
+
DATA
+
LanceDB Client
+
Indexer Bridge
+
+
+
+ +
↕ CLI & Filesystem
+ + +
+
+
DATA LAYER
+
LanceDB (vectors.lance)
+
Ollama (localhost:11434)
+
Obsidian Vault (filesystem)
+
+
+
INDEXER (Python CLI)
+
obsidian-rag index
+
obsidian-rag sync
+
obsidian-rag reindex
+
obsidian-rag status
+
+
+
+
+
+ +
+

Key Design Decisions

+
+

Chosen

+
    +
  • Layered protocol: Transport → Session → Tool → Data
    • +
  • Plugin is thin TS wrapper; heavy lifting in Python
    • +
  • LanceDB embedded — no server process
    • +
  • Vault watcher auto-syncs on file changes
    • +
  • Graceful degradation on dependency failures
    • +
+
+

Rejected

+
    +
  • Event bus (over-engineered for single plugin)
    • +
  • Minimal spec (no protocol contract = ad-hoc)
    • +
  • Server-based vector DB (unnecessary complexity)
    • +
  • On-demand-only sync (user chose auto-sync)
    • +
+
+
+
\ No newline at end of file diff --git a/.superpowers/brainstorm/27-1775849590/content/data-layer.html b/.superpowers/brainstorm/27-1775849590/content/data-layer.html new file mode 100644 index 0000000..750362a --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/content/data-layer.html @@ -0,0 +1,150 @@ +

Data Layer

+

LanceDB Schema, Indexer Bridge, Chunking & Embedding Pipeline

+ +
+

LanceDB Table Schema

+
+
obsidian_chunks table
+
+
┌──────────────────┬──────────────┬─────────────────────────────────────┐
+│ Column           │ Type         │ Description                         │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ vector           │ FixedList    │ 1024-dim float32 embedding          │
+│                  │ float32[1024]│ (mxbai-embed-large via Ollama)     │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ chunk_id         │ string       │ UUID v4, primary key                │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ chunk_text       │ string       │ Raw markdown text of the chunk      │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ source_file      │ string       │ Relative path from vault root       │
+│                  │              │ e.g. "Journal/2024-01-15.md"       │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ source_directory │ string       │ Top-level directory name            │
+│                  │              │ e.g. "Journal", "Entertainment"    │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ section          │ string|null  │ Section heading (structured notes) │
+│                  │              │ e.g. "#mentalhealth", "#finance"   │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ date             │ string|null  │ ISO 8601 date parsed from filename │
+│                  │              │ e.g. "2024-01-15"                  │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ tags             │ list<string> │ All hashtags in chunk              │
+│                  │              │ e.g. ["#mentalhealth", "#therapy"] │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ chunk_index      │ int32        │ Position within source document    │
+│                  │              │ 0-indexed                          │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ total_chunks     │ int32        │ Total chunks for this source file  │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ modified_at      │ string       │ File mtime, ISO 8601               │
+│                  │              │ Used for incremental sync           │
+├──────────────────┼──────────────┼─────────────────────────────────────┤
+│ indexed_at       │ string       │ When this chunk was indexed        │
+│                  │              │ ISO 8601 timestamp                 │
+└──────────────────┴──────────────┴─────────────────────────────────────┘
+
+
+
+ +
+

Indexer Bridge: TS ↔ Python

+
+
How the TypeScript plugin invokes the Python indexer
+
+
INVOCATION
+
child_process.spawn("obsidian-rag", [mode], { env: { ...config } })
+
Plugin spawns the Python CLI as a subprocess
+ +
COMMUNICATION
+
stdin: Not used
+
stdout: NDJSON progress lines (streamed to agent via status)
+
stderr: Error output (logged, surfaced on non-zero exit)
+
exit code: 0 = success, 1 = partial failure, 2 = fatal
+ +
PROGRESS EVENTS (stdout NDJSON)
+
+
{"type":"progress","phase":"embedding","current":150,"total":677}
+
{"type":"progress","phase":"storing","current":150,"total":677}
+
{"type":"complete","indexed_files":677,"total_chunks":3421,"duration_ms":45230,"errors":[]}
+
+ +
SYNC RESULT FILE
+
Written to ~/.obsidian-rag/sync-result.json on completion
+
Read by plugin on next tool call to update health state
+
Also serves as the source for status tool responses
+
+
+
+ +
+

Chunking Pipeline

+
+
From markdown file → embeddable chunks
+
+
+
+
1. SCAN
+
Walk vault dirs (respect deny/allow lists) → collect *.md files
+
+
↓
+
+
2. PARSE
+
Extract frontmatter, headings, tags, dates from filename/path
+
+
↓
+
+
3. CHUNK
+
Structured notes → split by section headers
+
Unstructured → sliding window (500 tok, 100 overlap)
+
+
↓
+
+
4. ENRICH
+
Attach metadata: source_file, section, date, tags, chunk_index
+
+
↓
+
+
5. EMBED
+
Batch chunks → Ollama /api/embed (mxbai-embed-large, 1024-dim)
+
Batch size: 64 chunks per request
+
+
↓
+
+
6. STORE
+
Write vectors + metadata to LanceDB (upsert by chunk_id)
+
+
+ +
INCREMENTAL SYNC LOGIC
+
+
1. Read last_sync_mtime from sync-result.json
+
2. Walk vault, compare each file's mtime vs last_sync_mtime
+
3. New/modified files → full chunk pipeline
+
4. Deleted files → remove chunks by source_file from LanceDB
+
5. Unchanged files → skip entirely
+
+
+
+
+ +
+

Key Design Points

+
+

Why this works

+
    +
  • Subprocess bridge is simple — no socket, no HTTP, just spawn+stdout
    • +
  • NDJSON progress lets plugin stream indexing status to agent in real-time
    • +
  • LanceDB upsert by chunk_id makes incremental sync idempotent
    • +
  • Batch embedding (64 chunks) balances throughput vs Ollama memory
    • +
  • Two chunking strategies handle both structured and unstructured notes
    • +
+
+

Trade-offs

+
    +
  • Subprocess spawn adds ~200ms per invocation (acceptable for CLI)
    • +
  • sync-result.json is a shared filesystem contract — needs careful locking
    • +
  • 500-token sliding window is configurable but not adaptive to content
    • +
+
+
+
\ No newline at end of file diff --git a/.superpowers/brainstorm/27-1775849590/content/security-layer.html b/.superpowers/brainstorm/27-1775849590/content/security-layer.html new file mode 100644 index 0000000..be8ed9e --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/content/security-layer.html @@ -0,0 +1,126 @@ +

Security & Privacy

+

Path Traversal Prevention, Input Sanitization, Sensitive Content Guards & Local-Only Enforcement

+ +
+

Security Architecture: Four Layers

+
+
Defense in depth — each layer blocks what the previous might miss
+
+
+ +
+
+
LAYER 1
+
Boundary
+
+
+
Path Traversal Prevention
+
All file reads restricted to vault_path. Reject:
+
• Any path containing ".." components
+
• Absolute paths not under vault_path
+
• Symlinks resolving outside vault_path
+
Implementation: path.resolve(vault_path, input) must start with vault_path
+
+
+ +
+
+
LAYER 2
+
Sanitize
+
+
+
Input Sanitization
+
All vault content treated as untrusted before embedding:
+
• Strip HTML tags (prevent XSS in chunk_text)
+
• Remove executable code blocks (```...```)
+
• Normalize whitespace (prevent injection via formatting)
+
• Cap chunk_text at 2000 chars (prevent oversized payloads)
+
+
+ +
+
+
LAYER 3
+
Guard
+
+
+
Sensitive Content Guard
+
Detect and gate sensitive content in search results:
+
• Health: #mentalhealth, #physicalhealth, medication, therapy
+
• Financial: owe, owed, debt, paid, $, spent
+
• Relations: #Relations (configurable section list)
+
Action: Set sensitive_detected=true in response
+
→ Agent must confirm before displaying to user
+
→ Never transmit sensitive content to external APIs
+
+
+ +
+
+
LAYER 4
+
Network
+
+
+
Local-Only Enforcement
+
All data stays on the local machine:
+
• Ollama: localhost:11434 only
+
• LanceDB: local filesystem only
+
• Config: "local_only": true enforced
+
• Network audit test: verify no outbound requests
+
If external embedding endpoint configured:
+
→ Require explicit user confirmation
+
→ Block if sensitive content detected in payload
+
+
+ +
+
+
+
+ +
+

Directory Access Control

+
+
deny_dirs and allow_dirs enforcement
+
+
DENY LIST (default)
+
.obsidian, .trash, zzz-Archive, .git
+
Always excluded — system dirs, trash, archives, VCS
+ +
ALLOW LIST (optional)
+
If set: ONLY these directories are indexed
+
If empty: all dirs except deny list are indexed
+
e.g. ["Journal", "Finance"] → only these two
+ +
FILTER LOGIC (Python indexer)
+
1. If allow_dirs is non-empty → only walk those dirs
+
2. Skip any path matching deny_dirs patterns
+
3. Skip hidden dirs (starting with .)
+ +
FILTER LOGIC (TS plugin)
+
directory_filter parameter validates against known dirs
+
Reject unknown dirs → prevent probing vault structure
+
+
+
+ +
+

Key Design Points

+
+

Why this works

+
    +
  • Four independent layers — bypass one and the others still protect
    • +
  • Sensitive content is flagged, not blocked — agent decides with user
    • +
  • Local-only is default; external APIs require explicit opt-in + confirmation
    • +
  • Directory controls applied in both Python (indexing) and TS (querying)
    • +
+
+

Trade-offs

+
    +
  • Sensitive content detection is pattern-based — may have false positives/negatives
    • +
  • Stripping code blocks loses technical notes content
    • +
  • Network audit test must be run manually — no runtime enforcement
    • +
+
+
+
\ No newline at end of file diff --git a/.superpowers/brainstorm/27-1775849590/content/session-layer.html b/.superpowers/brainstorm/27-1775849590/content/session-layer.html new file mode 100644 index 0000000..514d8d0 --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/content/session-layer.html @@ -0,0 +1,115 @@ +

Session Layer

+

Vault Watcher, Auto-Sync Scheduling & Plugin Health State Machine

+ +
+

Health State Machine

+
+
Plugin transitions through three states
+
+
+
+
HEALTHY
+
All deps up
+
→ full service
+
+
⇄
+
+
DEGRADED
+
Ollama down
+
→ stale results ok
+
+
⇄
+
+
UNAVAILABLE
+
No index / DB
+
→ cannot serve
+
+
+ +
+
HEALTHY → DEGRADED: Ollama health check fails (every 30s)
+
DEGRADED → HEALTHY: Ollama responds again
+
HEALTHY → UNAVAILABLE: Index deleted or DB corrupted
+
UNAVAILABLE → HEALTHY: Successful reindex after recovery
+
DEGRADED → UNAVAILABLE: Index corrupted while Ollama was down
+
UNAVAILABLE → DEGRADED: Index restored but Ollama still down
+
+
+
+
+ +
+

Vault Watcher: Auto-Sync Protocol

+
+
File change detection → incremental sync
+
+
DETECTION
+
chokidar watches vault_path (respects deny_dirs / allow_dirs)
+
Debounces: 2s after last change event
+
→ Avoids triggering sync for every keystroke in an editor
+ +
BATCHING
+
Collector window: 5s (configurable)
+
Groups add/update/delete events into a changeset
+
→ Single sync call for 10 files changed at once
+ +
SYNC TRIGGER
+
After debounce + collect: spawn indexer with obsidian-rag sync
+
Indexer processes only modified files (uses mtime comparison)
+
On completion: indexer writes status JSON to ~/.obsidian-rag/sync-result.json
+ +
SYNC RESULT
+
Plugin reads sync-result.json on next tool call
+
Updates health state: HEALTHY if indexed > 0 docs, UNAVAILABLE if 0
+
Stale flag: if last sync > 1h ago and vault changed, next search returns status: "degraded"
+
+
+
+ +
+

Session Lifecycle

+
+
What happens from plugin load to shutdown
+
+
+
+
ON LOAD
+
Read config → Probe Ollama → Probe LanceDB → Probe Vault → Set health state
+
+
+
REGISTER
+
Register 4 tools with OpenClaw → Start vault watcher
+
+
+
SERVE
+
Handle tool calls → Auto-sync on file changes → Health re-probe every 30s
+
+
+
ON SHUTDOWN
+
Stop vault watcher → Wait for in-flight sync → Clean exit
+
+
+
+
+
+ +
+

Key Design Points

+
+

Why this works

+
    +
  • Health state machine is simple — only 3 states, clear transitions
    • +
  • Debounce + batch prevents Ollama hammering during rapid edits
    • +
  • Stale flag on old sync lets agent warn user about freshness
    • +
  • Sync results written to filesystem — no IPC needed between indexer and plugin
    • +
+
+

Trade-offs

+
    +
  • Vault watcher uses chokidar — filesystem event reliability varies by OS
    • +
  • 30s health re-probe adds periodic Ollama load
    • +
  • Filesystem-based sync result is a polling model — up to 30s stale
    • +
+
+
+
\ No newline at end of file diff --git a/.superpowers/brainstorm/27-1775849590/content/testing-strategy.html b/.superpowers/brainstorm/27-1775849590/content/testing-strategy.html new file mode 100644 index 0000000..c991280 --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/content/testing-strategy.html @@ -0,0 +1,147 @@ +

Testing Strategy

+

Python Tests, TypeScript Tests, Integration Tests & Security Test Suites

+ +
+

Test Architecture: Four Suites

+
+
Each suite tests a distinct concern
+
+
+
PYTHON UNIT
+
pytest / python/tests/
+
+
test_chunker.py
+
→ Section splitting, sliding window, metadata
+
test_embedder.py
+
→ Mocked Ollama, batch embed, error handling
+
test_vector_store.py
+
→ LanceDB CRUD, upsert, delete by source
+
test_security.py
+
→ Path traversal, sanitization, sensitive detection
+
test_indexer.py
+
→ Full pipeline, incremental sync, config
+
+
+ +
+
TYPESCRIPT UNIT
+
vitest / tests/
+
+
search.test.ts
+
→ Parameter validation, filter logic, response shape
+
index.test.ts
+
→ Mode validation, subprocess spawn, progress parse
+
memory.test.ts
+
→ Key/value storage, auto-suggest patterns
+
vault-watcher.test.ts
+
→ Chokidar events, debounce, batching
+
security-guard.test.ts
+
→ Directory filter validation, sensitive flag
+
+
+ +
+
INTEGRATION
+
E2E / tests/integration/
+
+
full_pipeline.test.ts
+
→ Index vault → search → verify results
+
sync_cycle.test.ts
+
→ Modify file → auto-sync → search updated
+
health_state.test.ts
+
→ Stop Ollama → degraded → restart → healthy
+
openclaw_protocol.test.ts
+
→ Agent calls tools, validates envelope
+
+
+ +
+
SECURITY
+
Dedicated / tests/security/
+
+
path_traversal.test.ts
+
→ ../, symlinks, absolute, encoded paths
+
xss_prevention.test.ts
+
→ HTML/script injection in chunk text
+
prompt_injection.test.ts
+
→ Malicious content in vault notes
+
network_audit.test.ts
+
→ Verify zero outbound when local_only=true
+
sensitive_content.test.ts
+
→ Pattern detection, flagging, blocking
+
+
+
+
+
+ +
+

Testing Approach by Layer

+
+
What gets mocked vs what gets hit for real
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ComponentUnit TestIntegration Test
Ollama embeddingMocked — fixed 1024-dim vectorsReal — requires Ollama running
LanceDBReal — temp directory, cleaned upReal — temp directory, cleaned up
Obsidian vaultMocked — fixture markdown filesReal — temp vault with real files
Python CLIMocked subprocessReal — actual CLI invocation
Chokidar watcherMocked eventsReal — actual file system events
OpenClaw agentN/AReal — tool call envelope validation
+
+
+
+ +
+

Key Design Points

+
+

Why this works

+
    +
  • Four suites cover all four security layers + both languages
    • +
  • LanceDB uses real temp dirs — no mock drift for vector operations
    • +
  • Integration tests require Ollama but skip gracefully if unavailable
    • +
  • Security suite is standalone — always runs, no dependencies
    • +
+
+

Trade-offs

+
    +
  • Integration tests need Ollama installed locally
    • +
  • File watcher integration tests are timing-dependent (debounce/batch)
    • +
  • Sensitive content detection is pattern-based — test coverage is only as good as patterns
    • +
+
+
+
\ No newline at end of file diff --git a/.superpowers/brainstorm/27-1775849590/content/tool-layer.html b/.superpowers/brainstorm/27-1775849590/content/tool-layer.html new file mode 100644 index 0000000..1743638 --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/content/tool-layer.html @@ -0,0 +1,170 @@ +

Tool Layer

+

Four Plugin Tools, Their Contracts & the OpenClaw Interaction Protocol

+ +
+

Tool 1: obsidian_rag_search

+
+
Primary semantic search tool
+
+
+
PARAMETERS
+
query (required) string
+
Natural language question
+
max_results (opt) int, default 5
+
Max chunks to return (1-50)
+
directory_filter (opt) string[]
+
Limit to subdirs e.g. ["Journal"]
+
date_range (opt) {from, to}
+
ISO 8601 dates
+
tags (opt) string[]
+
Filter by hashtags
+
+
+
RESPONSE data
+
results: [{
+
chunk_text: string
+
score: number (0-1)
+
source_file: string
+
section: string | null
+
date: string | null
+
tags: string[]
+
chunk_index: number
+
}]
+
sensitive_detected: boolean
+
If true, agent should confirm before display
+
+
+
+
+ +
+

Tool 2: obsidian_rag_index

+
+
Trigger indexing from within OpenClaw
+
+
+
PARAMETERS
+
mode (required) enum
+
"full" | "sync" | "reindex"
+
full: initial index of entire vault
+
sync: incremental (mtime diff only)
+
reindex: nuke + rebuild from scratch
+
+
+
RESPONSE data
+
indexed_files: number
+
total_chunks: number
+
duration_ms: number
+
errors: [{file, message}]
+
Long-running: returns immediately with
+
job_id, agent polls via status tool
+
+
+
+
+ +
+

Tool 3: obsidian_rag_status

+
+
Index health and sync state
+
+
+
PARAMETERS
+
(none — always returns current state)
+
+
+
RESPONSE data
+
plugin_health: "healthy"|"degraded"|"unavailable"
+
total_docs: number
+
total_chunks: number
+
last_sync: string (ISO 8601)
+
unindexed_files: number
+
ollama_status: "up" | "down"
+
active_job: {id, mode, progress} | null
+
+
+
+
+ +
+

Tool 4: obsidian_rag_memory_store

+
+
Commit facts to OpenClaw memory for faster retrieval
+
+
+
PARAMETERS
+
key (required) string
+
Identifier for the fact
+
value (required) string
+
The fact to remember
+
source (required) string
+
Source file path in vault
+
+
+
RESPONSE data
+
stored: boolean
+
key: string (echoed back)
+
Auto-suggest: When search results
+
match financial/health/commitment
+
patterns, plugin flags
+
sensitive_detected=true and includes
+
memory_suggestion with key/value
+
+
+
+
+ +
+

OpenClaw Interaction Protocol

+
+
How the agent decides which tool to use and when
+
+
INTENT → TOOL MAPPING
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
User Intent PatternToolAgent Behavior
"What did I write about X?"
"How was my Y in Z?"		searchQuery vault, synthesize answer from chunks
"Index my vault"
"Update the search index"		indexChoose mode (sync default, full on no index), poll status
"Is the index up to date?"
"How many docs are indexed?"		statusReport health, suggest index if stale
"Remember this"
Auto-suggest after search		memory_storeConfirm with user, store key+value+source
+ +
PROTOCOL SEQUENCE
+
+
1. Agent receives user query with vault-related intent
+
2. Agent calls obsidian_rag_status (if first interaction or >5min since last check)
+
→ Skips if status already cached and healthy
+
3. If unavailable → agent suggests running obsidian_rag_index (mode: "full")
+
4. If degraded → agent warns user but proceeds with search
+
5. Agent calls obsidian_rag_search with appropriate filters
+
6. If sensitive_detected=true → agent confirms before displaying
+
7. Agent synthesizes answer from chunks + existing knowledge
+
8. If memory_suggestion present → agent asks if user wants to store
+
→ User confirms → obsidian_rag_memory_store
+
+
+
+
\ No newline at end of file diff --git a/.superpowers/brainstorm/27-1775849590/content/transport-layer.html b/.superpowers/brainstorm/27-1775849590/content/transport-layer.html new file mode 100644 index 0000000..da8e85a --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/content/transport-layer.html @@ -0,0 +1,130 @@ +

Transport Layer

+

Tool Registration, Request/Response Contracts & Error Normalization

+ +
+

Tool Registration Flow

+
+
Plugin Lifecycle: Install → Register → Serve
+
+
1. INSTALL
+
openclaw plugins install clawhub:obsidian-rag
+
→ Downloads plugin, reads openclaw.plugin.json
+ +
2. REGISTER
+
Plugin.register(tools) → OpenClaw Tool Registry
+
→ 4 tools registered: search, index, status, memory_store
+
→ Each tool declares: name, description, parameterSchema, requiredPermissions
+ +
3. SERVE
+
OpenClaw Agent → tool_call(obsidian_rag_search, {query: "..."}) → Plugin
+
Plugin → structured response → OpenClaw Agent
+ +
4. HEALTH CHECK
+
Plugin.onLoad() → probe Ollama, probe LanceDB, probe Vault
+
→ Reports status: healthy | degraded | unavailable
+
+
+
+ +
+

Standardized Response Envelope

+
+
All tool responses use this envelope
+
+
{
+  "status": "healthy" | "degraded" | "unavailable",
+  "data": T | null,
+  "error": {
+    "code": string,       // e.g. "OLLAMA_UNREACHABLE"
+    "message": string,    // human-readable
+    "recoverable": bool,  // can the agent retry?
+    "suggestion": string  // e.g. "Run obsidian-rag index first"
+  } | null,
+  "meta": {
+    "query_time_ms": number,
+    "chunks_scanned": number,
+    "index_version": string,
+    "vault_mtime": string   // ISO 8601
+  }
+}
+
+
+
+ +
+

Error Normalization Matrix

+
+
How each failure maps to status + error
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FailureStatusCodeRecoverable
Ollama downdegradedOLLAMA_UNREACHABLEyes — retry after start
Empty vault / no indexunavailableINDEX_NOT_FOUNDyes — run index first
LanceDB corruptedunavailableINDEX_CORRUPTEDyes — run reindex
Path traversal attemptunavailableSECURITY_VIOLATIONno
Sensitive content blockeddegradedSENSITIVE_FILTEREDyes — with confirmation
Indexer CLI crashunavailableINDEXER_FAILEDyes — retry once
+
+
+
+ +
+

Key Design Points

+
+

Why this works

+
    +
  • OpenClaw agent never sees raw exceptions — all errors normalized
    • +
  • Three-state health (healthy/degraded/unavailable) lets agent adapt behavior
    • +
  • Recoverable flag lets agent decide whether to retry or inform user
    • +
  • Suggestion field gives agent context to offer remediation
    • +
+
+

Trade-offs

+
    +
  • Envelope adds ~200 bytes per response (negligible)
    • +
  • Agent must handle three status states, not just success/fail
    • +
  • Health check on load adds ~1s startup latency
    • +
+
+
+
\ No newline at end of file diff --git a/.superpowers/brainstorm/27-1775849590/state/server-stopped b/.superpowers/brainstorm/27-1775849590/state/server-stopped new file mode 100644 index 0000000..6af0f08 --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/state/server-stopped @@ -0,0 +1 @@ +{"reason":"idle timeout","timestamp":1775855591203} diff --git a/.superpowers/brainstorm/27-1775849590/state/server.pid b/.superpowers/brainstorm/27-1775849590/state/server.pid new file mode 100644 index 0000000..f64f5d8 --- /dev/null +++ b/.superpowers/brainstorm/27-1775849590/state/server.pid @@ -0,0 +1 @@ +27