santhoshj/obsidian-rag
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

all in

Browse Source
This commit is contained in:
Santhosh Janardhanan
2026-04-10 19:00:38 -04:00
parent b8996d2ecb
commit 2c976bb75b
10 changed files with 957 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

147
.superpowers/brainstorm/27-1775849590/content/testing-strategy.html Normal file
View File

@@ -0,0 +1,147 @@
<h2>Testing Strategy</h2>
<p class="subtitle">Python Tests, TypeScript Tests, Integration Tests & Security Test Suites</p>
<div class="section">
<h3>Test Architecture: Four Suites</h3>
<div class="mockup">
<div class="mockup-header">Each suite tests a distinct concern</div>
<div class="mockup-body" style="font-family: monospace; font-size: 12px; padding: 16px; display: flex; gap: 16px; flex-wrap: wrap;">
<div style="flex: 1; min-width: 220px; background: #1a472a; border: 1px solid #2ecc71; border-radius: 8px; padding: 12px;">
<div style="color: #2ecc71; font-weight: bold; margin-bottom: 8px;">PYTHON UNIT</div>
<div style="color: #aaa; font-size: 10px; margin-bottom: 6px;">pytest / python/tests/</div>
<div style="color: #eee; line-height: 1.6;">
<div>test_chunker.py</div>
<div style="color: #888; margin-left: 12px;">→ Section splitting, sliding window, metadata</div>
<div>test_embedder.py</div>
<div style="color: #888; margin-left: 12px;">→ Mocked Ollama, batch embed, error handling</div>
<div>test_vector_store.py</div>
<div style="color: #888; margin-left: 12px;">→ LanceDB CRUD, upsert, delete by source</div>
<div>test_security.py</div>
<div style="color: #888; margin-left: 12px;">→ Path traversal, sanitization, sensitive detection</div>
<div>test_indexer.py</div>
<div style="color: #888; margin-left: 12px;">→ Full pipeline, incremental sync, config</div>
</div>
</div>
<div style="flex: 1; min-width: 220px; background: #1a3a5c; border: 1px solid #53a8b6; border-radius: 8px; padding: 12px;">
<div style="color: #53a8b6; font-weight: bold; margin-bottom: 8px;">TYPESCRIPT UNIT</div>
<div style="color: #aaa; font-size: 10px; margin-bottom: 6px;">vitest / tests/</div>
<div style="color: #eee; line-height: 1.6;">
<div>search.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Parameter validation, filter logic, response shape</div>
<div>index.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Mode validation, subprocess spawn, progress parse</div>
<div>memory.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Key/value storage, auto-suggest patterns</div>
<div>vault-watcher.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Chokidar events, debounce, batching</div>
<div>security-guard.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Directory filter validation, sensitive flag</div>
</div>
</div>
<div style="flex: 1; min-width: 220px; background: #3d3200; border: 1px solid #f0a500; border-radius: 8px; padding: 12px;">
<div style="color: #f0a500; font-weight: bold; margin-bottom: 8px;">INTEGRATION</div>
<div style="color: #aaa; font-size: 10px; margin-bottom: 6px;">E2E / tests/integration/</div>
<div style="color: #eee; line-height: 1.6;">
<div>full_pipeline.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Index vault → search → verify results</div>
<div>sync_cycle.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Modify file → auto-sync → search updated</div>
<div>health_state.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Stop Ollama → degraded → restart → healthy</div>
<div>openclaw_protocol.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Agent calls tools, validates envelope</div>
</div>
</div>
<div style="flex: 1; min-width: 220px; background: #3d0000; border: 1px solid #e94560; border-radius: 8px; padding: 12px;">
<div style="color: #e94560; font-weight: bold; margin-bottom: 8px;">SECURITY</div>
<div style="color: #aaa; font-size: 10px; margin-bottom: 6px;">Dedicated / tests/security/</div>
<div style="color: #eee; line-height: 1.6;">
<div>path_traversal.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ ../, symlinks, absolute, encoded paths</div>
<div>xss_prevention.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ HTML/script injection in chunk text</div>
<div>prompt_injection.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Malicious content in vault notes</div>
<div>network_audit.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Verify zero outbound when local_only=true</div>
<div>sensitive_content.test.ts</div>
<div style="color: #888; margin-left: 12px;">→ Pattern detection, flagging, blocking</div>
</div>
</div>
</div>
</div>
</div>
<div class="section" style="margin-top: 20px;">
<h3>Testing Approach by Layer</h3>
<div class="mockup">
<div class="mockup-header">What gets mocked vs what gets hit for real</div>
<div class="mockup-body" style="font-family: monospace; font-size: 12px; padding: 16px;">
<table style="width: 100%; border-collapse: collapse; color: #eee;">
<thead>
<tr style="border-bottom: 2px solid #53a8b6;">
<th style="text-align: left; padding: 6px;">Component</th>
<th style="text-align: left; padding: 6px;">Unit Test</th>
<th style="text-align: left; padding: 6px;">Integration Test</th>
</tr>
</thead>
<tbody>
<tr style="border-bottom: 1px solid #333;">
<td style="padding: 6px;">Ollama embedding</td>
<td style="padding: 6px; color: #f0a500;">Mocked — fixed 1024-dim vectors</td>
<td style="padding: 6px; color: #2ecc71;">Real — requires Ollama running</td>
</tr>
<tr style="border-bottom: 1px solid #333;">
<td style="padding: 6px;">LanceDB</td>
<td style="padding: 6px; color: #2ecc71;">Real — temp directory, cleaned up</td>
<td style="padding: 6px; color: #2ecc71;">Real — temp directory, cleaned up</td>
</tr>
<tr style="border-bottom: 1px solid #333;">
<td style="padding: 6px;">Obsidian vault</td>
<td style="padding: 6px; color: #f0a500;">Mocked — fixture markdown files</td>
<td style="padding: 6px; color: #2ecc71;">Real — temp vault with real files</td>
</tr>
<tr style="border-bottom: 1px solid #333;">
<td style="padding: 6px;">Python CLI</td>
<td style="padding: 6px; color: #f0a500;">Mocked subprocess</td>
<td style="padding: 6px; color: #2ecc71;">Real — actual CLI invocation</td>
</tr>
<tr style="border-bottom: 1px solid #333;">
<td style="padding: 6px;">Chokidar watcher</td>
<td style="padding: 6px; color: #f0a500;">Mocked events</td>
<td style="padding: 6px; color: #2ecc71;">Real — actual file system events</td>
</tr>
<tr>
<td style="padding: 6px;">OpenClaw agent</td>
<td style="padding: 6px; color: #888;">N/A</td>
<td style="padding: 6px; color: #2ecc71;">Real — tool call envelope validation</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div class="section" style="margin-top: 20px;">
<h3>Key Design Points</h3>
<div class="pros-cons">
<div class="pros"><h4>Why this works</h4>
<ul>
<li>Four suites cover all four security layers + both languages</li>
<li>LanceDB uses real temp dirs — no mock drift for vector operations</li>
<li>Integration tests require Ollama but skip gracefully if unavailable</li>
<li>Security suite is standalone — always runs, no dependencies</li>
</ul>
</div>
<div class="cons"><h4>Trade-offs</h4>
<ul>
<li>Integration tests need Ollama installed locally</li>
<li>File watcher integration tests are timing-dependent (debounce/batch)</li>
<li>Sensitive content detection is pattern-based — test coverage is only as good as patterns</li>
</ul>
</div>
</div>
</div>