video-gen/TODO.MD

TODO

Repo bootstrap

• Define project direction and constraints in agents.md
• Add requirements.txt (pip dependencies)
• Add environment.yml (conda environment; PyTorch CUDA 12.x runtime strategy)
• Add storyboard JSON template at templates/storyboard.template.json

Core implementation (next)

• Create repo structure: src/, tests/, docs/, templates/, outputs/
• Implement storyboard schema validator (pydantic) + loader
• Implement prompt compiler (global style + shot + camera)
• Implement shot planning (duration -> frame count, chunk plan)
• Implement model backend interface (BaseVideoBackend)
• Implement WAN backend (primary) with VRAM-safe defaults
• Implement fallback backend (SVD) for reliability testing
• Implement ffmpeg assembler (concat + optional audio + debug burn-in)
• Implement optional upscaling module (post-process)

Utilities

• Write storyboard "plain text -> JSON" utility script (fills storyboard.template.json)
• Add config file support (YAML/JSON) for global defaults

Testing (parallel work; required)

• Add pytest scaffolding
• Add tests for schema validation
• Add tests for prompt compilation determinism
• Add tests for shot planning (frames/chunks)
• Add tests for ffmpeg command generation (no actual render needed)
• Ensure every code change includes a corresponding test update

Documentation (maintained continuously)

• Create docs/developer.md (install, architecture, tests, adding backends)
• Create docs/user.md (quickstart, storyboard creation, running, outputs, troubleshooting)
• Keep docs updated whenever CLI/config/schema changes

Current Status

• Completed: 12/19 tasks
• In Progress: FFmpeg assembler implementation
• Next: CLI entry point, Documentation

Recent Updates

• Fixed environment.yml for PyTorch 2.5.1 compatibility
• Implemented WAN backend with lazy imports
• Created FFmpeg assembler module
• All core tests passing (29 tests)