p19-bug-fixes
This commit is contained in:
185
e2e/docs/test-strategy.md
Normal file
185
e2e/docs/test-strategy.md
Normal file
@@ -0,0 +1,185 @@
|
||||
# ClawFort UI/UX Test Strategy
|
||||
|
||||
## Capability-to-Test Mapping
|
||||
|
||||
This document maps OpenSpec capabilities to Playwright test coverage.
|
||||
|
||||
### New Capabilities
|
||||
|
||||
#### `playwright-ui-ux-regression-suite`
|
||||
| Requirement | Test File | Scenarios |
|
||||
|-------------|-----------|-----------|
|
||||
| Capability-mapped suite execution | All files | Tests organized by capability in directory structure |
|
||||
| Theme and breakpoint matrix coverage | `responsive/*.spec.ts`, `accessibility/*.spec.ts` | Cross-theme and cross-viewport test execution |
|
||||
| Failure artifact collection | `playwright.config.ts` | Trace, screenshot, video on failure enabled |
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
#### `end-to-end-system-testing`
|
||||
| Requirement | Test File | Scenarios |
|
||||
|-------------|-----------|-----------|
|
||||
| Core user flow E2E | `core-journeys/hero-feed.spec.ts` | Hero loading, feed browsing, modal flows |
|
||||
| Browser-native interaction E2E | `modal-experience/*.spec.ts` | Modal open/close, source links, share actions |
|
||||
| Edge-case workflows | `core-journeys/edge-cases.spec.ts` | Empty data, invalid permalinks, error states |
|
||||
| UI failure-path resilience | `accessibility/modal.spec.ts` | Fallback messages, navigable error states |
|
||||
|
||||
#### `platform-quality-gates`
|
||||
| Requirement | Test File | Scenarios |
|
||||
|-------------|-----------|-----------|
|
||||
| Release quality gates | CI workflow | Smoke profile gates PR, full profile gates main |
|
||||
| Playwright gate failure blocks release | CI workflow | Fail-on-regression policy enforced |
|
||||
| Gate manifest | `playwright.config.ts` | Explicit browser/tool versions |
|
||||
| Gate profiles documented | This file | Smoke vs full profile criteria defined |
|
||||
|
||||
#### `wcag-2-2-aa-accessibility`
|
||||
| Requirement | Test File | Scenarios |
|
||||
|-------------|-----------|-----------|
|
||||
| Keyboard-only interaction flow | `accessibility/keyboard.spec.ts` | Modal navigation, icon-only controls, focus visibility |
|
||||
| Contrast and non-text alternatives | `accessibility/contrast.spec.ts` | Color contrast assertions across themes |
|
||||
| Accessibility CI gate | CI workflow | Automated accessibility checks in pipeline |
|
||||
| Interactive accessibility states | `accessibility/states.spec.ts` | Focus-visible, keyboard traversal, accessible names |
|
||||
|
||||
#### `responsive-device-agnostic-layout`
|
||||
| Requirement | Test File | Scenarios |
|
||||
|-------------|-----------|-----------|
|
||||
| Mobile layout behavior | `responsive/breakpoints.spec.ts` | No overflow, reachable controls |
|
||||
| Desktop and tablet adaptation | `responsive/breakpoints.spec.ts` | Layout reflow, no clipping |
|
||||
| Sticky shrinking glass header | `responsive/sticky.spec.ts` | Header behavior across scroll |
|
||||
| Sticky footer overlap | `responsive/sticky.spec.ts` | Content readability, control accessibility |
|
||||
| Breakpoint regression matrix | `responsive/*.spec.ts` | Overflow/clipping detection across breakpoints |
|
||||
|
||||
#### `summary-modal-experience`
|
||||
| Requirement | Test File | Scenarios |
|
||||
|-------------|-----------|-----------|
|
||||
| Open summary modal | `modal-experience/summary.spec.ts` | Modal opens with correct content order |
|
||||
| Close summary modal | `modal-experience/summary.spec.ts` | Modal dismisses, returns to feed |
|
||||
| Permalink-driven modal open | `modal-experience/deep-link.spec.ts` | Direct article URL opens modal |
|
||||
| Keyboard dismissal and focus continuity | `modal-experience/summary.spec.ts` | Escape closes, focus returns |
|
||||
| Source link-out from modal | `modal-experience/summary.spec.ts` | Source opens in new tab |
|
||||
| Modal exposes share entry points | `modal-experience/summary.spec.ts` | Share controls available |
|
||||
| Modal interaction regression coverage | `modal-experience/*.spec.ts` | All entry paths tested |
|
||||
|
||||
#### `share-and-contact-microinteractions`
|
||||
| Requirement | Test File | Scenarios |
|
||||
|-------------|-----------|-----------|
|
||||
| Supported share providers | `microinteractions/share.spec.ts` | X, WhatsApp, LinkedIn icons present |
|
||||
| Light-theme icon visibility | `microinteractions/share.spec.ts` | Contrast, keyboard focusability |
|
||||
| Copy-link share action | `microinteractions/share.spec.ts` | Clipboard write, no navigation |
|
||||
| Share controls state accessibility | `microinteractions/share.spec.ts` | States perceivable across themes |
|
||||
| Config present/absent footer links | `microinteractions/footer.spec.ts` | GitHub/contact conditional rendering |
|
||||
| Contact link visible when configured | `microinteractions/footer.spec.ts` | CONTACT_EMAIL shows affordance |
|
||||
| Randomized helper tooltip | `microinteractions/tooltip.spec.ts` | Hover shows safe message |
|
||||
| Keyboard-triggered helper tooltip | `microinteractions/tooltip.spec.ts` | Focus shows tooltip, blur dismisses |
|
||||
|
||||
## Scenario Taxonomy
|
||||
|
||||
### Journey Scenarios
|
||||
- Hero article loading and display
|
||||
- Feed browsing and pagination
|
||||
- Modal open/close from hero
|
||||
- Modal open/close from feed
|
||||
- Source link navigation
|
||||
- Deep-link direct modal open
|
||||
|
||||
### Accessibility-State Scenarios
|
||||
- Keyboard navigation through all interactive elements
|
||||
- Focus-visible indicators on all controls
|
||||
- Color contrast for text and UI components
|
||||
- Accessible names for icon-only controls
|
||||
- Screen reader compatibility for dynamic content
|
||||
|
||||
### Responsive Scenarios
|
||||
- Mobile viewport (375x667): No horizontal overflow, touch targets sized
|
||||
- Tablet viewport (768x1024): Layout reflow, readable content
|
||||
- Desktop viewport (1280x720): Full layout, all features accessible
|
||||
- Widescreen viewport (1920x1080): Max-width constraints
|
||||
|
||||
### Modal Scenarios
|
||||
- Summary modal: Open from hero, open from feed, open from permalink
|
||||
- Policy modals: Terms, Attribution, open/close, escape key
|
||||
- Focus containment within modals
|
||||
- Focus return on modal close
|
||||
|
||||
### Microinteraction Scenarios
|
||||
- Share button hover/focus states
|
||||
- Copy link success feedback
|
||||
- Contact tooltip hover/move/leave
|
||||
- Contact tooltip keyboard focus/blur
|
||||
- Back-to-top visibility on scroll
|
||||
- Theme switch animation
|
||||
|
||||
### Deep-Link Scenarios
|
||||
- Valid article permalink opens modal
|
||||
- Invalid article permalink shows error state
|
||||
- Policy permalink opens policy modal
|
||||
- Hash-based navigation
|
||||
|
||||
## Execution Profiles
|
||||
|
||||
### Smoke Profile
|
||||
**Trigger:** Pull request validation
|
||||
**Duration:** ~2-3 minutes
|
||||
**Browsers:** Chromium only
|
||||
**Coverage:**
|
||||
- Hero loads with content
|
||||
- Feed displays articles
|
||||
- Modal opens and closes
|
||||
- Basic keyboard navigation
|
||||
- One theme (dark)
|
||||
- One viewport (desktop)
|
||||
|
||||
### Full Profile
|
||||
**Trigger:** Main branch merge, nightly builds
|
||||
**Duration:** ~15-20 minutes
|
||||
**Browsers:** Chrome, Firefox, Safari
|
||||
**Coverage:**
|
||||
- All journey scenarios
|
||||
- All accessibility scenarios across themes
|
||||
- All responsive breakpoints
|
||||
- All modal interaction paths
|
||||
- All microinteraction states
|
||||
- Cross-browser compatibility
|
||||
|
||||
## CI Integration
|
||||
|
||||
### Pull Request Quality Gate
|
||||
```yaml
|
||||
- name: Playwright Smoke Tests
|
||||
run: npm run test:smoke
|
||||
env:
|
||||
BASE_URL: http://localhost:8000
|
||||
continue-on-error: false
|
||||
```
|
||||
|
||||
### Main/Nightly Pipeline
|
||||
```yaml
|
||||
- name: Playwright Full Regression
|
||||
run: npm run test:full
|
||||
env:
|
||||
BASE_URL: http://localhost:8000
|
||||
continue-on-error: false
|
||||
|
||||
- name: Upload Test Artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
if: failure()
|
||||
with:
|
||||
name: playwright-report
|
||||
path: e2e/playwright-report/
|
||||
retention-days: 30
|
||||
```
|
||||
|
||||
## Test Data Assumptions
|
||||
|
||||
Tests assume:
|
||||
1. Application serves at least one article in hero position
|
||||
2. Feed contains multiple articles for pagination testing
|
||||
3. Backend API responds within 5 seconds
|
||||
4. Images load within 10 seconds (or placeholder shown)
|
||||
5. No authentication required for public content
|
||||
|
||||
## Failure Triage
|
||||
|
||||
1. **Smoke test failures:** Block PR merge, immediate investigation required
|
||||
2. **Full regression failures:** Create issue, assign to UI owner, fix before release
|
||||
3. **Flaky tests:** Quarantine after 3 consecutive failures, investigate separately
|
||||
4. **Browser-specific failures:** Check for polyfill or feature support issues
|
||||
156
e2e/docs/triage-workflow.md
Normal file
156
e2e/docs/triage-workflow.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# UI/UX Regression Test Triage Workflow
|
||||
|
||||
## Overview
|
||||
|
||||
This document defines the triage workflow and ownership for UI/UX regression test failures in the ClawFort Playwright test suite.
|
||||
|
||||
## Test Failure Severity
|
||||
|
||||
### Critical (Block Release)
|
||||
- Smoke test failures in PR gate
|
||||
- Core journey failures (hero/feed loading, modal open/close)
|
||||
- Accessibility violations (keyboard navigation, focus management)
|
||||
- Cross-browser compatibility issues
|
||||
|
||||
### High (Fix Before Release)
|
||||
- Full regression failures in main/nightly
|
||||
- Responsive layout issues on supported breakpoints
|
||||
- Theme-specific rendering problems
|
||||
- Deep-link functionality failures
|
||||
|
||||
### Medium (Fix in Next Sprint)
|
||||
- Minor visual inconsistencies
|
||||
- Non-critical microinteraction issues
|
||||
- Performance degradation in test execution
|
||||
|
||||
### Low (Backlog)
|
||||
- Cosmetic issues not affecting functionality
|
||||
- Test flakiness requiring investigation
|
||||
- Documentation gaps
|
||||
|
||||
## Ownership
|
||||
|
||||
### Primary Owner
|
||||
- **UI/UX Team Lead**: Overall test suite health and strategy
|
||||
|
||||
### Component Owners
|
||||
- **Core Journeys**: Frontend team
|
||||
- **Accessibility**: Accessibility specialist + Frontend team
|
||||
- **Responsive Design**: Frontend team + UX designer
|
||||
- **Modal Experience**: Frontend team
|
||||
- **Microinteractions**: Frontend team
|
||||
|
||||
### CI/CD Integration
|
||||
- **DevOps Team**: CI pipeline configuration and artifact management
|
||||
|
||||
## Triage Process
|
||||
|
||||
### Step 1: Detection (Automated)
|
||||
1. CI pipeline runs tests on PR or main branch
|
||||
2. Failures are reported in GitHub Actions
|
||||
3. Artifacts (screenshots, videos, traces) are uploaded
|
||||
4. Notifications sent to #ui-regression-alerts channel
|
||||
|
||||
### Step 2: Initial Assessment (Within 2 hours)
|
||||
1. Check if failure is reproducible locally
|
||||
2. Review failure artifacts in Playwright report
|
||||
3. Determine severity based on impact
|
||||
4. Assign to appropriate component owner
|
||||
|
||||
### Step 3: Investigation
|
||||
1. Review test logs and artifacts
|
||||
2. Check recent commits for related changes
|
||||
3. Attempt local reproduction
|
||||
4. Document findings in issue
|
||||
|
||||
### Step 4: Resolution
|
||||
1. **Critical**: Fix immediately, re-run tests
|
||||
2. **High**: Fix within 24 hours
|
||||
3. **Medium**: Schedule for next sprint
|
||||
4. **Low**: Add to backlog with priority
|
||||
|
||||
### Step 5: Verification
|
||||
1. Re-run failing tests
|
||||
2. Verify fix doesn't introduce new issues
|
||||
3. Update test if it was a false positive
|
||||
4. Document resolution
|
||||
|
||||
## Artifact Access
|
||||
|
||||
### Playwright Report
|
||||
- **Location**: GitHub Actions artifacts
|
||||
- **Retention**: 14 days (smoke), 30 days (full)
|
||||
- **Access**: Download from workflow run page
|
||||
|
||||
### Viewing Locally
|
||||
```bash
|
||||
# Download and extract artifact
|
||||
cd e2e
|
||||
npx playwright show-report playwright-report/
|
||||
```
|
||||
|
||||
### Key Artifacts
|
||||
- **trace.zip**: Full trace with DOM snapshots, network, console
|
||||
- **test-failed-*.png**: Screenshot at failure point
|
||||
- **video.webm**: Video recording of test execution
|
||||
|
||||
## Common Failure Patterns
|
||||
|
||||
### Flaky Tests
|
||||
- **Symptom**: Passes on retry
|
||||
- **Action**: Increase timeouts, add waits, stabilize selectors
|
||||
- **Owner**: Test maintainer
|
||||
|
||||
### Environment Issues
|
||||
- **Symptom**: Tests pass locally but fail in CI
|
||||
- **Action**: Check CI environment, browser versions, dependencies
|
||||
- **Owner**: DevOps + Test maintainer
|
||||
|
||||
### Application Regressions
|
||||
- **Symptom**: Consistent failures across runs
|
||||
- **Action**: Identify breaking change, fix application code
|
||||
- **Owner**: Component owner
|
||||
|
||||
### Test Data Issues
|
||||
- **Symptom**: Tests fail due to missing/changed data
|
||||
- **Action**: Update test fixtures, ensure deterministic data
|
||||
- **Owner**: Test maintainer
|
||||
|
||||
## Communication
|
||||
|
||||
### Slack Channels
|
||||
- `#ui-regression-alerts`: Automated failure notifications
|
||||
- `#frontend-team`: Discussion of UI issues
|
||||
- `#qa-team`: Test-related discussions
|
||||
|
||||
### Issue Labels
|
||||
- `ui-regression`: UI/UX test failures
|
||||
- `accessibility`: WCAG-related issues
|
||||
- `responsive`: Layout/breakpoint issues
|
||||
- `flaky-test`: Intermittent failures
|
||||
- `ci-blocker`: Blocking CI/CD pipeline
|
||||
|
||||
## Escalation Path
|
||||
|
||||
1. **Component Owner** investigates and attempts fix
|
||||
2. **UI/UX Team Lead** reviews if not resolved in 4 hours
|
||||
3. **Engineering Manager** escalates if blocking release
|
||||
4. **CTO** involved for critical production issues
|
||||
|
||||
## Prevention
|
||||
|
||||
### Pre-merge Checks
|
||||
- Run smoke tests locally before pushing
|
||||
- Review visual changes with design team
|
||||
- Test across themes and breakpoints for UI changes
|
||||
|
||||
### Monitoring
|
||||
- Weekly review of test flakiness metrics
|
||||
- Monthly review of test coverage
|
||||
- Quarterly review of test strategy
|
||||
|
||||
## Contact
|
||||
|
||||
- **UI/UX Test Suite**: ui-team@clawfort.ai
|
||||
- **CI/CD Issues**: devops@clawfort.ai
|
||||
- **Emergency Escalation**: engineering-manager@clawfort.ai
|
||||
Reference in New Issue
Block a user