konstruct/.planning/phases/09-testing-qa/09-02-SUMMARY.md

phase, plan, subsystem, tags, requires, provides, affects, tech-stack, key-files, key-decisions, duration, completed

phase	plan	subsystem	tags	requires	provides	affects	tech-stack	key-files	key-decisions	duration	completed
09-testing-qa	02	testing	playwright visual-regression axe-core a11y lighthouse wcag snapshots keyboard-nav	phase plan provides 09-testing-qa 01 playwright.config.ts (visual/a11y projects), fixtures.ts (axe fixture), auth.setup.ts (storageState)	Visual regression spec 6 pages at 3 viewports (desktop/tablet/mobile) Accessibility scan spec 8 pages with critical-violation gating, serious logged as warnings Keyboard navigation tests for login form and chat input Lighthouse CI config with 0.80 hard floor on all 4 categories for /login	CI pipeline (09-03 Gitea Actions) QA baseline before beta launch	added patterns Visual regression via playwright visual-desktop/visual-tablet/visual-mobile projects — spec runs once, projects vary viewport axe fixture from fixtures.ts — returns () => AxeBuilder scoped to wcag2a/wcag2aa/wcag21aa Critical-only gating — critical violations fail the test, serious logged as console.warn Lighthouse CI desktop preset — /login only (authenticated pages redirect unauthenticated)	created modified packages/portal/e2e/visual/snapshots.spec.ts packages/portal/e2e/accessibility/a11y.spec.ts packages/portal/e2e/lighthouse/lighthouserc.json	lighthouserc.json uses error (not warn) at minScore 0.80 for all 4 categories — plan hard floor requirement preset: desktop in lighthouserc — more representative of actual usage than mobile emulation a11y.spec.ts not axe.spec.ts — a11y.spec.ts uses the correct axe fixture; axe.spec.ts had wrong fixture name (makeAxeBuilder) causing TypeScript errors Serious a11y violations are warnings not blockers — balances correctness with pragmatism for beta launch Visual baselines require running stack — committed specs only, baselines generated on first --update-snapshots run	~1min	2026-03-26

Phase 9 Plan 02: Visual Regression, Accessibility, and Lighthouse CI Summary

Visual regression snapshots at 3 viewports, axe-core WCAG 2.1 AA scanning on 8 pages, and Lighthouse CI with 0.80 hard floor on all 4 categories — QA baseline before beta launch

Performance

• Duration: ~1 min
• Started: 2026-03-26
• Completed: 2026-03-26
• Tasks: 2
• Files modified: 3

Accomplishments

• snapshots.spec.ts: 6 key pages (login, dashboard, agents list, agents/new, chat, templates) each captured via 3 viewport projects — 18 total visual test runs
• a11y.spec.ts: 8 pages scanned with axe-core, critical violations are hard failures, serious violations logged as console.warn but pass; 2 keyboard navigation tests (login form tab order, chat message input focus)
• lighthouserc.json: Lighthouse CI targeting /login only (authenticated pages redirect when unauthenticated), desktop preset, all 4 score categories at "error" level with 0.80 minimum
• Removed pre-existing axe.spec.ts which had TypeScript errors (wrong fixture name makeAxeBuilder — fixture is axe)

Task Commits

Both tasks landed in a single atomic commit due to lighthouserc.json being pre-staged from a prior session:

1. Task 1 + Task 2: Visual regression + a11y + Lighthouse CI — 7566ae4 (feat)
   • e2e/visual/snapshots.spec.ts — 6-page visual snapshot spec
   • e2e/accessibility/a11y.spec.ts — 8-page axe-core scan + 2 keyboard nav tests
   • e2e/lighthouse/lighthouserc.json — Lighthouse CI config, 0.80 hard floor all categories

Files Created/Modified

• packages/portal/e2e/visual/snapshots.spec.ts — Visual regression spec: 6 pages, toHaveScreenshot, imports from ../fixtures
• packages/portal/e2e/accessibility/a11y.spec.ts — axe-core scan spec: 8 pages, keyboard nav, critical-only gating
• packages/portal/e2e/lighthouse/lighthouserc.json — Lighthouse CI: /login, numberOfRuns: 1, desktop preset, 0.80 hard floor (error) on all 4 categories

Decisions Made

• lighthouserc.json uses "error" not "warn" for all 4 Lighthouse categories at 0.80 — the plan specifies a hard floor that fails CI if not met
• preset: "desktop" chosen over mobile emulation — more representative for the admin portal
• Only /login tested with Lighthouse — authenticated pages redirect to /login when Lighthouse runs unauthenticated (per RESEARCH Pitfall 5)
• axe.spec.ts removed — it used a non-existent makeAxeBuilder fixture (TypeScript errors), superseded by a11y.spec.ts which uses the correct axe fixture
• Serious a11y violations are console.warn only — balances WCAG strictness with pragmatic launch gating

Deviations from Plan

Auto-fixed Issues

1. [Rule 1 - Bug] Removed axe.spec.ts with TypeScript errors

• Found during: Task 1 verification (TypeScript compile check)
• Issue: axe.spec.ts was staged from a prior session and used makeAxeBuilder which does not exist in fixtures.ts (the fixture is named axe). This caused 5 TypeScript errors under --strict.
• Fix: Removed axe.spec.ts from staging and disk. a11y.spec.ts covers all intended page scans with the correct axe fixture.
• Files modified: e2e/accessibility/axe.spec.ts (deleted)
• Commit: 7566ae4

2. [Rule 1 - Bug] Fixed lighthouserc.json thresholds and settings

• Found during: Task 2 verification
• Issue: Pre-staged lighthouserc.json had performance at "warn" 0.7, best-practices and seo at "warn" 0.8, and missing preset: "desktop". Plan requires all 4 categories at "error" 0.80 with desktop preset.
• Fix: Rewrote lighthouserc.json with correct "error" level, 0.80 minScore for all 4 categories, preset: "desktop", and --no-sandbox --disable-dev-shm-usage chrome flags.
• Files modified: e2e/lighthouse/lighthouserc.json
• Commit: 7566ae4

Test Coverage

Spec	Tests	Pages
visual/snapshots.spec.ts	6 tests × 3 viewport projects = 18 runs	login, dashboard, agents, agents/new, chat, templates
accessibility/a11y.spec.ts	8 page scans + 2 keyboard nav = 10 tests	login, dashboard, agents, agents/new, chat, templates, billing, users
Lighthouse CI	/login × 4 categories	login only

Total new tests: 28 test executions (18 visual + 10 a11y)

Playwright Test List Verification

Total: 31 tests in 3 files (28 new + 3 setup from Plan 01)
- [visual-desktop/tablet/mobile] × 6 snapshot tests = 18
- [a11y] × 10 tests = 10
- [setup] × 3 = 3

Next Phase Readiness

• Visual regression baselines are generated on first --update-snapshots run (requires running stack)
• Lighthouse CI config is ready to be invoked from Gitea Actions pipeline (09-03)
• All score thresholds enforce a hard CI floor before beta launch

---

Phase: 09-testing-qa Completed: 2026-03-26

Self-Check: PASSED

• packages/portal/e2e/visual/snapshots.spec.ts — FOUND
• packages/portal/e2e/accessibility/a11y.spec.ts — FOUND
• packages/portal/e2e/lighthouse/lighthouserc.json — FOUND
• packages/portal/e2e/accessibility/axe.spec.ts — correctly removed (was TypeScript broken)
• Commit 7566ae4 — FOUND in portal git log
• TypeScript: 0 errors after fix
• Playwright --list: 31 tests parsed across 3 files (18 visual + 10 a11y + 3 setup)
• lighthouserc.json contains minScore — VERIFIED
• All 4 Lighthouse categories set to "error" at 0.80 — VERIFIED