---
phase: 09-testing-qa
plan: "02"
subsystem: testing
tags: [playwright, visual-regression, axe-core, a11y, lighthouse, wcag, snapshots, keyboard-nav]

# Dependency graph
requires:
  - phase: 09-testing-qa
    plan: "01"
    provides: playwright.config.ts (visual/a11y projects), fixtures.ts (axe fixture), auth.setup.ts (storageState)
provides:
  - Visual regression spec: 6 pages at 3 viewports (desktop/tablet/mobile)
  - Accessibility scan spec: 8 pages with critical-violation gating, serious logged as warnings
  - Keyboard navigation tests for login form and chat input
  - Lighthouse CI config with 0.80 hard floor on all 4 categories for /login
affects: [CI pipeline (09-03 Gitea Actions), QA baseline before beta launch]

# Tech tracking
tech-stack:
  added: []
  patterns:
    - "Visual regression via playwright visual-desktop/visual-tablet/visual-mobile projects — spec runs once, projects vary viewport"
    - "axe fixture from fixtures.ts — returns () => AxeBuilder scoped to wcag2a/wcag2aa/wcag21aa"
    - "Critical-only gating — critical violations fail the test, serious logged as console.warn"
    - "Lighthouse CI desktop preset — /login only (authenticated pages redirect unauthenticated)"

key-files:
  created:
    - packages/portal/e2e/visual/snapshots.spec.ts
    - packages/portal/e2e/accessibility/a11y.spec.ts
    - packages/portal/e2e/lighthouse/lighthouserc.json
  modified: []

key-decisions:
  - "lighthouserc.json uses error (not warn) at minScore 0.80 for all 4 categories — plan hard floor requirement"
  - "preset: desktop in lighthouserc — more representative of actual usage than mobile emulation"
  - "a11y.spec.ts not axe.spec.ts — a11y.spec.ts uses the correct axe fixture; axe.spec.ts had wrong fixture name (makeAxeBuilder) causing TypeScript errors"
  - "Serious a11y violations are warnings not blockers — balances correctness with pragmatism for beta launch"
  - "Visual baselines require running stack — committed specs only, baselines generated on first --update-snapshots run"

# Metrics
duration: ~1min
completed: "2026-03-26"
---

# Phase 9 Plan 02: Visual Regression, Accessibility, and Lighthouse CI Summary

**Visual regression snapshots at 3 viewports, axe-core WCAG 2.1 AA scanning on 8 pages, and Lighthouse CI with 0.80 hard floor on all 4 categories — QA baseline before beta launch**

## Performance

- **Duration:** ~1 min
- **Started:** 2026-03-26
- **Completed:** 2026-03-26
- **Tasks:** 2
- **Files modified:** 3

## Accomplishments

- `snapshots.spec.ts`: 6 key pages (login, dashboard, agents list, agents/new, chat, templates) each captured via 3 viewport projects — 18 total visual test runs
- `a11y.spec.ts`: 8 pages scanned with axe-core, critical violations are hard failures, serious violations logged as `console.warn` but pass; 2 keyboard navigation tests (login form tab order, chat message input focus)
- `lighthouserc.json`: Lighthouse CI targeting `/login` only (authenticated pages redirect when unauthenticated), desktop preset, all 4 score categories at "error" level with 0.80 minimum
- Removed pre-existing `axe.spec.ts` which had TypeScript errors (wrong fixture name `makeAxeBuilder` — fixture is `axe`)

## Task Commits

Both tasks landed in a single atomic commit due to `lighthouserc.json` being pre-staged from a prior session:

1. **Task 1 + Task 2: Visual regression + a11y + Lighthouse CI** — `7566ae4` (feat)
   - `e2e/visual/snapshots.spec.ts` — 6-page visual snapshot spec
   - `e2e/accessibility/a11y.spec.ts` — 8-page axe-core scan + 2 keyboard nav tests
   - `e2e/lighthouse/lighthouserc.json` — Lighthouse CI config, 0.80 hard floor all categories

## Files Created/Modified

- `packages/portal/e2e/visual/snapshots.spec.ts` — Visual regression spec: 6 pages, `toHaveScreenshot`, imports from `../fixtures`
- `packages/portal/e2e/accessibility/a11y.spec.ts` — axe-core scan spec: 8 pages, keyboard nav, critical-only gating
- `packages/portal/e2e/lighthouse/lighthouserc.json` — Lighthouse CI: `/login`, numberOfRuns: 1, desktop preset, 0.80 hard floor (error) on all 4 categories

## Decisions Made

- `lighthouserc.json` uses `"error"` not `"warn"` for all 4 Lighthouse categories at 0.80 — the plan specifies a hard floor that fails CI if not met
- `preset: "desktop"` chosen over mobile emulation — more representative for the admin portal
- Only `/login` tested with Lighthouse — authenticated pages redirect to `/login` when Lighthouse runs unauthenticated (per RESEARCH Pitfall 5)
- `axe.spec.ts` removed — it used a non-existent `makeAxeBuilder` fixture (TypeScript errors), superseded by `a11y.spec.ts` which uses the correct `axe` fixture
- Serious a11y violations are `console.warn` only — balances WCAG strictness with pragmatic launch gating

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 1 - Bug] Removed axe.spec.ts with TypeScript errors**
- **Found during:** Task 1 verification (TypeScript compile check)
- **Issue:** `axe.spec.ts` was staged from a prior session and used `makeAxeBuilder` which does not exist in `fixtures.ts` (the fixture is named `axe`). This caused 5 TypeScript errors under `--strict`.
- **Fix:** Removed `axe.spec.ts` from staging and disk. `a11y.spec.ts` covers all intended page scans with the correct `axe` fixture.
- **Files modified:** `e2e/accessibility/axe.spec.ts` (deleted)
- **Commit:** `7566ae4`

**2. [Rule 1 - Bug] Fixed lighthouserc.json thresholds and settings**
- **Found during:** Task 2 verification
- **Issue:** Pre-staged `lighthouserc.json` had `performance` at `"warn"` 0.7, `best-practices` and `seo` at `"warn"` 0.8, and missing `preset: "desktop"`. Plan requires all 4 categories at `"error"` 0.80 with desktop preset.
- **Fix:** Rewrote `lighthouserc.json` with correct `"error"` level, 0.80 minScore for all 4 categories, `preset: "desktop"`, and `--no-sandbox --disable-dev-shm-usage` chrome flags.
- **Files modified:** `e2e/lighthouse/lighthouserc.json`
- **Commit:** `7566ae4`

## Test Coverage

| Spec | Tests | Pages |
|------|-------|-------|
| `visual/snapshots.spec.ts` | 6 tests × 3 viewport projects = 18 runs | login, dashboard, agents, agents/new, chat, templates |
| `accessibility/a11y.spec.ts` | 8 page scans + 2 keyboard nav = 10 tests | login, dashboard, agents, agents/new, chat, templates, billing, users |
| Lighthouse CI | `/login` × 4 categories | login only |

**Total new tests: 28 test executions (18 visual + 10 a11y)**

## Playwright Test List Verification

```
Total: 31 tests in 3 files (28 new + 3 setup from Plan 01)
- [visual-desktop/tablet/mobile] × 6 snapshot tests = 18
- [a11y] × 10 tests = 10
- [setup] × 3 = 3
```

## Next Phase Readiness

- Visual regression baselines are generated on first `--update-snapshots` run (requires running stack)
- Lighthouse CI config is ready to be invoked from Gitea Actions pipeline (09-03)
- All score thresholds enforce a hard CI floor before beta launch

---
*Phase: 09-testing-qa*
*Completed: 2026-03-26*

## Self-Check: PASSED

- `packages/portal/e2e/visual/snapshots.spec.ts` — FOUND
- `packages/portal/e2e/accessibility/a11y.spec.ts` — FOUND
- `packages/portal/e2e/lighthouse/lighthouserc.json` — FOUND
- `packages/portal/e2e/accessibility/axe.spec.ts` — correctly removed (was TypeScript broken)
- Commit `7566ae4` — FOUND in portal git log
- TypeScript: 0 errors after fix
- Playwright --list: 31 tests parsed across 3 files (18 visual + 10 a11y + 3 setup)
- `lighthouserc.json` contains `minScore` — VERIFIED
- All 4 Lighthouse categories set to `"error"` at 0.80 — VERIFIED