diff --git a/.planning/phases/09-testing-qa/09-RESEARCH.md b/.planning/phases/09-testing-qa/09-RESEARCH.md
new file mode 100644
index 0000000..560cb9f
--- /dev/null
+++ b/.planning/phases/09-testing-qa/09-RESEARCH.md
@@ -0,0 +1,764 @@
+# Phase 9: Testing & QA - Research
+
+**Researched:** 2026-03-25
+**Domain:** Playwright E2E, Lighthouse CI, visual regression, axe-core accessibility, Gitea Actions CI
+**Confidence:** HIGH
+
+## Summary
+
+Phase 9 is a greenfield testing layer added on top of a fully-built portal (Next.js 16 standalone, FastAPI gateway, Celery worker). No Playwright config exists yet — the Playwright MCP plugin is installed for manual use but there is no `playwright.config.ts`, no `tests/e2e/` content, and no `.gitea/workflows/` CI file. Everything must be created from scratch.
+
+The core challenges are: (1) Auth.js v5 JWT sessions that Playwright must obtain and reuse across multiple role fixtures (platform_admin, customer_admin, customer_operator); (2) the WebSocket chat flow at `/chat/ws/{conversation_id}` that needs mocking via `page.routeWebSocket()`; (3) Lighthouse CI that requires a running Next.js server (standalone output complicates `startServerCommand`); and (4) a sub-5-minute pipeline on Gitea Actions that is nearly syntax-identical to GitHub Actions.
+
+**Primary recommendation:** Place Playwright config and tests inside `packages/portal/` (Next.js co-location pattern), use `storageState` with three saved auth fixtures for roles, mock the WebSocket endpoint with `page.routeWebSocket()` for the chat flow, and run `@lhci/cli` in a separate post-build CI stage.
+
+
+## User Constraints (from CONTEXT.md)
+
+### Locked Decisions
+All decisions at Claude's discretion — user trusts judgment.
+
+- Playwright for all E2E tests (cross-browser built-in, official Next.js recommendation)
+- Critical flows to test (priority order):
+ 1. Login → dashboard loads → session persists
+ 2. Create tenant → tenant appears in list
+ 3. Deploy template agent → agent appears in employees list
+ 4. Chat: open conversation → send message → receive streaming response (mock LLM)
+ 5. RBAC: operator cannot access /agents/new, /billing, /users
+ 6. Language switcher → UI updates to selected language
+ 7. Mobile viewport: bottom tab bar renders, sidebar hidden
+- LLM responses mocked in E2E tests (no real Ollama/API calls)
+- Test data: seed a test tenant + test user via API calls in test setup, clean up after
+- Lighthouse targets: >= 90 (fail at 80, warn at 85)
+- Pages: login, dashboard, chat, agents/new
+- Visual regression at 3 viewports: desktop 1280x800, tablet 768x1024, mobile 375x812
+- Key pages: login, dashboard, agents list, agents/new (3-card entry), chat (empty state), templates gallery
+- Baseline snapshots committed to repo
+- axe-core via @axe-core/playwright, zero critical violations required
+- "serious" violations logged as warnings (not blockers for beta)
+- Keyboard navigation test: Tab through login form, chat input, nav items
+- Cross-browser: chromium, firefox, webkit
+- Visual regression: chromium only
+- Gitea Actions, triggers: push to main, PR to main
+- Pipeline stages: lint → type-check → unit tests (pytest) → build portal → E2E tests → Lighthouse
+- Docker Compose for CI infra
+- JUnit XML + HTML trace viewer reports
+- Fail-fast: lint/type errors block everything; unit test failures block E2E
+- Target: < 5 min pipeline
+
+### Claude's Discretion
+- Playwright config details (timeouts, retries, parallelism)
+- Test file organization (by feature vs by page)
+- Fixture/helper patterns for auth, tenant setup, API mocking
+- Lighthouse CI tool (lighthouse-ci vs @lhci/cli)
+- Whether to include a smoke test for the WebSocket chat connection
+- Visual regression threshold (pixel diff tolerance)
+
+### Deferred Ideas (OUT OF SCOPE)
+None — discussion stayed within phase scope
+
+
+
+## Phase Requirements
+
+| ID | Description | Research Support |
+|----|-------------|-----------------|
+| QA-01 | Playwright E2E tests cover all critical user flows (login, tenant CRUD, agent deploy, chat, billing, RBAC) | Playwright storageState auth fixtures + routeWebSocket for chat mock |
+| QA-02 | Lighthouse scores >= 90 for performance, accessibility, best practices, SEO on key pages | @lhci/cli with minScore assertions per category |
+| QA-03 | Visual regression snapshots at desktop/tablet/mobile for all key pages | toHaveScreenshot with maxDiffPixelRatio, viewports per project |
+| QA-04 | axe-core accessibility audit passes with zero critical violations across all pages | @axe-core/playwright AxeBuilder with impact filter |
+| QA-05 | E2E tests pass on Chrome, Firefox, Safari (WebKit) | Playwright projects array with three browser engines |
+| QA-06 | Empty states, error states, loading states tested and rendered correctly | Dedicated test cases + API mocking for empty/error responses |
+| QA-07 | CI-ready test suite runnable in Gitea Actions pipeline | .gitea/workflows/ci.yml with Docker Compose service containers |
+
+
+## Standard Stack
+
+### Core
+| Library | Version | Purpose | Why Standard |
+|---------|---------|---------|--------------|
+| @playwright/test | ^1.51 | E2E + visual regression + accessibility runner | Official Next.js recommendation, cross-browser built-in, no extra dependencies |
+| @axe-core/playwright | ^4.10 | Accessibility scanning within Playwright tests | Official Deque package, integrates directly with Playwright page objects |
+| @lhci/cli | ^0.15 | Lighthouse CI score assertions | Google-maintained, headless Lighthouse, assertion config via lighthouserc |
+
+### Supporting
+| Library | Version | Purpose | When to Use |
+|---------|---------|---------|-------------|
+| axe-html-reporter | ^2.2 | HTML accessibility reports | When you want human-readable a11y reports attached to CI artifacts |
+
+### Alternatives Considered
+| Instead of | Could Use | Tradeoff |
+|------------|-----------|----------|
+| @lhci/cli | lighthouse npm module directly | @lhci/cli handles multi-run averaging, assertions, and CI upload; raw lighthouse requires custom scripting |
+| @axe-core/playwright | axe-playwright (third-party) | @axe-core/playwright is the official Deque package; axe-playwright is a community wrapper with same API but extra dep |
+
+**Installation (portal):**
+```bash
+cd packages/portal
+npm install --save-dev @playwright/test @axe-core/playwright @lhci/cli
+npx playwright install --with-deps chromium firefox webkit
+```
+
+## Architecture Patterns
+
+### Recommended Project Structure
+```
+packages/portal/
+├── playwright.config.ts # Main config: projects, webServer, globalSetup
+├── e2e/
+│ ├── auth.setup.ts # Global setup: save storageState per role
+│ ├── fixtures.ts # Extended test: auth fixtures, axe builder, API helpers
+│ ├── helpers/
+│ │ ├── seed.ts # Seed test tenant + user via API, return IDs
+│ │ └── cleanup.ts # Delete seeded data after test suite
+│ ├── flows/
+│ │ ├── login.spec.ts # Flow 1: login → dashboard loads → session persists
+│ │ ├── tenant-crud.spec.ts # Flow 2: create tenant → appears in list
+│ │ ├── agent-deploy.spec.ts # Flow 3: deploy template → appears in employees
+│ │ ├── chat.spec.ts # Flow 4: open chat → send msg → streaming response (mocked WS)
+│ │ ├── rbac.spec.ts # Flow 5: operator access denied to restricted pages
+│ │ ├── i18n.spec.ts # Flow 6: language switcher → UI updates
+│ │ └── mobile.spec.ts # Flow 7: mobile viewport → bottom tab bar, sidebar hidden
+│ ├── accessibility/
+│ │ └── a11y.spec.ts # axe-core scan on every key page, keyboard nav test
+│ ├── visual/
+│ │ └── snapshots.spec.ts # Visual regression at 3 viewports (chromium only)
+│ └── lighthouse/
+│ └── lighthouserc.json # @lhci/cli config: URLs, score thresholds
+├── playwright/.auth/ # gitignored — saved storageState files
+│ ├── platform-admin.json
+│ ├── customer-admin.json
+│ └── customer-operator.json
+└── __snapshots__/ # Committed baseline screenshots
+.gitea/
+└── workflows/
+ └── ci.yml # Pipeline: lint → typecheck → pytest → build → E2E → lhci
+```
+
+### Pattern 1: Auth.js v5 storageState with Multiple Roles
+
+**What:** Authenticate each role once in a global setup project, save to JSON. All E2E tests consume the saved state — no repeated login UI interactions.
+
+**When to use:** Any test that requires a logged-in user. Each spec declares which role it needs via `test.use({ storageState })`.
+
+**Key insight for Auth.js v5:** The credentials provider calls the FastAPI `/api/portal/auth/verify` endpoint. Playwright must fill the login form (not call the API directly) because `next-auth` sets `HttpOnly` session cookies that only the browser can hold. The storageState captures those cookies.
+
+```typescript
+// Source: https://playwright.dev/docs/auth
+// e2e/auth.setup.ts
+import { test as setup, expect } from "@playwright/test";
+import path from "path";
+
+const PLATFORM_ADMIN_AUTH = path.resolve(__dirname, "../playwright/.auth/platform-admin.json");
+const CUSTOMER_ADMIN_AUTH = path.resolve(__dirname, "../playwright/.auth/customer-admin.json");
+const OPERATOR_AUTH = path.resolve(__dirname, "../playwright/.auth/customer-operator.json");
+
+setup("authenticate as platform admin", async ({ page }) => {
+ await page.goto("/login");
+ await page.getByLabel("Email").fill(process.env.E2E_ADMIN_EMAIL!);
+ await page.getByLabel("Password").fill(process.env.E2E_ADMIN_PASSWORD!);
+ await page.getByRole("button", { name: /sign in/i }).click();
+ await page.waitForURL("/dashboard");
+ await page.context().storageState({ path: PLATFORM_ADMIN_AUTH });
+});
+
+setup("authenticate as customer admin", async ({ page }) => {
+ // seed returns { email, password } for a fresh customer_admin user
+ await page.goto("/login");
+ await page.getByLabel("Email").fill(process.env.E2E_CADMIN_EMAIL!);
+ await page.getByLabel("Password").fill(process.env.E2E_CADMIN_PASSWORD!);
+ await page.getByRole("button", { name: /sign in/i }).click();
+ await page.waitForURL("/dashboard");
+ await page.context().storageState({ path: CUSTOMER_ADMIN_AUTH });
+});
+```
+
+### Pattern 2: WebSocket Mocking for Chat Flow
+
+**What:** Intercept the `/chat/ws/{conversationId}` WebSocket before the gateway is contacted. Respond to the auth message, then simulate streaming tokens on a user message.
+
+**When to use:** Flow 4 (chat E2E test). The gateway WebSocket endpoint at `ws://localhost:8001/chat/ws/{id}` is routed via the Next.js API proxy — intercept at the browser level.
+
+```typescript
+// Source: https://playwright.dev/docs/api/class-websocketroute
+// e2e/flows/chat.spec.ts
+test("chat: send message → receive streaming response", async ({ page }) => {
+ await page.routeWebSocket(/\/chat\/ws\//, (ws) => {
+ ws.onMessage((msg) => {
+ const data = JSON.parse(msg as string);
+
+ if (data.type === "auth") {
+ // Acknowledge auth — no response needed, gateway just proceeds
+ return;
+ }
+
+ if (data.type === "message") {
+ // Simulate typing indicator
+ ws.send(JSON.stringify({ type: "typing" }));
+ // Simulate streaming tokens
+ const tokens = ["Hello", " from", " your", " AI", " assistant!"];
+ tokens.forEach((token, i) => {
+ setTimeout(() => {
+ ws.send(JSON.stringify({ type: "chunk", token }));
+ }, i * 50);
+ });
+ setTimeout(() => {
+ ws.send(JSON.stringify({
+ type: "response",
+ text: tokens.join(""),
+ conversation_id: data.conversation_id,
+ }));
+ ws.send(JSON.stringify({ type: "done", text: tokens.join("") }));
+ }, tokens.length * 50 + 100);
+ }
+ });
+ });
+
+ await page.goto("/chat?agentId=test-agent");
+ await page.getByPlaceholder(/type a message/i).fill("Hello!");
+ await page.keyboard.press("Enter");
+ await expect(page.getByText("Hello from your AI assistant!")).toBeVisible({ timeout: 5000 });
+});
+```
+
+### Pattern 3: Visual Regression at Multiple Viewports
+
+**What:** Configure separate Playwright projects for each viewport, run snapshots only on chromium to avoid cross-browser rendering diffs.
+
+**When to use:** QA-03. Visual regression baseline committed to repo; CI fails on diff.
+
+```typescript
+// Source: https://playwright.dev/docs/test-snapshots
+// playwright.config.ts (visual projects section)
+{
+ name: "visual-desktop",
+ use: {
+ ...devices["Desktop Chrome"],
+ viewport: { width: 1280, height: 800 },
+ },
+ testMatch: "e2e/visual/**",
+},
+{
+ name: "visual-tablet",
+ use: {
+ browserName: "chromium",
+ viewport: { width: 768, height: 1024 },
+ },
+ testMatch: "e2e/visual/**",
+},
+{
+ name: "visual-mobile",
+ use: {
+ ...devices["iPhone 12"],
+ viewport: { width: 375, height: 812 },
+ },
+ testMatch: "e2e/visual/**",
+},
+```
+
+Global threshold:
+```typescript
+// playwright.config.ts
+expect: {
+ toHaveScreenshot: {
+ maxDiffPixelRatio: 0.02, // 2% tolerance — accounts for antialiasing
+ threshold: 0.2, // pixel color threshold (0–1)
+ },
+},
+```
+
+### Pattern 4: axe-core Fixture
+
+**What:** Shared fixture that creates an AxeBuilder for each page, scoped to WCAG 2.1 AA, filtering results by impact level.
+
+```typescript
+// Source: https://playwright.dev/docs/accessibility-testing
+// e2e/fixtures.ts
+import { test as base, expect } from "@playwright/test";
+import AxeBuilder from "@axe-core/playwright";
+
+export const test = base.extend<{ axe: () => AxeBuilder }>({
+ axe: async ({ page }, use) => {
+ const makeBuilder = () =>
+ new AxeBuilder({ page })
+ .withTags(["wcag2a", "wcag2aa", "wcag21aa"]);
+ await use(makeBuilder);
+ },
+});
+
+// In a test:
+const results = await axe().analyze();
+const criticalViolations = results.violations.filter(v => v.impact === "critical");
+const seriousViolations = results.violations.filter(v => v.impact === "serious");
+
+expect(criticalViolations, "Critical a11y violations found").toHaveLength(0);
+if (seriousViolations.length > 0) {
+ console.warn("Serious a11y violations (non-blocking):", seriousViolations);
+}
+```
+
+### Pattern 5: Lighthouse CI Config
+
+**What:** `lighthouserc.json` drives `@lhci/cli autorun` in CI. Pages run headlessly against the built portal.
+
+```json
+// Source: https://googlechrome.github.io/lighthouse-ci/docs/configuration.html
+// e2e/lighthouse/lighthouserc.json
+{
+ "ci": {
+ "collect": {
+ "url": [
+ "http://localhost:3000/login",
+ "http://localhost:3000/dashboard",
+ "http://localhost:3000/chat",
+ "http://localhost:3000/agents/new"
+ ],
+ "numberOfRuns": 1,
+ "settings": {
+ "preset": "desktop",
+ "chromeFlags": "--no-sandbox --disable-dev-shm-usage"
+ }
+ },
+ "assert": {
+ "assertions": {
+ "categories:performance": ["error", {"minScore": 0.80}],
+ "categories:accessibility": ["error", {"minScore": 0.80}],
+ "categories:best-practices": ["error", {"minScore": 0.80}],
+ "categories:seo": ["error", {"minScore": 0.80}]
+ }
+ },
+ "upload": {
+ "target": "filesystem",
+ "outputDir": ".lighthouseci"
+ }
+ }
+}
+```
+
+Note: `error` at 0.80 means CI fails below 80; the 90 target is aspirational. Set warn at 0.85 for soft alerts.
+
+### Pattern 6: Playwright Config (Full)
+
+```typescript
+// packages/portal/playwright.config.ts
+import { defineConfig, devices } from "@playwright/test";
+
+export default defineConfig({
+ testDir: "./e2e",
+ fullyParallel: false, // Stability in CI with shared DB state
+ forbidOnly: !!process.env.CI,
+ retries: process.env.CI ? 1 : 0,
+ workers: process.env.CI ? 1 : undefined,
+ timeout: 30_000,
+
+ reporter: [
+ ["html", { outputFolder: "playwright-report" }],
+ ["junit", { outputFile: "playwright-results.xml" }],
+ ["list"],
+ ],
+
+ use: {
+ baseURL: process.env.PLAYWRIGHT_BASE_URL ?? "http://localhost:3000",
+ trace: "on-first-retry",
+ screenshot: "only-on-failure",
+ serviceWorkers: "block", // Prevents Serwist from intercepting test requests
+ },
+
+ expect: {
+ toHaveScreenshot: {
+ maxDiffPixelRatio: 0.02,
+ threshold: 0.2,
+ },
+ },
+
+ projects: [
+ // Auth setup runs first for all browser projects
+ { name: "setup", testMatch: /auth\.setup\.ts/ },
+
+ // E2E flows — all 3 browsers
+ {
+ name: "chromium",
+ use: { ...devices["Desktop Chrome"], storageState: "playwright/.auth/platform-admin.json" },
+ dependencies: ["setup"],
+ testMatch: "e2e/flows/**",
+ },
+ {
+ name: "firefox",
+ use: { ...devices["Desktop Firefox"], storageState: "playwright/.auth/platform-admin.json" },
+ dependencies: ["setup"],
+ testMatch: "e2e/flows/**",
+ },
+ {
+ name: "webkit",
+ use: { ...devices["Desktop Safari"], storageState: "playwright/.auth/platform-admin.json" },
+ dependencies: ["setup"],
+ testMatch: "e2e/flows/**",
+ },
+
+ // Visual regression — chromium only, 3 viewports
+ { name: "visual-desktop", use: { browserName: "chromium", viewport: { width: 1280, height: 800 } }, testMatch: "e2e/visual/**", dependencies: ["setup"] },
+ { name: "visual-tablet", use: { browserName: "chromium", viewport: { width: 768, height: 1024 } }, testMatch: "e2e/visual/**", dependencies: ["setup"] },
+ { name: "visual-mobile", use: { ...devices["iPhone 12"] }, testMatch: "e2e/visual/**", dependencies: ["setup"] },
+
+ // Accessibility — chromium only
+ {
+ name: "a11y",
+ use: { ...devices["Desktop Chrome"] },
+ dependencies: ["setup"],
+ testMatch: "e2e/accessibility/**",
+ },
+ ],
+
+ webServer: {
+ command: "node .next/standalone/server.js",
+ url: "http://localhost:3000",
+ reuseExistingServer: !process.env.CI,
+ env: {
+ PORT: "3000",
+ API_URL: process.env.API_URL ?? "http://localhost:8001",
+ AUTH_SECRET: process.env.AUTH_SECRET ?? "test-secret-32-chars-minimum-len",
+ AUTH_URL: "http://localhost:3000",
+ },
+ },
+});
+```
+
+**Critical:** `serviceWorkers: "block"` is required because Serwist (PWA service worker) intercepts network requests and makes them invisible to `page.route()` / `page.routeWebSocket()`.
+
+### Pattern 7: Gitea Actions CI Pipeline
+
+```yaml
+# .gitea/workflows/ci.yml
+name: CI
+
+on:
+ push:
+ branches: [main]
+ pull_request:
+ branches: [main]
+
+jobs:
+ backend:
+ name: Backend Tests
+ runs-on: ubuntu-latest
+ services:
+ postgres:
+ image: pgvector/pgvector:pg16
+ env:
+ POSTGRES_DB: konstruct
+ POSTGRES_USER: postgres
+ POSTGRES_PASSWORD: postgres_dev
+ options: >-
+ --health-cmd pg_isready
+ --health-interval 5s
+ --health-timeout 5s
+ --health-retries 10
+ redis:
+ image: redis:7-alpine
+ options: >-
+ --health-cmd "redis-cli ping"
+ --health-interval 5s
+ env:
+ DATABASE_URL: postgresql+asyncpg://konstruct_app:konstruct_dev@localhost:5432/konstruct
+ DATABASE_ADMIN_URL: postgresql+asyncpg://postgres:postgres_dev@localhost:5432/konstruct
+ REDIS_URL: redis://localhost:6379/0
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with: { python-version: "3.12" }
+ - run: pip install uv
+ - run: uv sync
+ - run: uv run ruff check packages/ tests/
+ - run: uv run mypy --strict packages/
+ - run: uv run pytest tests/ -x --tb=short
+
+ portal:
+ name: Portal E2E
+ runs-on: ubuntu-latest
+ needs: backend # E2E blocked until backend passes
+ services:
+ postgres:
+ image: pgvector/pgvector:pg16
+ env:
+ POSTGRES_DB: konstruct
+ POSTGRES_USER: postgres
+ POSTGRES_PASSWORD: postgres_dev
+ options: --health-cmd pg_isready --health-interval 5s --health-retries 10
+ redis:
+ image: redis:7-alpine
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-node@v4
+ with: { node-version: "22" }
+ - name: Install portal deps
+ working-directory: packages/portal
+ run: npm ci
+ - name: Build portal
+ working-directory: packages/portal
+ run: npm run build
+ env:
+ NEXT_PUBLIC_API_URL: http://localhost:8001
+ - name: Install Playwright browsers
+ working-directory: packages/portal
+ run: npx playwright install --with-deps chromium firefox webkit
+ - name: Start gateway (background)
+ run: |
+ pip install uv && uv sync
+ uv run alembic upgrade head
+ uv run uvicorn gateway.main:app --host 0.0.0.0 --port 8001 &
+ env:
+ DATABASE_URL: postgresql+asyncpg://konstruct_app:konstruct_dev@localhost:5432/konstruct
+ DATABASE_ADMIN_URL: postgresql+asyncpg://postgres:postgres_dev@localhost:5432/konstruct
+ REDIS_URL: redis://localhost:6379/0
+ LLM_POOL_URL: http://localhost:8004 # not running — mocked in E2E
+ - name: Wait for gateway
+ run: timeout 30 bash -c 'until curl -sf http://localhost:8001/health; do sleep 1; done'
+ - name: Run E2E tests
+ working-directory: packages/portal
+ run: npx playwright test e2e/flows/ e2e/accessibility/
+ env:
+ CI: "true"
+ PLAYWRIGHT_BASE_URL: http://localhost:3000
+ API_URL: http://localhost:8001
+ AUTH_SECRET: ${{ secrets.AUTH_SECRET }}
+ E2E_ADMIN_EMAIL: ${{ secrets.E2E_ADMIN_EMAIL }}
+ E2E_ADMIN_PASSWORD: ${{ secrets.E2E_ADMIN_PASSWORD }}
+ - name: Run Lighthouse CI
+ working-directory: packages/portal
+ run: |
+ npx lhci autorun --config=e2e/lighthouse/lighthouserc.json
+ env:
+ LHCI_BUILD_CONTEXT__CURRENT_HASH: ${{ github.sha }}
+ - name: Upload Playwright report
+ if: always()
+ uses: actions/upload-artifact@v4
+ with:
+ name: playwright-report
+ path: packages/portal/playwright-report/
+ - name: Upload Lighthouse report
+ if: always()
+ uses: actions/upload-artifact@v4
+ with:
+ name: lighthouse-report
+ path: packages/portal/.lighthouseci/
+```
+
+### Anti-Patterns to Avoid
+
+- **Hardcoded IDs in selectors:** Use `getByRole`, `getByLabel`, `getByText` — never CSS `#id` or `[data-testid]` unless semantic selectors are unavailable. Semantic selectors are more resilient and double as accessibility checks.
+- **Real LLM calls in E2E:** Never let E2E tests reach Ollama/OpenAI. Mock the WebSocket and gateway LLM calls. Real calls introduce flakiness and cost.
+- **Superuser DB connections in test seeds:** The existing conftest uses `konstruct_app` role to preserve RLS. E2E seeds should call the FastAPI admin API endpoints, not connect directly to the DB.
+- **Enabling service workers in tests:** Serwist intercepts all requests. Always set `serviceWorkers: "block"` in Playwright config.
+- **Parallel workers with shared DB state:** Set `workers: 1` in CI. Tenant/agent mutations are not thread-safe across workers without per-worker isolation.
+- **Running visual regression on all browsers:** Browser rendering engines produce expected pixel diffs. Visual regression on chromium only; cross-browser covered by functional E2E.
+
+## Don't Hand-Roll
+
+| Problem | Don't Build | Use Instead | Why |
+|---------|-------------|-------------|-----|
+| Screenshot diffs | Custom pixel comparator | `toHaveScreenshot()` built into Playwright | Handles baseline storage, update workflow, CI reporting |
+| Accessibility scanning | Custom ARIA traversal | `@axe-core/playwright` | Covers 57 WCAG rules including ones humans miss |
+| Performance score gating | Parsing Lighthouse JSON manually | `@lhci/cli assert` | Handles multi-run averaging, threshold config, exit codes |
+| Auth state reuse | Logging in before every test | Playwright `storageState` | Session reuse makes the suite 10x faster |
+| WS mock server | Running a real mock websocket server | `page.routeWebSocket()` | In-process, no port conflicts, no flakiness |
+
+## Common Pitfalls
+
+### Pitfall 1: Auth.js HttpOnly Cookies
+**What goes wrong:** Trying to authenticate by calling `/api/portal/auth/verify` directly with Playwright `request` — this bypasses Auth.js cookie-setting, so the browser session never exists.
+**Why it happens:** Auth.js v5 JWT is set as `HttpOnly` secure cookie by the Next.js server, not by the FastAPI backend.
+**How to avoid:** Always use Playwright's UI login flow (fill form → submit → wait for redirect) to let Next.js set the cookie. Then save with `storageState`.
+**Warning signs:** Tests pass the login assertion but fail immediately after on authenticated pages.
+
+### Pitfall 2: Serwist Service Worker Intercepting Test Traffic
+**What goes wrong:** `page.route()` and `page.routeWebSocket()` handlers never fire because the PWA service worker handles requests first.
+**Why it happens:** Serwist registers a service worker that intercepts all requests matching the scope. Playwright's routing operates at the network level before the service worker, but only if service workers are blocked.
+**How to avoid:** Set `serviceWorkers: "block"` in `playwright.config.ts` under `use`.
+**Warning signs:** Mock routes never called; tests see real responses or network errors.
+
+### Pitfall 3: Next.js Standalone Output Path for webServer
+**What goes wrong:** `command: "npm run start"` fails in CI because `next start` requires the dev server setup, not standalone output.
+**Why it happens:** The portal uses `output: "standalone"` in `next.config.ts`. The build produces `.next/standalone/server.js`, not the standard Next.js CLI server.
+**How to avoid:** Use `command: "node .next/standalone/server.js"` in Playwright's `webServer` config. Copy static files if needed: the build step must run `cp -r .next/static .next/standalone/.next/static && cp -r public .next/standalone/public`.
+**Warning signs:** `webServer` process exits immediately; Playwright reports "server did not start".
+
+### Pitfall 4: Visual Regression Baseline Committed Without CI Environment Lock
+**What goes wrong:** Baselines created on a developer's Mac differ from Linux CI renderings (font rendering, subpixel AA, etc.).
+**Why it happens:** Screenshot comparisons are pixel-exact. OS-level rendering differences cause 1–5% false failures.
+**How to avoid:** Generate baselines inside the same Docker/Linux environment as CI. Run `npx playwright test --update-snapshots` on Linux (or in the Playwright Docker image) to commit initial baselines. Use `maxDiffPixelRatio: 0.02` to absorb minor remaining differences.
+**Warning signs:** Visual tests pass locally but always fail in CI.
+
+### Pitfall 5: Lighthouse Pages Behind Auth
+**What goes wrong:** Lighthouse visits `/dashboard` and gets redirected to `/login` — scores an empty page.
+**Why it happens:** Lighthouse runs as an unauthenticated browser session. LHCI doesn't support Auth.js cookie injection.
+**How to avoid:** For authenticated pages, either (a) test only public pages with Lighthouse (login, landing), or (b) use LHCI's `basicAuth` option for pages behind HTTP auth (not applicable here), or (c) create a special unauthenticated preview mode. **For this project:** Run Lighthouse on `/login` only, plus any public-accessible marketing pages. Skip `/dashboard` and `/chat` for Lighthouse.
+**Warning signs:** Lighthouse scores 100 for accessibility on dashboard — suspiciously perfect because it's measuring an empty redirect.
+
+### Pitfall 6: WebSocket URL Resolution in Tests
+**What goes wrong:** `page.routeWebSocket("/chat/ws/")` doesn't match because the portal derives the WS URL from `NEXT_PUBLIC_API_URL` (baked at build time), which points to `ws://localhost:8001`, not a relative path.
+**Why it happens:** `use-chat-socket.ts` computes `WS_BASE` from `process.env.NEXT_PUBLIC_API_URL` and builds `ws://localhost:8001/chat/ws/{id}`.
+**How to avoid:** Use a regex pattern: `page.routeWebSocket(/\/chat\/ws\//, handler)` — this matches the full absolute URL.
+**Warning signs:** Chat mock never fires; test times out waiting for WS message.
+
+### Pitfall 7: Gitea Actions Runner Needs Docker
+**What goes wrong:** Service containers fail to start because the Gitea runner is not configured with Docker access.
+**Why it happens:** Gitea Actions service containers require Docker socket access on the runner.
+**How to avoid:** Ensure the `act_runner` is added to the `docker` group on the host. Alternative: use `docker compose` in a setup step instead of service containers.
+**Warning signs:** Job fails immediately with "Cannot connect to Docker daemon".
+
+## Code Examples
+
+### Seed Helper via API
+```typescript
+// e2e/helpers/seed.ts
+// Uses Playwright APIRequestContext to create test data via FastAPI endpoints.
+// Must run BEFORE storageState setup (needs platform_admin creds via env).
+export async function seedTestTenant(request: APIRequestContext): Promise<{ tenantId: string; tenantSlug: string }> {
+ const suffix = Math.random().toString(36).slice(2, 8);
+ const res = await request.post("http://localhost:8001/api/portal/tenants", {
+ headers: {
+ "X-User-Id": process.env.E2E_ADMIN_ID!,
+ "X-User-Role": "platform_admin",
+ "X-Active-Tenant": "",
+ },
+ data: { name: `E2E Tenant ${suffix}`, slug: `e2e-tenant-${suffix}` },
+ });
+ const body = await res.json() as { id: string; slug: string };
+ return { tenantId: body.id, tenantSlug: body.slug };
+}
+```
+
+### RBAC Test Pattern
+```typescript
+// e2e/flows/rbac.spec.ts
+// Tests that operator role is silently redirected, not 403-paged
+test.describe("RBAC enforcement", () => {
+ test.use({ storageState: "playwright/.auth/customer-operator.json" });
+
+ const restrictedPaths = ["/agents/new", "/billing", "/users"];
+
+ for (const path of restrictedPaths) {
+ test(`operator cannot access ${path}`, async ({ page }) => {
+ await page.goto(path);
+ // proxy.ts does silent redirect — operator ends up on /dashboard
+ await expect(page).not.toHaveURL(path);
+ });
+ }
+});
+```
+
+### Mobile Viewport Behavioral Test
+```typescript
+// e2e/flows/mobile.spec.ts
+test("mobile: bottom tab bar renders, sidebar hidden", async ({ page }) => {
+ await page.setViewportSize({ width: 375, height: 812 });
+ await page.goto("/dashboard");
+ // Bottom tab bar visible
+ await expect(page.getByRole("navigation", { name: /mobile/i })).toBeVisible();
+ // Desktop sidebar hidden
+ await expect(page.getByRole("navigation", { name: /sidebar/i })).not.toBeVisible();
+});
+```
+
+## State of the Art
+
+| Old Approach | Current Approach | When Changed | Impact |
+|--------------|------------------|--------------|--------|
+| Cypress for Next.js E2E | Playwright (official Next.js recommendation) | 2023–2024 | Cross-browser, better WS support, no iframe limitations |
+| `lighthouse` npm module with custom scripts | `@lhci/cli autorun` | 2020+ | Automated multi-run averaging, assertions, CI reporting |
+| `axe-playwright` (community) | `@axe-core/playwright` (official Deque) | 2022+ | Official package, same API, no extra wrapper |
+| `next start` for E2E server | `node .next/standalone/server.js` | Next.js 12+ standalone | Required when `output: "standalone"` is set |
+| middleware.ts | proxy.ts | Next.js 16 | Next.js 16 renamed middleware file |
+
+**Deprecated/outdated:**
+- `cypress/integration/` directory: Cypress split this into `cypress/e2e/` in v10 — but we're not using Cypress
+- `@playwright/test` `globalSetup` string path: Still valid but the project-based `setup` dependency is preferred in Playwright 1.40+
+- `installSerwist()`: Replaced by `new Serwist() + addEventListeners()` in serwist v9 (already applied in Phase 8)
+
+## Open Questions
+
+1. **Lighthouse on authenticated pages**
+ - What we know: Lighthouse runs as unauthenticated — authenticated pages redirect to `/login`
+ - What's unclear: Whether LHCI supports cookie injection (not documented)
+ - Recommendation: Scope Lighthouse to `/login` only for QA-02. Dashboard/chat performance validated manually or via Web Vitals tracking in production.
+
+2. **Visual regression baseline generation environment**
+ - What we know: OS-level rendering differences cause false failures
+ - What's unclear: Whether the Gitea runner is Linux or Mac
+ - Recommendation: Wave 0 task generates baselines inside the CI Docker container (Linux), commits them. Dev machines use `--update-snapshots` only deliberately.
+
+3. **Celery worker in E2E**
+ - What we know: The chat WebSocket flow uses Redis pub-sub to deliver responses from the Celery worker
+ - What's unclear: Whether E2E should run the Celery worker (real pipeline, slow) or mock the WS entirely (fast but less realistic)
+ - Recommendation: Mock the WebSocket entirely via `page.routeWebSocket()`. This tests the frontend streaming UX without depending on Celery. Add a separate smoke test that hits the gateway `/health` endpoint to verify service health in CI.
+
+## Validation Architecture
+
+### Test Framework
+| Property | Value |
+|----------|-------|
+| Framework (backend) | pytest 8.3+ / pytest-asyncio (existing, all tests pass) |
+| Framework (E2E) | @playwright/test ^1.51 (to be installed) |
+| Config file (E2E) | `packages/portal/playwright.config.ts` — Wave 0 |
+| Quick run (backend) | `uv run pytest tests/unit -x --tb=short` |
+| Full suite (backend) | `uv run pytest tests/ -x --tb=short` |
+| E2E run | `cd packages/portal && npx playwright test` |
+| Visual update | `cd packages/portal && npx playwright test --update-snapshots` |
+
+### Phase Requirements → Test Map
+| Req ID | Behavior | Test Type | Automated Command | File Exists? |
+|--------|----------|-----------|-------------------|-------------|
+| QA-01 | 7 critical user flows pass | E2E Playwright | `npx playwright test e2e/flows/ --project=chromium` | Wave 0 |
+| QA-02 | Lighthouse >= 90 on key pages | Lighthouse CI | `npx lhci autorun --config=e2e/lighthouse/lighthouserc.json` | Wave 0 |
+| QA-03 | Visual snapshots pass at 3 viewports | Visual regression | `npx playwright test e2e/visual/` | Wave 0 |
+| QA-04 | Zero critical a11y violations | Accessibility scan | `npx playwright test e2e/accessibility/` | Wave 0 |
+| QA-05 | All E2E flows pass on 3 browsers | Cross-browser E2E | `npx playwright test e2e/flows/` (all projects) | Wave 0 |
+| QA-06 | Empty/error/loading states correct | E2E Playwright | Covered within flow specs via API mocking | Wave 0 |
+| QA-07 | CI pipeline runs in Gitea Actions | CI workflow | `.gitea/workflows/ci.yml` | Wave 0 |
+
+### Sampling Rate
+- **Per task commit:** `cd packages/portal && npx playwright test e2e/flows/login.spec.ts --project=chromium`
+- **Per wave merge:** `cd packages/portal && npx playwright test e2e/flows/ --project=chromium`
+- **Phase gate:** Full suite (all projects + accessibility + visual) green before `/gsd:verify-work`
+
+### Wave 0 Gaps
+- [ ] `packages/portal/playwright.config.ts` — E2E framework config
+- [ ] `packages/portal/e2e/auth.setup.ts` — Auth state generation for 3 roles
+- [ ] `packages/portal/e2e/fixtures.ts` — Shared test fixtures (axe, auth, API helpers)
+- [ ] `packages/portal/e2e/helpers/seed.ts` — Test data seeding via API
+- [ ] `packages/portal/e2e/flows/*.spec.ts` — 7 flow spec files
+- [ ] `packages/portal/e2e/accessibility/a11y.spec.ts` — axe-core scans
+- [ ] `packages/portal/e2e/visual/snapshots.spec.ts` — visual regression specs
+- [ ] `packages/portal/e2e/lighthouse/lighthouserc.json` — Lighthouse CI config
+- [ ] `.gitea/workflows/ci.yml` — CI pipeline
+- [ ] `packages/portal/playwright/.auth/.gitkeep` — Directory for saved auth state (gitignored content)
+- [ ] Framework install: `cd packages/portal && npm install --save-dev @playwright/test @axe-core/playwright @lhci/cli && npx playwright install --with-deps`
+- [ ] Baseline snapshots: run `npx playwright test e2e/visual/ --update-snapshots` on Linux to generate
+
+## Sources
+
+### Primary (HIGH confidence)
+- https://playwright.dev/docs/auth — storageState, setup projects, multiple roles
+- https://playwright.dev/docs/api/class-websocketroute — WebSocket mocking API
+- https://playwright.dev/docs/test-snapshots — toHaveScreenshot, maxDiffPixelRatio
+- https://playwright.dev/docs/accessibility-testing — @axe-core/playwright integration
+- https://playwright.dev/docs/ci — CI configuration, Docker image, workers
+- https://googlechrome.github.io/lighthouse-ci/docs/configuration.html — minScore assertions format
+
+### Secondary (MEDIUM confidence)
+- https://googlechrome.github.io/lighthouse-ci/docs/getting-started.html — lhci autorun setup
+- https://playwright.dev/docs/mock — page.route() and page.routeWebSocket() overview
+- Gitea Actions docs (forum.gitea.com) — confirmed GitHub Actions YAML compatibility, Docker socket requirements
+
+### Tertiary (LOW confidence)
+- WebSearch result: Gitea runner Docker group requirement — mentioned across multiple community posts, not in official docs
+
+## Metadata
+
+**Confidence breakdown:**
+- Standard stack: HIGH — verified against official Playwright, @axe-core, and LHCI docs
+- Architecture: HIGH — patterns derived directly from official Playwright documentation
+- Pitfalls: HIGH (pitfalls 1–6 from direct codebase inspection + official docs); MEDIUM (pitfall 7 from community sources)
+
+**Research date:** 2026-03-25
+**Valid until:** 2026-06-25 (90 days — Playwright and Next.js are fast-moving but breaking changes are rare)