--- phase: 09-testing-qa plan: 01 type: execute wave: 1 depends_on: [] files_modified: - packages/portal/playwright.config.ts - packages/portal/e2e/auth.setup.ts - packages/portal/e2e/fixtures.ts - packages/portal/e2e/helpers/seed.ts - packages/portal/e2e/flows/login.spec.ts - packages/portal/e2e/flows/tenant-crud.spec.ts - packages/portal/e2e/flows/agent-deploy.spec.ts - packages/portal/e2e/flows/chat.spec.ts - packages/portal/e2e/flows/rbac.spec.ts - packages/portal/e2e/flows/i18n.spec.ts - packages/portal/e2e/flows/mobile.spec.ts - packages/portal/package.json autonomous: true requirements: - QA-01 - QA-05 - QA-06 must_haves: truths: - "Playwright E2E tests cover all 7 critical user flows and pass on chromium" - "Tests pass on all 3 browsers (chromium, firefox, webkit)" - "Empty states, error states, and loading states are tested within flow specs" - "Auth setup saves storageState for 3 roles (platform_admin, customer_admin, customer_operator)" artifacts: - path: "packages/portal/playwright.config.ts" provides: "Playwright configuration with 3 browser projects + setup project" contains: "defineConfig" - path: "packages/portal/e2e/auth.setup.ts" provides: "Auth state generation for 3 roles" contains: "storageState" - path: "packages/portal/e2e/fixtures.ts" provides: "Shared test fixtures with axe builder and role-based auth" exports: ["test", "expect"] - path: "packages/portal/e2e/helpers/seed.ts" provides: "Test data seeding via FastAPI admin API" exports: ["seedTestTenant"] - path: "packages/portal/e2e/flows/login.spec.ts" provides: "Login flow E2E test" - path: "packages/portal/e2e/flows/chat.spec.ts" provides: "Chat flow E2E test with WebSocket mock" - path: "packages/portal/e2e/flows/rbac.spec.ts" provides: "RBAC enforcement E2E test" key_links: - from: "packages/portal/e2e/auth.setup.ts" to: "playwright/.auth/*.json" via: "storageState save" pattern: "storageState.*path" - from: "packages/portal/e2e/flows/*.spec.ts" to: "packages/portal/e2e/fixtures.ts" via: "import { test } from fixtures" pattern: "from.*fixtures" - from: "packages/portal/playwright.config.ts" to: ".next/standalone/server.js" via: "webServer command" pattern: "node .next/standalone/server.js" --- Set up Playwright E2E testing infrastructure and implement all 7 critical user flow tests covering login, tenant CRUD, agent deployment, chat with mocked WebSocket, RBAC enforcement, i18n language switching, and mobile viewport behavior. Purpose: Establishes the automated E2E test suite that validates all critical user paths work end-to-end across Chrome, Firefox, and Safari -- the primary quality gate for beta readiness. Output: Playwright config, auth fixtures for 3 roles, seed helpers, and 7 flow spec files that pass on all 3 browsers. @/home/adelorenzo/.claude/get-shit-done/workflows/execute-plan.md @/home/adelorenzo/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/09-testing-qa/09-CONTEXT.md @.planning/phases/09-testing-qa/09-RESEARCH.md Key codebase references: @packages/portal/package.json @packages/portal/next.config.ts @packages/portal/app/layout.tsx @packages/portal/app/login/page.tsx @packages/portal/lib/use-chat-socket.ts @packages/portal/app/(app)/chat/page.tsx Task 1: Install Playwright and create test infrastructure packages/portal/package.json packages/portal/playwright.config.ts packages/portal/e2e/auth.setup.ts packages/portal/e2e/fixtures.ts packages/portal/e2e/helpers/seed.ts packages/portal/playwright/.auth/.gitkeep packages/portal/.gitignore 1. Install test dependencies: ``` cd packages/portal npm install --save-dev @playwright/test @axe-core/playwright @lhci/cli npx playwright install --with-deps chromium firefox webkit ``` 2. Create `packages/portal/playwright.config.ts` following the RESEARCH Pattern 6 exactly: - testDir: "./e2e" - fullyParallel: false (CI stability with shared DB state) - forbidOnly: !!process.env.CI - retries: process.env.CI ? 1 : 0 - workers: process.env.CI ? 1 : undefined - timeout: 30_000 - reporter: html + junit + list - use.baseURL from PLAYWRIGHT_BASE_URL env or localhost:3000 - use.trace: "on-first-retry" - use.screenshot: "only-on-failure" - use.serviceWorkers: "block" (CRITICAL: prevents Serwist from intercepting test requests) - expect.toHaveScreenshot: maxDiffPixelRatio 0.02, threshold 0.2 - Projects: setup, chromium, firefox, webkit (all depend on setup, testMatch "e2e/flows/**") - Visual projects: visual-desktop (1280x800), visual-tablet (768x1024), visual-mobile (iPhone 12 375x812) -- all chromium only, testMatch "e2e/visual/**" - A11y project: chromium, testMatch "e2e/accessibility/**" - webServer: command "node .next/standalone/server.js", url localhost:3000, reuseExistingServer: !process.env.CI - webServer env: PORT 3000, API_URL from env or localhost:8001, AUTH_SECRET test-secret, AUTH_URL localhost:3000 - Default storageState for chromium/firefox/webkit: "playwright/.auth/platform-admin.json" 3. Create `packages/portal/e2e/auth.setup.ts`: - 3 setup blocks: platform admin, customer admin, customer operator - Each: goto /login, fill Email + Password from env vars (E2E_ADMIN_EMAIL/E2E_ADMIN_PASSWORD, E2E_CADMIN_EMAIL/E2E_CADMIN_PASSWORD, E2E_OPERATOR_EMAIL/E2E_OPERATOR_PASSWORD), click Sign In button, waitForURL /dashboard, save storageState to playwright/.auth/{role}.json - Use path.resolve(__dirname, ...) for auth file paths 4. Create `packages/portal/e2e/fixtures.ts`: - Extend base test with: axe fixture (returns () => AxeBuilder with wcag2a, wcag2aa, wcag21aa tags), auth state paths as constants - Export `test` and `expect` from the extended fixture - Export AUTH_PATHS object: { platformAdmin, customerAdmin, operator } with resolved paths 5. Create `packages/portal/e2e/helpers/seed.ts`: - seedTestTenant(request: APIRequestContext) -- POST to /api/portal/tenants with X-User-Id, X-User-Role headers, returns { tenantId, tenantSlug } - cleanupTenant(request: APIRequestContext, tenantId: string) -- DELETE /api/portal/tenants/{id} - Use random suffix for tenant names to avoid collisions 6. Create `packages/portal/playwright/.auth/.gitkeep` (empty file) 7. Add to packages/portal/.gitignore: `playwright/.auth/*.json`, `playwright-report/`, `playwright-results.xml`, `.lighthouseci/` cd /home/adelorenzo/repos/konstruct/packages/portal && npx playwright --version && test -f playwright.config.ts && test -f e2e/auth.setup.ts && test -f e2e/fixtures.ts && test -f e2e/helpers/seed.ts && echo "PASS" Playwright installed, config created with 3 browser + 3 visual + 1 a11y projects, auth setup saves storageState for 3 roles, fixtures export axe builder and auth paths, seed helper creates/deletes test tenants via API Task 2: Implement all 7 critical flow E2E tests packages/portal/e2e/flows/login.spec.ts packages/portal/e2e/flows/tenant-crud.spec.ts packages/portal/e2e/flows/agent-deploy.spec.ts packages/portal/e2e/flows/chat.spec.ts packages/portal/e2e/flows/rbac.spec.ts packages/portal/e2e/flows/i18n.spec.ts packages/portal/e2e/flows/mobile.spec.ts All specs import `{ test, expect }` from `../fixtures`. Use semantic selectors (getByRole, getByLabel, getByText) -- never CSS IDs or data-testid unless no semantic selector exists. 1. `login.spec.ts` (Flow 1): - Test "login -> dashboard loads -> session persists": goto /login, fill credentials, click Sign In, waitForURL /dashboard, assert dashboard heading visible. Reload page, assert still on /dashboard (session persists). - Test "invalid credentials show error": fill wrong password, submit, assert error message visible. - Test "empty state: no session redirects to login": use empty storageState ({}), goto /dashboard, assert redirected to /login. 2. `tenant-crud.spec.ts` (Flow 2): - Uses platform_admin storageState - Test "create tenant -> appears in list": navigate to tenants page, click create button, fill tenant name + slug (random suffix), submit, assert new tenant appears in list. - Test "delete tenant": create tenant, then delete it, assert it disappears from list. - Use seed helper for setup where possible. 3. `agent-deploy.spec.ts` (Flow 3): - Uses customer_admin storageState (or platform_admin with tenant context) - Test "deploy template agent -> appears in employees": navigate to /agents/new, select template option, pick first available template, click deploy, assert agent appears in agents list. - Test "loading state: template gallery shows loading skeleton": mock API to delay, assert skeleton/loading indicator visible. 4. `chat.spec.ts` (Flow 4): - Uses routeWebSocket per RESEARCH Pattern 2 - Test "send message -> receive streaming response": routeWebSocket matching /\/chat\/ws\//, mock auth acknowledgment and message response with simulated streaming tokens. Open chat page, select an agent/conversation, type message, press Enter, assert response text appears. - Test "typing indicator shows during response": assert typing indicator visible between message send and response arrival. - Test "empty state: no conversations shows prompt": navigate to /chat without selecting agent, assert empty state message visible. - IMPORTANT: Use regex pattern for routeWebSocket: `/\/chat\/ws\//` (not string) -- the portal derives WS URL from NEXT_PUBLIC_API_URL which is absolute. 5. `rbac.spec.ts` (Flow 5): - Uses customer_operator storageState - Test "operator cannot access restricted paths": for each of ["/agents/new", "/billing", "/users"], goto path, assert NOT on that URL (proxy.ts redirects to /dashboard). - Test "operator can view dashboard and chat": goto /dashboard, assert visible. Goto /chat, assert visible. - Uses customer_admin storageState for contrast test: "admin can access /agents/new". 6. `i18n.spec.ts` (Flow 6): - Test "language switcher changes UI to Spanish": find language switcher, select Espanol, assert key UI elements render in Spanish (check a known label like "Dashboard" -> "Panel" or whatever the Spanish translation is -- read from the messages/es.json file). - Test "language persists across page navigation": switch to Portuguese, navigate to another page, assert Portuguese labels still showing. 7. `mobile.spec.ts` (Flow 7): - Test "mobile viewport: bottom tab bar renders, sidebar hidden": setViewportSize 375x812, goto /dashboard, assert mobile bottom navigation visible, assert desktop sidebar not visible. - Test "mobile chat: full screen message view": setViewportSize 375x812, navigate to chat, assert chat interface fills viewport. - Test "error state: offline banner" (if applicable): if the PWA has offline detection, test it shows a banner. For QA-06 coverage, embed empty/error/loading state tests within the relevant flow specs (noted above with specific test cases). cd /home/adelorenzo/repos/konstruct/packages/portal && npx playwright test e2e/flows/ --project=chromium --reporter=list 2>&1 | tail -20 All 7 flow spec files exist with tests for critical paths plus empty/error/loading states. Tests pass on chromium. Cross-browser pass (firefox, webkit) confirmed by running full project suite. 1. `cd packages/portal && npx playwright test e2e/flows/ --project=chromium` -- all flow tests pass on chromium 2. `cd packages/portal && npx playwright test e2e/flows/` -- all flow tests pass on chromium + firefox + webkit 3. Each flow spec covers at least one happy path and one edge/error case 4. Auth setup generates 3 storageState files in playwright/.auth/ - 7 flow spec files exist and pass on chromium - Cross-browser (chromium + firefox + webkit) all green - Empty/error/loading states tested within flow specs - Auth storageState generated for 3 roles without manual intervention - No real LLM calls in any test (WebSocket mocked) After completion, create `.planning/phases/09-testing-qa/09-01-SUMMARY.md`