docs(09-testing-qa): create phase plan

2026-03-25 22:26:03 -06:00
parent a46ff0a970
commit e31690e37a
4 changed files with 607 additions and 5 deletions
--- a/.planning/phases/09-testing-qa/09-01-PLAN.md
+++ b/.planning/phases/09-testing-qa/09-01-PLAN.md
@@ -0,0 +1,239 @@
+---
+phase: 09-testing-qa
+plan: 01
+type: execute
+wave: 1
+depends_on: []
+files_modified:
+  - packages/portal/playwright.config.ts
+  - packages/portal/e2e/auth.setup.ts
+  - packages/portal/e2e/fixtures.ts
+  - packages/portal/e2e/helpers/seed.ts
+  - packages/portal/e2e/flows/login.spec.ts
+  - packages/portal/e2e/flows/tenant-crud.spec.ts
+  - packages/portal/e2e/flows/agent-deploy.spec.ts
+  - packages/portal/e2e/flows/chat.spec.ts
+  - packages/portal/e2e/flows/rbac.spec.ts
+  - packages/portal/e2e/flows/i18n.spec.ts
+  - packages/portal/e2e/flows/mobile.spec.ts
+  - packages/portal/package.json
+autonomous: true
+requirements:
+  - QA-01
+  - QA-05
+  - QA-06
+
+must_haves:
+  truths:
+    - "Playwright E2E tests cover all 7 critical user flows and pass on chromium"
+    - "Tests pass on all 3 browsers (chromium, firefox, webkit)"
+    - "Empty states, error states, and loading states are tested within flow specs"
+    - "Auth setup saves storageState for 3 roles (platform_admin, customer_admin, customer_operator)"
+  artifacts:
+    - path: "packages/portal/playwright.config.ts"
+      provides: "Playwright configuration with 3 browser projects + setup project"
+      contains: "defineConfig"
+    - path: "packages/portal/e2e/auth.setup.ts"
+      provides: "Auth state generation for 3 roles"
+      contains: "storageState"
+    - path: "packages/portal/e2e/fixtures.ts"
+      provides: "Shared test fixtures with axe builder and role-based auth"
+      exports: ["test", "expect"]
+    - path: "packages/portal/e2e/helpers/seed.ts"
+      provides: "Test data seeding via FastAPI admin API"
+      exports: ["seedTestTenant"]
+    - path: "packages/portal/e2e/flows/login.spec.ts"
+      provides: "Login flow E2E test"
+    - path: "packages/portal/e2e/flows/chat.spec.ts"
+      provides: "Chat flow E2E test with WebSocket mock"
+    - path: "packages/portal/e2e/flows/rbac.spec.ts"
+      provides: "RBAC enforcement E2E test"
+  key_links:
+    - from: "packages/portal/e2e/auth.setup.ts"
+      to: "playwright/.auth/*.json"
+      via: "storageState save"
+      pattern: "storageState.*path"
+    - from: "packages/portal/e2e/flows/*.spec.ts"
+      to: "packages/portal/e2e/fixtures.ts"
+      via: "import { test } from fixtures"
+      pattern: "from.*fixtures"
+    - from: "packages/portal/playwright.config.ts"
+      to: ".next/standalone/server.js"
+      via: "webServer command"
+      pattern: "node .next/standalone/server.js"
+---
+
+<objective>
+Set up Playwright E2E testing infrastructure and implement all 7 critical user flow tests covering login, tenant CRUD, agent deployment, chat with mocked WebSocket, RBAC enforcement, i18n language switching, and mobile viewport behavior.
+
+Purpose: Establishes the automated E2E test suite that validates all critical user paths work end-to-end across Chrome, Firefox, and Safari -- the primary quality gate for beta readiness.
+
+Output: Playwright config, auth fixtures for 3 roles, seed helpers, and 7 flow spec files that pass on all 3 browsers.
+</objective>
+
+<execution_context>
+@/home/adelorenzo/.claude/get-shit-done/workflows/execute-plan.md
+@/home/adelorenzo/.claude/get-shit-done/templates/summary.md
+</execution_context>
+
+<context>
+@.planning/PROJECT.md
+@.planning/ROADMAP.md
+@.planning/STATE.md
+@.planning/phases/09-testing-qa/09-CONTEXT.md
+@.planning/phases/09-testing-qa/09-RESEARCH.md
+
+Key codebase references:
+@packages/portal/package.json
+@packages/portal/next.config.ts
+@packages/portal/app/layout.tsx
+@packages/portal/app/login/page.tsx
+@packages/portal/lib/use-chat-socket.ts
+@packages/portal/app/(app)/chat/page.tsx
+</context>
+
+<tasks>
+
+<task type="auto">
+  <name>Task 1: Install Playwright and create test infrastructure</name>
+  <files>
+    packages/portal/package.json
+    packages/portal/playwright.config.ts
+    packages/portal/e2e/auth.setup.ts
+    packages/portal/e2e/fixtures.ts
+    packages/portal/e2e/helpers/seed.ts
+    packages/portal/playwright/.auth/.gitkeep
+    packages/portal/.gitignore
+  </files>
+  <action>
+1. Install test dependencies:
+   ```
+   cd packages/portal
+   npm install --save-dev @playwright/test @axe-core/playwright @lhci/cli
+   npx playwright install --with-deps chromium firefox webkit
+   ```
+
+2. Create `packages/portal/playwright.config.ts` following the RESEARCH Pattern 6 exactly:
+   - testDir: "./e2e"
+   - fullyParallel: false (CI stability with shared DB state)
+   - forbidOnly: !!process.env.CI
+   - retries: process.env.CI ? 1 : 0
+   - workers: process.env.CI ? 1 : undefined
+   - timeout: 30_000
+   - reporter: html + junit + list
+   - use.baseURL from PLAYWRIGHT_BASE_URL env or localhost:3000
+   - use.trace: "on-first-retry"
+   - use.screenshot: "only-on-failure"
+   - use.serviceWorkers: "block" (CRITICAL: prevents Serwist from intercepting test requests)
+   - expect.toHaveScreenshot: maxDiffPixelRatio 0.02, threshold 0.2
+   - Projects: setup, chromium, firefox, webkit (all depend on setup, testMatch "e2e/flows/**")
+   - Visual projects: visual-desktop (1280x800), visual-tablet (768x1024), visual-mobile (iPhone 12 375x812) -- all chromium only, testMatch "e2e/visual/**"
+   - A11y project: chromium, testMatch "e2e/accessibility/**"
+   - webServer: command "node .next/standalone/server.js", url localhost:3000, reuseExistingServer: !process.env.CI
+   - webServer env: PORT 3000, API_URL from env or localhost:8001, AUTH_SECRET test-secret, AUTH_URL localhost:3000
+   - Default storageState for chromium/firefox/webkit: "playwright/.auth/platform-admin.json"
+
+3. Create `packages/portal/e2e/auth.setup.ts`:
+   - 3 setup blocks: platform admin, customer admin, customer operator
+   - Each: goto /login, fill Email + Password from env vars (E2E_ADMIN_EMAIL/E2E_ADMIN_PASSWORD, E2E_CADMIN_EMAIL/E2E_CADMIN_PASSWORD, E2E_OPERATOR_EMAIL/E2E_OPERATOR_PASSWORD), click Sign In button, waitForURL /dashboard, save storageState to playwright/.auth/{role}.json
+   - Use path.resolve(__dirname, ...) for auth file paths
+
+4. Create `packages/portal/e2e/fixtures.ts`:
+   - Extend base test with: axe fixture (returns () => AxeBuilder with wcag2a, wcag2aa, wcag21aa tags), auth state paths as constants
+   - Export `test` and `expect` from the extended fixture
+   - Export AUTH_PATHS object: { platformAdmin, customerAdmin, operator } with resolved paths
+
+5. Create `packages/portal/e2e/helpers/seed.ts`:
+   - seedTestTenant(request: APIRequestContext) -- POST to /api/portal/tenants with X-User-Id, X-User-Role headers, returns { tenantId, tenantSlug }
+   - cleanupTenant(request: APIRequestContext, tenantId: string) -- DELETE /api/portal/tenants/{id}
+   - Use random suffix for tenant names to avoid collisions
+
+6. Create `packages/portal/playwright/.auth/.gitkeep` (empty file)
+7. Add to packages/portal/.gitignore: `playwright/.auth/*.json`, `playwright-report/`, `playwright-results.xml`, `.lighthouseci/`
+  </action>
+  <verify>
+    <automated>cd /home/adelorenzo/repos/konstruct/packages/portal && npx playwright --version && test -f playwright.config.ts && test -f e2e/auth.setup.ts && test -f e2e/fixtures.ts && test -f e2e/helpers/seed.ts && echo "PASS"</automated>
+  </verify>
+  <done>Playwright installed, config created with 3 browser + 3 visual + 1 a11y projects, auth setup saves storageState for 3 roles, fixtures export axe builder and auth paths, seed helper creates/deletes test tenants via API</done>
+</task>
+
+<task type="auto">
+  <name>Task 2: Implement all 7 critical flow E2E tests</name>
+  <files>
+    packages/portal/e2e/flows/login.spec.ts
+    packages/portal/e2e/flows/tenant-crud.spec.ts
+    packages/portal/e2e/flows/agent-deploy.spec.ts
+    packages/portal/e2e/flows/chat.spec.ts
+    packages/portal/e2e/flows/rbac.spec.ts
+    packages/portal/e2e/flows/i18n.spec.ts
+    packages/portal/e2e/flows/mobile.spec.ts
+  </files>
+  <action>
+All specs import `{ test, expect }` from `../fixtures`. Use semantic selectors (getByRole, getByLabel, getByText) -- never CSS IDs or data-testid unless no semantic selector exists.
+
+1. `login.spec.ts` (Flow 1):
+   - Test "login -> dashboard loads -> session persists": goto /login, fill credentials, click Sign In, waitForURL /dashboard, assert dashboard heading visible. Reload page, assert still on /dashboard (session persists).
+   - Test "invalid credentials show error": fill wrong password, submit, assert error message visible.
+   - Test "empty state: no session redirects to login": use empty storageState ({}), goto /dashboard, assert redirected to /login.
+
+2. `tenant-crud.spec.ts` (Flow 2):
+   - Uses platform_admin storageState
+   - Test "create tenant -> appears in list": navigate to tenants page, click create button, fill tenant name + slug (random suffix), submit, assert new tenant appears in list.
+   - Test "delete tenant": create tenant, then delete it, assert it disappears from list.
+   - Use seed helper for setup where possible.
+
+3. `agent-deploy.spec.ts` (Flow 3):
+   - Uses customer_admin storageState (or platform_admin with tenant context)
+   - Test "deploy template agent -> appears in employees": navigate to /agents/new, select template option, pick first available template, click deploy, assert agent appears in agents list.
+   - Test "loading state: template gallery shows loading skeleton": mock API to delay, assert skeleton/loading indicator visible.
+
+4. `chat.spec.ts` (Flow 4):
+   - Uses routeWebSocket per RESEARCH Pattern 2
+   - Test "send message -> receive streaming response": routeWebSocket matching /\/chat\/ws\//, mock auth acknowledgment and message response with simulated streaming tokens. Open chat page, select an agent/conversation, type message, press Enter, assert response text appears.
+   - Test "typing indicator shows during response": assert typing indicator visible between message send and response arrival.
+   - Test "empty state: no conversations shows prompt": navigate to /chat without selecting agent, assert empty state message visible.
+   - IMPORTANT: Use regex pattern for routeWebSocket: `/\/chat\/ws\//` (not string) -- the portal derives WS URL from NEXT_PUBLIC_API_URL which is absolute.
+
+5. `rbac.spec.ts` (Flow 5):
+   - Uses customer_operator storageState
+   - Test "operator cannot access restricted paths": for each of ["/agents/new", "/billing", "/users"], goto path, assert NOT on that URL (proxy.ts redirects to /dashboard).
+   - Test "operator can view dashboard and chat": goto /dashboard, assert visible. Goto /chat, assert visible.
+   - Uses customer_admin storageState for contrast test: "admin can access /agents/new".
+
+6. `i18n.spec.ts` (Flow 6):
+   - Test "language switcher changes UI to Spanish": find language switcher, select Espanol, assert key UI elements render in Spanish (check a known label like "Dashboard" -> "Panel" or whatever the Spanish translation is -- read from the messages/es.json file).
+   - Test "language persists across page navigation": switch to Portuguese, navigate to another page, assert Portuguese labels still showing.
+
+7. `mobile.spec.ts` (Flow 7):
+   - Test "mobile viewport: bottom tab bar renders, sidebar hidden": setViewportSize 375x812, goto /dashboard, assert mobile bottom navigation visible, assert desktop sidebar not visible.
+   - Test "mobile chat: full screen message view": setViewportSize 375x812, navigate to chat, assert chat interface fills viewport.
+   - Test "error state: offline banner" (if applicable): if the PWA has offline detection, test it shows a banner.
+
+For QA-06 coverage, embed empty/error/loading state tests within the relevant flow specs (noted above with specific test cases).
+  </action>
+  <verify>
+    <automated>cd /home/adelorenzo/repos/konstruct/packages/portal && npx playwright test e2e/flows/ --project=chromium --reporter=list 2>&1 | tail -20</automated>
+  </verify>
+  <done>All 7 flow spec files exist with tests for critical paths plus empty/error/loading states. Tests pass on chromium. Cross-browser pass (firefox, webkit) confirmed by running full project suite.</done>
+</task>
+
+</tasks>
+
+<verification>
+1. `cd packages/portal && npx playwright test e2e/flows/ --project=chromium` -- all flow tests pass on chromium
+2. `cd packages/portal && npx playwright test e2e/flows/` -- all flow tests pass on chromium + firefox + webkit
+3. Each flow spec covers at least one happy path and one edge/error case
+4. Auth setup generates 3 storageState files in playwright/.auth/
+</verification>
+
+<success_criteria>
+- 7 flow spec files exist and pass on chromium
+- Cross-browser (chromium + firefox + webkit) all green
+- Empty/error/loading states tested within flow specs
+- Auth storageState generated for 3 roles without manual intervention
+- No real LLM calls in any test (WebSocket mocked)
+</success_criteria>
+
+<output>
+After completion, create `.planning/phases/09-testing-qa/09-01-SUMMARY.md`
+</output>