240 lines
12 KiB
Markdown
240 lines
12 KiB
Markdown
---
|
|
phase: 09-testing-qa
|
|
plan: 01
|
|
type: execute
|
|
wave: 1
|
|
depends_on: []
|
|
files_modified:
|
|
- packages/portal/playwright.config.ts
|
|
- packages/portal/e2e/auth.setup.ts
|
|
- packages/portal/e2e/fixtures.ts
|
|
- packages/portal/e2e/helpers/seed.ts
|
|
- packages/portal/e2e/flows/login.spec.ts
|
|
- packages/portal/e2e/flows/tenant-crud.spec.ts
|
|
- packages/portal/e2e/flows/agent-deploy.spec.ts
|
|
- packages/portal/e2e/flows/chat.spec.ts
|
|
- packages/portal/e2e/flows/rbac.spec.ts
|
|
- packages/portal/e2e/flows/i18n.spec.ts
|
|
- packages/portal/e2e/flows/mobile.spec.ts
|
|
- packages/portal/package.json
|
|
autonomous: true
|
|
requirements:
|
|
- QA-01
|
|
- QA-05
|
|
- QA-06
|
|
|
|
must_haves:
|
|
truths:
|
|
- "Playwright E2E tests cover all 7 critical user flows and pass on chromium"
|
|
- "Tests pass on all 3 browsers (chromium, firefox, webkit)"
|
|
- "Empty states, error states, and loading states are tested within flow specs"
|
|
- "Auth setup saves storageState for 3 roles (platform_admin, customer_admin, customer_operator)"
|
|
artifacts:
|
|
- path: "packages/portal/playwright.config.ts"
|
|
provides: "Playwright configuration with 3 browser projects + setup project"
|
|
contains: "defineConfig"
|
|
- path: "packages/portal/e2e/auth.setup.ts"
|
|
provides: "Auth state generation for 3 roles"
|
|
contains: "storageState"
|
|
- path: "packages/portal/e2e/fixtures.ts"
|
|
provides: "Shared test fixtures with axe builder and role-based auth"
|
|
exports: ["test", "expect"]
|
|
- path: "packages/portal/e2e/helpers/seed.ts"
|
|
provides: "Test data seeding via FastAPI admin API"
|
|
exports: ["seedTestTenant"]
|
|
- path: "packages/portal/e2e/flows/login.spec.ts"
|
|
provides: "Login flow E2E test"
|
|
- path: "packages/portal/e2e/flows/chat.spec.ts"
|
|
provides: "Chat flow E2E test with WebSocket mock"
|
|
- path: "packages/portal/e2e/flows/rbac.spec.ts"
|
|
provides: "RBAC enforcement E2E test"
|
|
key_links:
|
|
- from: "packages/portal/e2e/auth.setup.ts"
|
|
to: "playwright/.auth/*.json"
|
|
via: "storageState save"
|
|
pattern: "storageState.*path"
|
|
- from: "packages/portal/e2e/flows/*.spec.ts"
|
|
to: "packages/portal/e2e/fixtures.ts"
|
|
via: "import { test } from fixtures"
|
|
pattern: "from.*fixtures"
|
|
- from: "packages/portal/playwright.config.ts"
|
|
to: ".next/standalone/server.js"
|
|
via: "webServer command"
|
|
pattern: "node .next/standalone/server.js"
|
|
---
|
|
|
|
<objective>
|
|
Set up Playwright E2E testing infrastructure and implement all 7 critical user flow tests covering login, tenant CRUD, agent deployment, chat with mocked WebSocket, RBAC enforcement, i18n language switching, and mobile viewport behavior.
|
|
|
|
Purpose: Establishes the automated E2E test suite that validates all critical user paths work end-to-end across Chrome, Firefox, and Safari -- the primary quality gate for beta readiness.
|
|
|
|
Output: Playwright config, auth fixtures for 3 roles, seed helpers, and 7 flow spec files that pass on all 3 browsers.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@/home/adelorenzo/.claude/get-shit-done/workflows/execute-plan.md
|
|
@/home/adelorenzo/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/09-testing-qa/09-CONTEXT.md
|
|
@.planning/phases/09-testing-qa/09-RESEARCH.md
|
|
|
|
Key codebase references:
|
|
@packages/portal/package.json
|
|
@packages/portal/next.config.ts
|
|
@packages/portal/app/layout.tsx
|
|
@packages/portal/app/login/page.tsx
|
|
@packages/portal/lib/use-chat-socket.ts
|
|
@packages/portal/app/(app)/chat/page.tsx
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto">
|
|
<name>Task 1: Install Playwright and create test infrastructure</name>
|
|
<files>
|
|
packages/portal/package.json
|
|
packages/portal/playwright.config.ts
|
|
packages/portal/e2e/auth.setup.ts
|
|
packages/portal/e2e/fixtures.ts
|
|
packages/portal/e2e/helpers/seed.ts
|
|
packages/portal/playwright/.auth/.gitkeep
|
|
packages/portal/.gitignore
|
|
</files>
|
|
<action>
|
|
1. Install test dependencies:
|
|
```
|
|
cd packages/portal
|
|
npm install --save-dev @playwright/test @axe-core/playwright @lhci/cli
|
|
npx playwright install --with-deps chromium firefox webkit
|
|
```
|
|
|
|
2. Create `packages/portal/playwright.config.ts` following the RESEARCH Pattern 6 exactly:
|
|
- testDir: "./e2e"
|
|
- fullyParallel: false (CI stability with shared DB state)
|
|
- forbidOnly: !!process.env.CI
|
|
- retries: process.env.CI ? 1 : 0
|
|
- workers: process.env.CI ? 1 : undefined
|
|
- timeout: 30_000
|
|
- reporter: html + junit + list
|
|
- use.baseURL from PLAYWRIGHT_BASE_URL env or localhost:3000
|
|
- use.trace: "on-first-retry"
|
|
- use.screenshot: "only-on-failure"
|
|
- use.serviceWorkers: "block" (CRITICAL: prevents Serwist from intercepting test requests)
|
|
- expect.toHaveScreenshot: maxDiffPixelRatio 0.02, threshold 0.2
|
|
- Projects: setup, chromium, firefox, webkit (all depend on setup, testMatch "e2e/flows/**")
|
|
- Visual projects: visual-desktop (1280x800), visual-tablet (768x1024), visual-mobile (iPhone 12 375x812) -- all chromium only, testMatch "e2e/visual/**"
|
|
- A11y project: chromium, testMatch "e2e/accessibility/**"
|
|
- webServer: command "node .next/standalone/server.js", url localhost:3000, reuseExistingServer: !process.env.CI
|
|
- webServer env: PORT 3000, API_URL from env or localhost:8001, AUTH_SECRET test-secret, AUTH_URL localhost:3000
|
|
- Default storageState for chromium/firefox/webkit: "playwright/.auth/platform-admin.json"
|
|
|
|
3. Create `packages/portal/e2e/auth.setup.ts`:
|
|
- 3 setup blocks: platform admin, customer admin, customer operator
|
|
- Each: goto /login, fill Email + Password from env vars (E2E_ADMIN_EMAIL/E2E_ADMIN_PASSWORD, E2E_CADMIN_EMAIL/E2E_CADMIN_PASSWORD, E2E_OPERATOR_EMAIL/E2E_OPERATOR_PASSWORD), click Sign In button, waitForURL /dashboard, save storageState to playwright/.auth/{role}.json
|
|
- Use path.resolve(__dirname, ...) for auth file paths
|
|
|
|
4. Create `packages/portal/e2e/fixtures.ts`:
|
|
- Extend base test with: axe fixture (returns () => AxeBuilder with wcag2a, wcag2aa, wcag21aa tags), auth state paths as constants
|
|
- Export `test` and `expect` from the extended fixture
|
|
- Export AUTH_PATHS object: { platformAdmin, customerAdmin, operator } with resolved paths
|
|
|
|
5. Create `packages/portal/e2e/helpers/seed.ts`:
|
|
- seedTestTenant(request: APIRequestContext) -- POST to /api/portal/tenants with X-User-Id, X-User-Role headers, returns { tenantId, tenantSlug }
|
|
- cleanupTenant(request: APIRequestContext, tenantId: string) -- DELETE /api/portal/tenants/{id}
|
|
- Use random suffix for tenant names to avoid collisions
|
|
|
|
6. Create `packages/portal/playwright/.auth/.gitkeep` (empty file)
|
|
7. Add to packages/portal/.gitignore: `playwright/.auth/*.json`, `playwright-report/`, `playwright-results.xml`, `.lighthouseci/`
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/adelorenzo/repos/konstruct/packages/portal && npx playwright --version && test -f playwright.config.ts && test -f e2e/auth.setup.ts && test -f e2e/fixtures.ts && test -f e2e/helpers/seed.ts && echo "PASS"</automated>
|
|
</verify>
|
|
<done>Playwright installed, config created with 3 browser + 3 visual + 1 a11y projects, auth setup saves storageState for 3 roles, fixtures export axe builder and auth paths, seed helper creates/deletes test tenants via API</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 2: Implement all 7 critical flow E2E tests</name>
|
|
<files>
|
|
packages/portal/e2e/flows/login.spec.ts
|
|
packages/portal/e2e/flows/tenant-crud.spec.ts
|
|
packages/portal/e2e/flows/agent-deploy.spec.ts
|
|
packages/portal/e2e/flows/chat.spec.ts
|
|
packages/portal/e2e/flows/rbac.spec.ts
|
|
packages/portal/e2e/flows/i18n.spec.ts
|
|
packages/portal/e2e/flows/mobile.spec.ts
|
|
</files>
|
|
<action>
|
|
All specs import `{ test, expect }` from `../fixtures`. Use semantic selectors (getByRole, getByLabel, getByText) -- never CSS IDs or data-testid unless no semantic selector exists.
|
|
|
|
1. `login.spec.ts` (Flow 1):
|
|
- Test "login -> dashboard loads -> session persists": goto /login, fill credentials, click Sign In, waitForURL /dashboard, assert dashboard heading visible. Reload page, assert still on /dashboard (session persists).
|
|
- Test "invalid credentials show error": fill wrong password, submit, assert error message visible.
|
|
- Test "empty state: no session redirects to login": use empty storageState ({}), goto /dashboard, assert redirected to /login.
|
|
|
|
2. `tenant-crud.spec.ts` (Flow 2):
|
|
- Uses platform_admin storageState
|
|
- Test "create tenant -> appears in list": navigate to tenants page, click create button, fill tenant name + slug (random suffix), submit, assert new tenant appears in list.
|
|
- Test "delete tenant": create tenant, then delete it, assert it disappears from list.
|
|
- Use seed helper for setup where possible.
|
|
|
|
3. `agent-deploy.spec.ts` (Flow 3):
|
|
- Uses customer_admin storageState (or platform_admin with tenant context)
|
|
- Test "deploy template agent -> appears in employees": navigate to /agents/new, select template option, pick first available template, click deploy, assert agent appears in agents list.
|
|
- Test "loading state: template gallery shows loading skeleton": mock API to delay, assert skeleton/loading indicator visible.
|
|
|
|
4. `chat.spec.ts` (Flow 4):
|
|
- Uses routeWebSocket per RESEARCH Pattern 2
|
|
- Test "send message -> receive streaming response": routeWebSocket matching /\/chat\/ws\//, mock auth acknowledgment and message response with simulated streaming tokens. Open chat page, select an agent/conversation, type message, press Enter, assert response text appears.
|
|
- Test "typing indicator shows during response": assert typing indicator visible between message send and response arrival.
|
|
- Test "empty state: no conversations shows prompt": navigate to /chat without selecting agent, assert empty state message visible.
|
|
- IMPORTANT: Use regex pattern for routeWebSocket: `/\/chat\/ws\//` (not string) -- the portal derives WS URL from NEXT_PUBLIC_API_URL which is absolute.
|
|
|
|
5. `rbac.spec.ts` (Flow 5):
|
|
- Uses customer_operator storageState
|
|
- Test "operator cannot access restricted paths": for each of ["/agents/new", "/billing", "/users"], goto path, assert NOT on that URL (proxy.ts redirects to /dashboard).
|
|
- Test "operator can view dashboard and chat": goto /dashboard, assert visible. Goto /chat, assert visible.
|
|
- Uses customer_admin storageState for contrast test: "admin can access /agents/new".
|
|
|
|
6. `i18n.spec.ts` (Flow 6):
|
|
- Test "language switcher changes UI to Spanish": find language switcher, select Espanol, assert key UI elements render in Spanish (check a known label like "Dashboard" -> "Panel" or whatever the Spanish translation is -- read from the messages/es.json file).
|
|
- Test "language persists across page navigation": switch to Portuguese, navigate to another page, assert Portuguese labels still showing.
|
|
|
|
7. `mobile.spec.ts` (Flow 7):
|
|
- Test "mobile viewport: bottom tab bar renders, sidebar hidden": setViewportSize 375x812, goto /dashboard, assert mobile bottom navigation visible, assert desktop sidebar not visible.
|
|
- Test "mobile chat: full screen message view": setViewportSize 375x812, navigate to chat, assert chat interface fills viewport.
|
|
- Test "error state: offline banner" (if applicable): if the PWA has offline detection, test it shows a banner.
|
|
|
|
For QA-06 coverage, embed empty/error/loading state tests within the relevant flow specs (noted above with specific test cases).
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/adelorenzo/repos/konstruct/packages/portal && npx playwright test e2e/flows/ --project=chromium --reporter=list 2>&1 | tail -20</automated>
|
|
</verify>
|
|
<done>All 7 flow spec files exist with tests for critical paths plus empty/error/loading states. Tests pass on chromium. Cross-browser pass (firefox, webkit) confirmed by running full project suite.</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
1. `cd packages/portal && npx playwright test e2e/flows/ --project=chromium` -- all flow tests pass on chromium
|
|
2. `cd packages/portal && npx playwright test e2e/flows/` -- all flow tests pass on chromium + firefox + webkit
|
|
3. Each flow spec covers at least one happy path and one edge/error case
|
|
4. Auth setup generates 3 storageState files in playwright/.auth/
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- 7 flow spec files exist and pass on chromium
|
|
- Cross-browser (chromium + firefox + webkit) all green
|
|
- Empty/error/loading states tested within flow specs
|
|
- Auth storageState generated for 3 roles without manual intervention
|
|
- No real LLM calls in any test (WebSocket mocked)
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/09-testing-qa/09-01-SUMMARY.md`
|
|
</output>
|