11 KiB
phase, verified, status, score, re_verification, human_verification
| phase | verified | status | score | re_verification | human_verification | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 05-employee-design | 2026-03-24T12:00:00Z | human_needed | 11/11 must-haves verified |
|
|
Phase 5: Employee Design Verification Report
Phase Goal: Operators and customer admins can create AI employees through a guided wizard or deploy from pre-built templates Verified: 2026-03-24 Status: human_needed — all automated checks pass; 4 items require human verification Re-verification: Yes — after gap closure (05-04 fixes applied)
Re-Verification Summary
Previous score: 9/11 (gaps_found) Current score: 11/11 (human_needed)
Gaps Closed
Gap 1 — Frontend RBAC (EMPL-04): Closed. /agents/new added to CUSTOMER_OPERATOR_RESTRICTED array in proxy.ts at line 23. The existing startsWith(${prefix}/) logic at line 59 extends this restriction to all three sub-paths (/agents/new/templates, /agents/new/wizard, /agents/new/advanced) without requiring them to be listed individually. The New Employee button in agents/page.tsx is also hidden via {role && role !== "customer_operator" && ...} at line 77 — both UI and route layers now enforce the restriction.
Gap 2 — Wizard deploy error handling: Closed. step-review.tsx catch block now re-throws (throw err at line 52) after logging. Because mutateAsync throws on rejection and the error propagates out of the catch block, TanStack Query's mutation state sets isError = true and populates error. The error display div at line 142 ({createAgent.error && ...}) will now render when deploy fails.
Regressions
None. All 9 previously verified truths remain intact — spot-checked:
templates.pyRBAC guards and endpoints unchangedsystem-prompt-builder.tstransparency clause presentuseTemplatesanduseDeployTemplatehooks at lines 404 and 411 ofqueries.tsunchanged
Goal Achievement
Observable Truths
| # | Truth | Status | Evidence |
|---|---|---|---|
| 1 | GET /api/portal/templates returns 7+ pre-built agent templates | VERIFIED | templates.py list_templates; migration 007 seeds 7 templates |
| 2 | POST /api/portal/templates/{id}/deploy creates independent agent snapshot with is_active=True | VERIFIED | deploy_template creates Agent snapshot, no FK back to template |
| 3 | Template deploy is blocked for customer_operator (403) | VERIFIED | require_tenant_admin called; integration test test_deploy_template_rbac confirms 403 |
| 4 | System prompt builder produces coherent prompt with AI transparency clause | VERIFIED | build_system_prompt() always appends AI_TRANSPARENCY_CLAUSE; 17 unit tests pass |
| 5 | New Employee button presents three options: Templates, Guided Setup, Advanced | VERIFIED | /agents/new/page.tsx renders three Cards with correct labels, icons, and routes |
| 6 | Template gallery shows card grid with name, role, tools — one-click deploy creates agent | VERIFIED | TemplateGallery uses useTemplates hook, Deploy button calls useDeployTemplate mutation |
| 7 | Wizard walks through 5 steps: Role, Persona, Tools, Channels, Escalation + Review | VERIFIED | EmployeeWizard renders all 6 step components (5 steps + review), all substantive |
| 8 | Wizard auto-generates system prompt (hidden from user) with AI transparency clause | VERIFIED | buildSystemPrompt called in step-review.tsx; catch block re-throws so createAgent.error surfaces to user |
| 9 | Advanced option opens existing Agent Designer with full control | VERIFIED | /agents/new/advanced/page.tsx renders AgentDesigner component in create mode |
| 10 | Wizard-created and template-deployed agents appear in Agent Designer for editing | VERIFIED | Both paths redirect to /agents/{id}?tenant={tenantId} on success |
| 11 | Only platform_admin and customer_admin can access creation paths | VERIFIED | proxy.ts CUSTOMER_OPERATOR_RESTRICTED includes /agents/new (covers all sub-paths via startsWith); agents/page.tsx New Employee button gated on role !== "customer_operator" |
Score: 11/11 truths verified
Gap Closure Detail
Gap 1: proxy.ts — /agents/new restriction
File: packages/portal/proxy.ts
Before (previous state): /agents/new absent from CUSTOMER_OPERATOR_RESTRICTED. Operators could navigate into all three creation sub-paths and only learned of the restriction when the final API call returned 403.
After (current state): /agents/new present in CUSTOMER_OPERATOR_RESTRICTED at line 23:
const CUSTOMER_OPERATOR_RESTRICTED = ["/billing", "/settings/api-keys", "/users", "/admin", "/agents/new"];
The restriction check at line 59 uses pathname.startsWith(${prefix}/), so /agents/new/templates, /agents/new/wizard, and /agents/new/advanced are all covered by the single entry. Operators are redirected to /agents on approach.
Gap 1b: agents/page.tsx — New Employee button gating
File: packages/portal/app/(dashboard)/agents/page.tsx
Before: Button rendered unconditionally for all roles.
After: useSession imported and role extracted at lines 66-67. Button renders only when role && role !== "customer_operator" at line 77. Operators see no button entry point.
Gap 2: step-review.tsx — error re-throw
File: packages/portal/components/wizard-steps/step-review.tsx
Before: catch block called console.error only; mutateAsync error was absorbed, mutation stayed in success state, error div never rendered.
After: catch block at lines 50-52 logs then re-throws:
console.error("Failed to deploy agent:", err);
throw err;
createAgent.isError is now set on deploy failure and the error div at line 142 renders with createAgent.error.message.
Required Artifacts (Regression Check)
| Artifact | Status | Notes |
|---|---|---|
packages/portal/proxy.ts |
VERIFIED | /agents/new now in CUSTOMER_OPERATOR_RESTRICTED |
packages/portal/app/(dashboard)/agents/page.tsx |
VERIFIED | Role check gates New Employee button |
packages/portal/components/wizard-steps/step-review.tsx |
VERIFIED | catch re-throws; error div renders on failure |
packages/shared/shared/api/templates.py |
VERIFIED (no change) | RBAC guards intact |
packages/portal/lib/system-prompt-builder.ts |
VERIFIED (no change) | Transparency clause present |
packages/portal/lib/queries.ts |
VERIFIED (no change) | useTemplates at 404, useDeployTemplate at 411 |
Requirements Coverage
| Requirement | Description | Status | Evidence |
|---|---|---|---|
| EMPL-01 | Multi-step wizard guides user through AI employee creation without knowledge of system prompt format | VERIFIED | 5-step wizard (Role, Persona, Tools, Channels, Escalation) + Review; system prompt auto-generated, hidden |
| EMPL-02 | Pre-built agent templates for one-click deployment | VERIFIED | 7 templates in migration 007; GET /api/portal/templates; TemplateGallery card grid |
| EMPL-03 | Template-deployed agents immediately functional | VERIFIED | Agent snapshot created with is_active=True; human verification in 05-03 confirmed |
| EMPL-04 | Wizard and templates accessible to platform admins and customer admins (RBAC-enforced, not operators) | VERIFIED | Backend: require_tenant_admin (403 on operator). Frontend: proxy.ts blocks /agents/new; button hidden for customer_operator |
| EMPL-05 | Agents created via wizard or template appear in Agent Designer for customization | VERIFIED | Both paths redirect to /agents/{id} on success |
Anti-Patterns Found
None remaining from previous gaps. Previously flagged items resolved:
| File | Previous Issue | Resolution |
|---|---|---|
proxy.ts |
/agents/new missing from restricted list | Fixed — added to CUSTOMER_OPERATOR_RESTRICTED |
agents/page.tsx |
New Employee button had no role check | Fixed — gated with role !== "customer_operator" |
step-review.tsx |
catch block swallowed deploy errors | Fixed — catch re-throws after console.error |
Human Verification Required
1. Three-Option Entry Screen Visual
Test: Load /agents/new and inspect card rendering at mobile (375px) and desktop (1280px) widths Expected: Templates card has "Recommended" badge and primary-colored border; all three cards are legible and buttons navigable Why human: CSS grid layout, badge positioning, and responsive breakpoints cannot be verified programmatically
2. Template Preview Dialog
Test: Click "Preview" on any template card in /agents/new/templates Expected: Dialog opens showing role, persona, model preference, tool badges, and escalation rules list; "Deploy Now" inside dialog triggers deploy and redirects to /agents/{id} Why human: Dialog open/close interaction and content rendering require visual inspection
3. Wizard Back-Navigation State Retention
Test: Complete steps 1-3 in wizard, click Back from step 3 to step 2, then Back to step 1 Expected: Each step pre-fills with previously entered data; no data loss Why human: React state retention across back-navigation requires interactive testing
4. Channels Step Empty State
Test: Open wizard with a tenant that has no active channel connections Expected: Step 4 shows "No channels connected yet. Your employee will be deployed and can be assigned to channels later." message, and Next button is still clickable Why human: Requires dev environment configured with no active channel connections
Verified: 2026-03-24 Verifier: Claude (gsd-verifier)