konstruct/.planning/phases/03-operator-experience/03-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created

phase	slug	status	nyquist_compliant	wave_0_complete	created
3	operator-experience	draft	false	false	2026-03-24

Phase 3 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

---

Test Infrastructure

Property	Value
Framework	pytest 8.x + pytest-asyncio (existing)
Config file	pyproject.toml (existing)
Quick run command	pytest tests/unit -x -q
Full suite command	pytest tests/ -x
Estimated runtime	~45 seconds

---

Sampling Rate

• After every task commit: Run pytest tests/unit -x -q
• After every plan wave: Run pytest tests/ -x
• Before /gsd:verify-work: Full suite must be green
• Max feedback latency: 45 seconds

---

Per-Task Verification Map

Task ID	Plan	Wave	Requirement	Test Type	Automated Command	File Exists	Status
03-xx	01	1	PRTA-03	unit	pytest tests/unit/test_slack_oauth.py -x	❌ W0	⬜ pending
03-xx	01	1	PRTA-03	integration	pytest tests/integration/test_slack_oauth.py -x	❌ W0	⬜ pending
03-xx	01	1	PRTA-04	integration	pytest tests/integration/test_channel_test_message.py -x	❌ W0	⬜ pending
03-xx	02	1	PRTA-05	unit	pytest tests/unit/test_stripe_webhooks.py -x	❌ W0	⬜ pending
03-xx	02	1	LLM-03	unit	pytest tests/unit/test_key_encryption.py -x	❌ W0	⬜ pending
03-xx	02	1	AGNT-07	unit	pytest tests/unit/test_usage_aggregation.py tests/unit/test_budget_alerts.py -x	❌ W0	⬜ pending
03-xx	02	1	PRTA-06	unit	pytest tests/unit/test_usage_aggregation.py -x	❌ W0	⬜ pending

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

---

Wave 0 Requirements

• tests/unit/test_key_encryption.py — LLM-03: Fernet encrypt/decrypt/rotate
• tests/unit/test_slack_oauth.py — PRTA-03: state HMAC generation/verification
• tests/unit/test_stripe_webhooks.py — PRTA-05: idempotency, status updates, cancellation
• tests/unit/test_usage_aggregation.py — AGNT-07, PRTA-06: SQL aggregate queries
• tests/unit/test_budget_alerts.py — AGNT-07: threshold logic
• tests/integration/test_slack_oauth.py — PRTA-03: full OAuth callback flow
• tests/integration/test_channel_test_message.py — PRTA-04: test message endpoint
• Install: uv add stripe cryptography in shared, npm install recharts @stripe/stripe-js stripe in portal

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
Onboarding stepper transitions through all 3 steps	PRTA-04	UI flow requires browser	Walk through onboarding wizard: connect → configure → test
Slack OAuth redirect and token exchange	PRTA-03	Requires live Slack app + OAuth redirect	Click "Add to Slack", authorize, verify channel_connection created
Stripe Checkout session and billing portal	PRTA-05	Requires live Stripe test keys	Subscribe, upgrade agent count, cancel, verify webhook events

---

Validation Sign-Off

• All tasks have <automated> verify or Wave 0 dependencies
• Sampling continuity: no 3 consecutive tasks without automated verify
• Wave 0 covers all MISSING references
• No watch-mode flags
• Feedback latency < 45s
• nyquist_compliant: true set in frontmatter

Approval: pending