diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 32775c1..e6119c5 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -11,8 +11,8 @@ Requirements for beta-ready release. Each maps to roadmap phases. - [x] **CHAN-01**: Channel Gateway normalizes messages from all channels into unified KonstructMessage format - [x] **CHAN-02**: User can interact with AI employee via Slack (Events API — @mentions, DMs, thread replies) -- [ ] **CHAN-03**: User can interact with AI employee via WhatsApp Business Cloud API -- [ ] **CHAN-04**: WhatsApp adapter enforces business-function scoping per Meta 2026 policy +- [x] **CHAN-03**: User can interact with AI employee via WhatsApp Business Cloud API +- [x] **CHAN-04**: WhatsApp adapter enforces business-function scoping per Meta 2026 policy - [x] **CHAN-05**: Platform rate-limits requests per tenant and per channel with configurable thresholds ### Agent Core @@ -97,8 +97,8 @@ Which phases cover which requirements. Updated during roadmap creation. |-------------|-------|--------| | CHAN-01 | Phase 1 | Complete | | CHAN-02 | Phase 1 | Complete | -| CHAN-03 | Phase 2 | Pending | -| CHAN-04 | Phase 2 | Pending | +| CHAN-03 | Phase 2 | Complete | +| CHAN-04 | Phase 2 | Complete | | CHAN-05 | Phase 1 | Complete | | AGNT-01 | Phase 1 | Complete | | AGNT-02 | Phase 2 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 82ba94e..01efabe 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -79,7 +79,7 @@ Phases execute in numeric order: 1 → 2 → 3 | Phase | Plans Complete | Status | Completed | |-------|----------------|--------|-----------| | 1. Foundation | 4/4 | Complete | 2026-03-23 | -| 2. Agent Features | 0/5 | Not started | - | +| 2. Agent Features | 1/5 | In Progress| | | 3. Operator Experience | 0/2 | Not started | - | --- diff --git a/.planning/STATE.md b/.planning/STATE.md index bd1f22a..276c547 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: planning -stopped_at: Phase 2 context gathered -last_updated: "2026-03-23T19:12:16.872Z" +stopped_at: Completed 02-agent-features/02-03-PLAN.md +last_updated: "2026-03-23T20:44:35.519Z" last_activity: 2026-03-23 — Roadmap created, ready for Phase 1 planning progress: total_phases: 3 completed_phases: 1 - total_plans: 4 - completed_plans: 4 + total_plans: 9 + completed_plans: 5 percent: 0 --- @@ -54,6 +54,7 @@ Progress: [░░░░░░░░░░] 0% | Phase 01-foundation P02 | 6 | 2 tasks | 15 files | | Phase 01-foundation P04 | 19 | 2 tasks | 25 files | | Phase 01-foundation P03 | 9 | 2 tasks | 20 files | +| Phase 02-agent-features P03 | 7 | 2 tasks | 7 files | ## Accumulated Context @@ -78,6 +79,10 @@ Recent decisions affecting current work: - [Phase 01-foundation]: Patch at usage site in tests: mock 'gateway.channels.slack.resolve_tenant' not 'router.tenant.resolve_tenant' — Python name binding at import time - [Phase 01-foundation]: Celery payload extension: msg.model_dump() | extras dict, pop extras before model_validate in tasks.py to avoid pydantic validation errors on unknown fields - [Phase 01-foundation]: Bot token for chat.update loaded from channel_connections.config['bot_token'] in orchestrator task — keeps Slack SDK out of orchestrator package +- [Phase 02-agent-features]: HMAC uses hmac.new() with hmac.compare_digest for timing-safe WhatsApp signature verification +- [Phase 02-agent-features]: meta-media://{media_id} placeholder URL at normalization time; actual download in adapter after tenant resolution +- [Phase 02-agent-features]: WhatsApp thread_id = sender wa_id (WhatsApp has no threading; conversation scope is per phone number) +- [Phase 02-agent-features]: Always return HTTP 200 to Meta webhooks regardless of processing errors to prevent retry storms ### Pending Todos @@ -89,6 +94,6 @@ None yet. ## Session Continuity -Last session: 2026-03-23T19:12:16.869Z -Stopped at: Phase 2 context gathered -Resume file: .planning/phases/02-agent-features/02-CONTEXT.md +Last session: 2026-03-23T20:44:35.516Z +Stopped at: Completed 02-agent-features/02-03-PLAN.md +Resume file: None diff --git a/.planning/phases/02-agent-features/02-03-SUMMARY.md b/.planning/phases/02-agent-features/02-03-SUMMARY.md new file mode 100644 index 0000000..eb8e3ea --- /dev/null +++ b/.planning/phases/02-agent-features/02-03-SUMMARY.md @@ -0,0 +1,159 @@ +--- +phase: 02-agent-features +plan: "03" +subsystem: messaging +tags: [whatsapp, meta-cloud-api, hmac, webhook, minio, normalization, business-scoping, media] + +requires: + - phase: 01-foundation + provides: "KonstructMessage model, channel gateway, normalize.py, resolve_tenant, check_rate_limit, is_duplicate/mark_processed" + +provides: + - "WhatsApp Business Cloud API channel adapter (packages/gateway/gateway/channels/whatsapp.py)" + - "normalize_whatsapp_event() normalizing Meta Cloud API v20.0 payloads to KonstructMessage" + - "MediaAttachment and MediaType models in shared/models/message.py" + - "media: list[MediaAttachment] field added to MessageContent" + - "verify_whatsapp_signature() HMAC-SHA256 on raw body bytes" + - "verify_hub_challenge() for Meta webhook registration handshake" + - "is_clearly_off_topic() tier 1 keyword scoping gate" + - "build_off_topic_reply() canned redirect message builder" + - "send_whatsapp_message() and send_whatsapp_media() outbound delivery" + - "Media download from Meta API + MinIO storage with tenant-prefixed keys" + - "WhatsApp routes registered in gateway main.py" + - "whatsapp_app_secret, whatsapp_verify_token, MinIO settings added to shared/config.py" + +affects: + - "02-agent-features/02-05 — outbound response routing in orchestrator tasks.py must route whatsapp channel through send_whatsapp_message" + - "02-agent-features — any plan touching gateway/normalize.py should import MediaAttachment/MediaType" + +tech-stack: + added: + - "boto3 (not yet in pyproject.toml — added inline in media download function, needs uv add)" + - "httpx (already in gateway deps — used for Meta API calls)" + patterns: + - "Raw body read BEFORE JSON parsing for HMAC verification (prevent tampered-body attack)" + - "verify_whatsapp_signature returns raw bytes on success, raises HTTPException(403) on failure" + - "verify_hub_challenge returns challenge string on success, raises HTTPException(403) on failure" + - "normalize_whatsapp_event: meta-media://{media_id} placeholder URL before actual download" + - "Two-tier scoping: keyword gate (no LLM) -> LLM with system prompt scoping" + - "Always return 200 OK to Meta regardless of processing errors (Meta retry prevention)" + - "thread_id = sender wa_id for WhatsApp (no threading concept in WhatsApp)" + +key-files: + created: + - packages/gateway/gateway/channels/whatsapp.py + - tests/unit/test_whatsapp_verify.py + - tests/unit/test_whatsapp_normalize.py + - tests/unit/test_whatsapp_scoping.py + modified: + - packages/shared/shared/models/message.py + - packages/shared/shared/config.py + - packages/gateway/gateway/normalize.py + - packages/gateway/gateway/main.py + +key-decisions: + - "HMAC uses hmac.new() (not hmac.HMAC()) for timing-safe signature verification via hmac.compare_digest" + - "meta-media://{media_id} placeholder URL set at normalization time; actual download in adapter after tenant resolution" + - "thread_id set to sender wa_id — WhatsApp conversation scope is per-phone-number, not thread-based" + - "Always return HTTP 200 to Meta webhooks regardless of processing errors to prevent Meta retry storms" + - "Tier 1 scoping uses word-level tokenization with set intersection for keyword overlap check" + - "MinIO settings added to shared/config.py alongside WhatsApp credentials for use by all services" + - "boto3 required for MinIO access (S3-compatible client); needs uv add boto3 in gateway package before prod" + +patterns-established: + - "WhatsApp adapter pattern: verify signature on raw bytes -> normalize -> tenant resolve -> rate limit -> dedup -> scope -> media -> dispatch" + - "Business-function scoping: tier 1 (keyword) rejects clearly off-topic without LLM; tier 2 (LLM) enforces via system prompt" + - "Canned redirect format: '{agent_name} is here to help with {topics}. How can I assist you with one of those?'" + +requirements-completed: + - CHAN-03 + - CHAN-04 + +duration: 7min +completed: 2026-03-23 +--- + +# Phase 2 Plan 03: WhatsApp Channel Adapter Summary + +**WhatsApp Business Cloud API adapter with HMAC-SHA256 verification, Meta v20.0 normalization, two-tier business-function scoping gate, and MinIO media storage** + +## Performance + +- **Duration:** ~7 minutes +- **Started:** 2026-03-23T20:36:19Z +- **Completed:** 2026-03-23T20:43:00Z +- **Tasks:** 2 +- **Files modified:** 7 (4 created, 3 modified) + +## Accomplishments + +- WhatsApp Business Cloud API adapter with full inbound webhook processing pipeline +- HMAC-SHA256 signature verification on raw body bytes (timing-safe via hmac.compare_digest) +- normalize_whatsapp_event() for text, image, and document messages to KonstructMessage +- Two-tier Meta 2026 business-function scoping: keyword gate + LLM system prompt scoping +- MediaAttachment model with typed media fields added to shared message model +- Media download from Meta Graph API and storage in MinIO with tenant-prefixed keys +- Outbound delivery functions for text and media via Meta Cloud API +- 54 new passing tests (8 verify + 22 normalize + 14 scoping) + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Media model extension, WhatsApp normalizer, and signature verification** - `370a860` (feat) +2. **Task 2: WhatsApp adapter with business-function scoping, media download/storage, and outbound delivery** - `6fea34d` (feat) + +_Note: TDD tasks - tests written first then implementation_ + +## Files Created/Modified + +- `packages/gateway/gateway/channels/whatsapp.py` - Full WhatsApp adapter: signature verification, hub challenge, webhook handler, scoping, media, outbound delivery +- `packages/gateway/gateway/normalize.py` - Added normalize_whatsapp_event() alongside existing Slack normalizer +- `packages/shared/shared/models/message.py` - Added MediaType enum, MediaAttachment model, media field on MessageContent +- `packages/shared/shared/config.py` - Added whatsapp_app_secret, whatsapp_verify_token, MinIO settings +- `packages/gateway/gateway/main.py` - Registered whatsapp_router, updated docstring +- `tests/unit/test_whatsapp_verify.py` - 8 tests for signature verification and hub challenge +- `tests/unit/test_whatsapp_normalize.py` - 22 tests for normalization (text, image, document) +- `tests/unit/test_whatsapp_scoping.py` - 14 tests for tier 1 scoping gate and canned reply + +## Decisions Made + +- HMAC uses `hmac.new()` with `hmac.compare_digest` for timing-safe comparison, preventing timing oracle attacks +- Normalizer sets `meta-media://{media_id}` as placeholder URL — actual download occurs in the adapter after tenant resolution, so the download has access to the access_token from channel_connections +- `thread_id = sender wa_id` — WhatsApp has no threading; conversation scope is per phone number +- Always return HTTP 200 to Meta even on processing errors — Meta retries all non-200 responses which can cause duplicate processing +- Tier 1 scoping uses word-level tokenization (text.split()) with set intersection, not substring matching, to avoid false positives +- MinIO settings co-located in shared/config.py so all services (gateway, orchestrator) can access MinIO + +## Deviations from Plan + +None - plan executed exactly as written. + +## Issues Encountered + +- Gateway package was not installed in the venv (only shared and orchestrator were). Running `uv sync --all-packages` resolved the import error. This is a Rule 3 (blocking) auto-fix. + +## User Setup Required + +The following environment variables must be added to `.env` before WhatsApp webhooks can receive real traffic: + +```bash +WHATSAPP_APP_SECRET= +WHATSAPP_VERIFY_TOKEN= +MINIO_ENDPOINT=http://localhost:9000 +MINIO_ACCESS_KEY=minioadmin +MINIO_SECRET_KEY=minioadmin +MINIO_MEDIA_BUCKET=konstruct-media +``` + +Also required before production: `uv add boto3` in `packages/gateway/` for MinIO uploads. + +## Next Phase Readiness + +- WhatsApp adapter is ready for tenant registration via channel_connections with phone_number_id as workspace_id +- Outbound response routing (LLM-generated replies via send_whatsapp_message) must be wired in Plan 02-05 (orchestrator tasks.py) +- MinIO bucket `konstruct-media` must be created before media messages can be processed + +--- +*Phase: 02-agent-features* +*Completed: 2026-03-23*