Commit Graph

14 Commits

Author SHA1 Message Date
4cbf192fa5 feat(03-01): backend API endpoints — channels, billing, usage, and audit logger enhancement
- Create channels.py: HMAC-signed OAuth state generation/verification, Slack OAuth install/callback, WhatsApp manual connect, test message endpoint
- Create billing.py: Stripe Checkout session, billing portal session, webhook handler with idempotency (StripeEvent table), subscription lifecycle management
- Update usage.py: add _aggregate_rows_by_agent and _aggregate_rows_by_provider helpers (unit-testable without DB), complete usage endpoints
- Fix audit.py: rename 'metadata' attribute to 'event_metadata' (SQLAlchemy 2.0 DeclarativeBase reserves 'metadata')
- Enhance runner.py: audit log now includes prompt_tokens, completion_tokens, total_tokens, cost_usd, provider in LLM call metadata
- Update api/__init__.py to export all new routers
- All 27 unit tests passing
2026-03-23 21:24:08 -06:00
215e67a7eb feat(03-01): DB migrations, models, encryption service, and test scaffolds
- Add stripe and cryptography to shared pyproject.toml
- Add recharts, @stripe/stripe-js, stripe to portal package.json (submodule)
- Add billing fields to Tenant model (stripe_customer_id, subscription_status, agent_quota, trial_ends_at)
- Add budget_limit_usd to Agent model
- Create TenantLlmKey and StripeEvent models in billing.py (AuditBase and Base respectively)
- Create KeyEncryptionService (MultiFernet encrypt/decrypt/rotate) in crypto.py
- Create compute_budget_status helper in usage.py (threshold logic: ok/warning/exceeded)
- Add platform_encryption_key, stripe_, slack_oauth settings to config.py
- Create Alembic migration 005 with all schema changes, RLS, grants, and composite index
- All 12 tests passing (key encryption roundtrip, rotation, budget thresholds)
2026-03-23 21:19:09 -06:00
bd217a4113 feat(02-06): re-wire escalation and WhatsApp outbound routing in pipeline
- Move key imports to module level in tasks.py for testability and clarity
- Pop WhatsApp extras (phone_number_id, bot_token) in handle_message before model_validate
- Build unified extras dict and extract wa_id from sender.user_id
- Change _process_message signature to accept extras dict
- Add _build_response_extras() helper for channel-aware extras assembly
- Replace all _update_slack_placeholder calls in _process_message with _send_response()
- Add escalation pre-check: skip LLM when Redis escalation_status_key == 'escalated'
- Add escalation post-check: check_escalation_rules after run_agent; call escalate_to_human
  when rule matches and agent.escalation_assignee is set
- Add _build_conversation_metadata() helper (billing keyword v1 detection)
- Add channel parameter to build_system_prompt(), build_messages_with_memory(),
  build_messages_with_media() for WhatsApp tier-2 business-function scoping
- WhatsApp scoping appends 'You only handle: {topics}' when tool_assignments non-empty
- Pass msg.channel to build_messages_with_memory() in _process_message
- All 26 new tests pass; all existing escalation/WhatsApp tests pass (no regressions)
2026-03-23 19:15:20 -06:00
77c9cfc825 test(02-06): add failing tests for escalation wiring and WhatsApp outbound routing
- Tests for handle_message WhatsApp extra extraction (phone_number_id, bot_token)
- Tests for _send_response routing to Slack and WhatsApp
- Tests for _process_message using _send_response (not _update_slack_placeholder directly)
- Tests for escalation pre-check (skip LLM when already escalated)
- Tests for escalation post-check (check_escalation_rules + escalate_to_human)
- Tests for _build_conversation_metadata billing keyword extraction
- Tests for build_system_prompt WhatsApp tier-2 scoping (Task 2)
- Tests for build_messages_with_memory channel parameter passthrough
2026-03-23 19:08:59 -06:00
669c0b52b3 feat(02-05): multimodal LLM interpretation with image_url content blocks
- Add supports_vision(model_name) to builder.py — detects vision-capable models
  (claude-3*, gpt-4o*, gpt-4-vision*, gemini-pro-vision*, gemini-1.5*, gemini-2*)
  with provider prefix stripping support
- Add generate_presigned_url(storage_key, expiry=3600) to builder.py — generates
  1-hour MinIO presigned URLs via boto3 S3 client
- Add build_messages_with_media() to builder.py — extends build_messages_with_memory()
  with media injection: IMAGE -> image_url blocks for vision models / text fallback for
  non-vision models, DOCUMENT -> text reference with presigned URL
- image_url blocks use 'detail: auto' per OpenAI/LiteLLM multipart format
- Add 27 unit tests in test_multimodal_messages.py (TDD)
2026-03-23 15:09:18 -06:00
9dd7c481a3 feat(02-05): Slack file_share extraction and channel-aware outbound routing
- Add gateway/channels/slack_media.py with is_file_share_event, media_type_from_mime,
  build_slack_storage_key, build_attachment_from_slack_file, download_and_store_slack_file
- Add _send_response() helper to orchestrator/tasks.py for channel-aware dispatch
  (Slack -> chat.update, WhatsApp -> send_whatsapp_message)
- Add send_whatsapp_message import to orchestrator/tasks.py for WhatsApp outbound
- Add boto3>=1.35.0 to gateway dependencies for MinIO S3 client
- Add 23 unit tests in test_slack_media.py (TDD)
2026-03-23 15:06:45 -06:00
f49927888e feat(02-02): tool registry, executor, and 4 built-in tools
- ToolDefinition Pydantic model with JSON Schema parameters + handler
- BUILTIN_TOOLS: web_search, kb_search, http_request, calendar_lookup
- http_request requires_confirmation=True (outbound side effects)
- get_tools_for_agent filters by agent.tool_assignments
- to_litellm_format converts to OpenAI function-calling schema
- execute_tool: jsonschema validation before handler call
- execute_tool: confirmation gate for requires_confirmation=True
- execute_tool: audit logging on every invocation (success + failure)
- web_search: Brave Search API with BRAVE_API_KEY env var
- kb_search: pgvector cosine similarity with HNSW index
- http_request: 30s timeout, 1MB cap, GET/POST/PUT/DELETE only
- calendar_lookup: Google Calendar events.list read-only
- jsonschema dependency added to orchestrator pyproject.toml
- [Rule 1 - Bug] Added missing execute_tool import in test
2026-03-23 14:54:14 -06:00
420294b8fe test(02-02): add failing tool registry and executor unit tests
- Tests for BUILTIN_TOOLS (4 tools present, correct fields, confirmation flags)
- Tests for get_tools_for_agent filtering and to_litellm_format conversion
- Tests for execute_tool: valid args, invalid args, unknown tool, confirmation flow
- Tests for audit logger called on every invocation
2026-03-23 14:51:42 -06:00
d489551130 test(02-04): add failing tests for escalation handler
- Unit tests: rule matching, natural language escalation, transcript formatting
- Integration tests: Slack API calls, Redis key, audit log, return value
2026-03-23 14:49:54 -06:00
6fea34db28 feat(02-03): WhatsApp adapter with business-function scoping and router registration
- Register whatsapp_router in gateway main.py (GET + POST /whatsapp/webhook)
- Implement is_clearly_off_topic() tier 1 keyword scoping gate
- Implement build_off_topic_reply() canned redirect message builder
- Full webhook handler: verify -> normalize -> tenant -> rate limit -> dedup -> scope -> media -> dispatch
- Outbound delivery via send_whatsapp_message() and send_whatsapp_media()
- Media download from Meta API and storage in MinIO with tenant-prefixed keys
- 14 new passing scoping tests
2026-03-23 14:43:04 -06:00
28a5ee996e feat(02-01): add two-layer memory system — Redis sliding window + pgvector long-term
- ConversationEmbedding ORM model with Vector(384) column (pgvector)
- memory_short_key, escalation_status_key, pending_tool_confirm_key in redis_keys.py
- orchestrator/memory/short_term.py: RPUSH/LTRIM sliding window (get_recent_messages, append_message)
- orchestrator/memory/long_term.py: pgvector HNSW cosine search (retrieve_relevant, store_embedding)
- Migration 002: conversation_embeddings table, HNSW index, RLS with FORCE, SELECT/INSERT only
- 10 unit tests (fakeredis), 6 integration tests (pgvector) — all passing
- Auto-fix [Rule 3]: postgres image updated to pgvector/pgvector:pg16 (extension required)
2026-03-23 14:41:57 -06:00
370a860622 feat(02-03): add MediaAttachment model, WhatsApp normalizer, and signature verification
- Add MediaType(StrEnum) and MediaAttachment(BaseModel) to shared/models/message.py
- Add media: list[MediaAttachment] field to MessageContent
- Add whatsapp_app_secret, whatsapp_verify_token, and MinIO settings to shared/config.py
- Add normalize_whatsapp_event() to gateway/normalize.py (text, image, document support)
- Create whatsapp.py adapter with verify_whatsapp_signature() and verify_hub_challenge()
- 30 new passing tests (signature verification + normalizer)
2026-03-23 14:41:48 -06:00
74326dfc3d feat(01-03): integration tests for Slack flow, rate limiting, and agent persona
- tests/unit/test_ratelimit.py: 11 tests for Redis token bucket (CHAN-05)
  - allows requests under limit, rejects 31st request
  - per-tenant isolation, per-channel isolation
  - TTL key expiry and window reset
- tests/integration/test_slack_flow.py: 15 tests for end-to-end Slack flow (CHAN-02)
  - normalization: bot token stripped, channel=slack, thread_id set
  - @mention: placeholder posted in-thread, Celery dispatched with placeholder_ts
  - DM flow: same pipeline triggered for channel_type=im
  - bot messages silently ignored (no infinite loop)
  - unknown workspace_id silently ignored
  - duplicate events (Slack retries) skipped via idempotency
- tests/integration/test_agent_persona.py: 15 tests for persona in prompts (AGNT-01)
  - system prompt contains name, role, persona, AI transparency clause
  - model_preference forwarded to LLM pool
  - full messages array: [system, user] structure verified
- tests/integration/test_ratelimit.py: 4 tests for rate limit integration
  - over-limit -> ephemeral rejection posted
  - over-limit -> Celery NOT dispatched, placeholder NOT posted
  - within-limit -> no rejection
  - ephemeral message includes actionable retry hint
All 45 tests pass
2026-03-23 10:32:48 -06:00
47e78627fd feat(01-foundation-01): Alembic migrations with RLS and tenant isolation tests
- alembic.ini + migrations/env.py: async SQLAlchemy migration setup using asyncpg
- migrations/versions/001_initial_schema.py: creates tenants, agents, channel_connections, portal_users
  - ENABLE + FORCE ROW LEVEL SECURITY on agents and channel_connections
  - RLS policy: tenant_id = current_setting('app.current_tenant', TRUE)::uuid
  - konstruct_app role created with SELECT/INSERT/UPDATE/DELETE on all tables
- packages/shared/shared/rls.py: idempotent configure_rls_hook, UUID-sanitized SET LOCAL
- tests/conftest.py: test_db_name (session-scoped), db_engine + db_session as konstruct_app
- tests/unit/test_normalize.py: 11 tests for KonstructMessage Slack normalization (CHAN-01)
- tests/unit/test_tenant_resolution.py: 7 tests for workspace_id → tenant resolution (TNNT-02)
- tests/unit/test_redis_namespacing.py: 15 tests for Redis key namespace isolation (TNNT-03)
- tests/integration/test_tenant_isolation.py: 7 tests proving RLS tenant isolation (TNNT-01)
  - tenant_b cannot see tenant_a's agents or channel_connections
  - FORCE ROW LEVEL SECURITY verified via pg_class.relforcerowsecurity
2026-03-23 09:57:29 -06:00