Commit Graph

25 Commits

Author SHA1 Message Date
9654982433 feat(07-01): localized emails, locale-aware templates API, language preference endpoint
- email.py: send_invite_email() adds language param (en/es/pt), sends localized subject+body
- templates.py: list_templates()/get_template() accept ?locale= param, merge translations on response
- portal.py: PATCH /api/portal/users/me/language endpoint persists language preference
- portal.py: /api/portal/auth/verify response includes user.language field
- portal.py: AuthVerifyResponse adds language field (default 'en')
- test_portal_auth.py: fix _make_user mock to set language='en' (auto-fix Rule 1)
- test_language_preference.py: 4 integration tests for language preference endpoint
- test_templates_i18n.py: 5 integration tests for locale-aware templates (all passing)
2026-03-25 16:27:14 -06:00
7a3a4f0fdd feat(07-01): DB migration 009, ORM updates, and LANGUAGE_INSTRUCTION in system prompts
- Migration 009: adds language col (VARCHAR 10, NOT NULL, default 'en') to portal_users
- Migration 009: adds translations col (JSONB, NOT NULL, default '{}') to agent_templates
- Migration 009: backfills es+pt translations for all 7 seed templates
- PortalUser ORM: language mapped column added
- AgentTemplate ORM: translations mapped column added
- system_prompt_builder.py: LANGUAGE_INSTRUCTION constant + appended before AI_TRANSPARENCY_CLAUSE
- system-prompt-builder.ts: LANGUAGE_INSTRUCTION constant + appended before AI transparency clause
- tests: TestLanguageInstruction class with 3 tests (all pass, 20 total)
2026-03-25 16:22:53 -06:00
c72beb916b feat(06-01): add web channel type, Redis key, ORM models, migration, and tests
- Add ChannelType.WEB = 'web' to shared/models/message.py
- Add webchat_response_key() to shared/redis_keys.py
- Create WebConversation and WebConversationMessage ORM models (SQLAlchemy 2.0)
- Create migration 008_web_chat.py with RLS, indexes, and channel_type CHECK update
- Pop conversation_id/portal_user_id extras in handle_message before model_validate
- Add web case to _build_response_extras and _send_response (Redis pub-sub publish)
- Import webchat_response_key in orchestrator/tasks.py
- Write 19 unit tests covering CHAT-01 through CHAT-05 (all pass)
2026-03-25 10:26:34 -06:00
f9ce3d650f feat(05-01): template list/detail/deploy API + RBAC + integration tests
- Create shared/api/templates.py with templates_router
- GET /api/portal/templates: list active templates (any authenticated user)
- GET /api/portal/templates/{id}: get template detail (any authenticated user)
- POST /api/portal/templates/{id}/deploy: create Agent snapshot (tenant_admin only)
- customer_operator returns 403 on deploy (RBAC enforced)
- Export templates_router from shared/api/__init__.py
- Mount templates_router in gateway/main.py (Phase 5 section)
- 11 integration tests pass (list, detail, deploy, RBAC, 404, snapshot independence)
2026-03-24 20:32:30 -06:00
d1acb292a1 feat(05-01): AgentTemplate ORM model, migration 007, and system prompt builder
- Add AgentTemplate ORM model to tenant.py (global, not tenant-scoped)
- Create migration 007 with agent_templates table and 7 seed templates
- Create shared/prompts/system_prompt_builder.py with build_system_prompt()
- AI transparency clause always present (non-negotiable per Phase 1 decision)
- Unit tests pass (17 tests, all sections verified)
2026-03-24 20:27:54 -06:00
9515c5374a test(04-rbac-03): add failing integration tests for RBAC enforcement and invite flow
RED phase — tests are written, will pass when connected to live DB.
Tests cover:
- Full RBAC matrix: platform_admin/customer_admin/operator on all endpoints
- Operator can POST /test but not POST /agents (create)
- Missing headers return 422
- Impersonation creates AuditEvent row
- Full invite flow: create -> accept -> login with correct role
- Expired invite rejection
- Resend generates new token and extends expiry
- Double-accept prevention
2026-03-24 17:16:13 -06:00
7b0594e7cc test(04-rbac-01): unit tests for RBAC guards, invitation system, portal auth
- test_rbac_guards.py: 11 tests covering platform_admin pass-through,
  customer_admin/operator 403 rejection, tenant membership checks,
  and platform_admin bypass for tenant-scoped guards
- test_invitations.py: 11 tests covering HMAC token roundtrip,
  tamper/expiry rejection, invitation create/accept/resend/list
- test_portal_auth.py: 7 tests covering role field (not is_admin),
  tenant_ids list, active_tenant_id, platform_admin all-tenants,
  customer_admin own-tenants-only
- All 27 tests pass
2026-03-24 13:55:55 -06:00
3c8fc255bc feat(03-01): LLM key CRUD API endpoints with encryption
- Create llm_keys.py: GET list (redacted, key_hint only), POST (encrypt + store), DELETE (204 or 404)
- LlmKeyResponse never exposes encrypted_key or raw api_key
- 409 returned on duplicate (tenant_id, provider) key
- Cross-tenant deletion prevented by tenant_id verification in DELETE query
- Update api/__init__.py to export llm_keys_router
- All 5 LLM key CRUD tests passing (32 total unit tests green)
2026-03-23 21:36:08 -06:00
4cbf192fa5 feat(03-01): backend API endpoints — channels, billing, usage, and audit logger enhancement
- Create channels.py: HMAC-signed OAuth state generation/verification, Slack OAuth install/callback, WhatsApp manual connect, test message endpoint
- Create billing.py: Stripe Checkout session, billing portal session, webhook handler with idempotency (StripeEvent table), subscription lifecycle management
- Update usage.py: add _aggregate_rows_by_agent and _aggregate_rows_by_provider helpers (unit-testable without DB), complete usage endpoints
- Fix audit.py: rename 'metadata' attribute to 'event_metadata' (SQLAlchemy 2.0 DeclarativeBase reserves 'metadata')
- Enhance runner.py: audit log now includes prompt_tokens, completion_tokens, total_tokens, cost_usd, provider in LLM call metadata
- Update api/__init__.py to export all new routers
- All 27 unit tests passing
2026-03-23 21:24:08 -06:00
215e67a7eb feat(03-01): DB migrations, models, encryption service, and test scaffolds
- Add stripe and cryptography to shared pyproject.toml
- Add recharts, @stripe/stripe-js, stripe to portal package.json (submodule)
- Add billing fields to Tenant model (stripe_customer_id, subscription_status, agent_quota, trial_ends_at)
- Add budget_limit_usd to Agent model
- Create TenantLlmKey and StripeEvent models in billing.py (AuditBase and Base respectively)
- Create KeyEncryptionService (MultiFernet encrypt/decrypt/rotate) in crypto.py
- Create compute_budget_status helper in usage.py (threshold logic: ok/warning/exceeded)
- Add platform_encryption_key, stripe_, slack_oauth settings to config.py
- Create Alembic migration 005 with all schema changes, RLS, grants, and composite index
- All 12 tests passing (key encryption roundtrip, rotation, budget thresholds)
2026-03-23 21:19:09 -06:00
bd217a4113 feat(02-06): re-wire escalation and WhatsApp outbound routing in pipeline
- Move key imports to module level in tasks.py for testability and clarity
- Pop WhatsApp extras (phone_number_id, bot_token) in handle_message before model_validate
- Build unified extras dict and extract wa_id from sender.user_id
- Change _process_message signature to accept extras dict
- Add _build_response_extras() helper for channel-aware extras assembly
- Replace all _update_slack_placeholder calls in _process_message with _send_response()
- Add escalation pre-check: skip LLM when Redis escalation_status_key == 'escalated'
- Add escalation post-check: check_escalation_rules after run_agent; call escalate_to_human
  when rule matches and agent.escalation_assignee is set
- Add _build_conversation_metadata() helper (billing keyword v1 detection)
- Add channel parameter to build_system_prompt(), build_messages_with_memory(),
  build_messages_with_media() for WhatsApp tier-2 business-function scoping
- WhatsApp scoping appends 'You only handle: {topics}' when tool_assignments non-empty
- Pass msg.channel to build_messages_with_memory() in _process_message
- All 26 new tests pass; all existing escalation/WhatsApp tests pass (no regressions)
2026-03-23 19:15:20 -06:00
77c9cfc825 test(02-06): add failing tests for escalation wiring and WhatsApp outbound routing
- Tests for handle_message WhatsApp extra extraction (phone_number_id, bot_token)
- Tests for _send_response routing to Slack and WhatsApp
- Tests for _process_message using _send_response (not _update_slack_placeholder directly)
- Tests for escalation pre-check (skip LLM when already escalated)
- Tests for escalation post-check (check_escalation_rules + escalate_to_human)
- Tests for _build_conversation_metadata billing keyword extraction
- Tests for build_system_prompt WhatsApp tier-2 scoping (Task 2)
- Tests for build_messages_with_memory channel parameter passthrough
2026-03-23 19:08:59 -06:00
669c0b52b3 feat(02-05): multimodal LLM interpretation with image_url content blocks
- Add supports_vision(model_name) to builder.py — detects vision-capable models
  (claude-3*, gpt-4o*, gpt-4-vision*, gemini-pro-vision*, gemini-1.5*, gemini-2*)
  with provider prefix stripping support
- Add generate_presigned_url(storage_key, expiry=3600) to builder.py — generates
  1-hour MinIO presigned URLs via boto3 S3 client
- Add build_messages_with_media() to builder.py — extends build_messages_with_memory()
  with media injection: IMAGE -> image_url blocks for vision models / text fallback for
  non-vision models, DOCUMENT -> text reference with presigned URL
- image_url blocks use 'detail: auto' per OpenAI/LiteLLM multipart format
- Add 27 unit tests in test_multimodal_messages.py (TDD)
2026-03-23 15:09:18 -06:00
9dd7c481a3 feat(02-05): Slack file_share extraction and channel-aware outbound routing
- Add gateway/channels/slack_media.py with is_file_share_event, media_type_from_mime,
  build_slack_storage_key, build_attachment_from_slack_file, download_and_store_slack_file
- Add _send_response() helper to orchestrator/tasks.py for channel-aware dispatch
  (Slack -> chat.update, WhatsApp -> send_whatsapp_message)
- Add send_whatsapp_message import to orchestrator/tasks.py for WhatsApp outbound
- Add boto3>=1.35.0 to gateway dependencies for MinIO S3 client
- Add 23 unit tests in test_slack_media.py (TDD)
2026-03-23 15:06:45 -06:00
f49927888e feat(02-02): tool registry, executor, and 4 built-in tools
- ToolDefinition Pydantic model with JSON Schema parameters + handler
- BUILTIN_TOOLS: web_search, kb_search, http_request, calendar_lookup
- http_request requires_confirmation=True (outbound side effects)
- get_tools_for_agent filters by agent.tool_assignments
- to_litellm_format converts to OpenAI function-calling schema
- execute_tool: jsonschema validation before handler call
- execute_tool: confirmation gate for requires_confirmation=True
- execute_tool: audit logging on every invocation (success + failure)
- web_search: Brave Search API with BRAVE_API_KEY env var
- kb_search: pgvector cosine similarity with HNSW index
- http_request: 30s timeout, 1MB cap, GET/POST/PUT/DELETE only
- calendar_lookup: Google Calendar events.list read-only
- jsonschema dependency added to orchestrator pyproject.toml
- [Rule 1 - Bug] Added missing execute_tool import in test
2026-03-23 14:54:14 -06:00
420294b8fe test(02-02): add failing tool registry and executor unit tests
- Tests for BUILTIN_TOOLS (4 tools present, correct fields, confirmation flags)
- Tests for get_tools_for_agent filtering and to_litellm_format conversion
- Tests for execute_tool: valid args, invalid args, unknown tool, confirmation flow
- Tests for audit logger called on every invocation
2026-03-23 14:51:42 -06:00
d489551130 test(02-04): add failing tests for escalation handler
- Unit tests: rule matching, natural language escalation, transcript formatting
- Integration tests: Slack API calls, Redis key, audit log, return value
2026-03-23 14:49:54 -06:00
df7a5a922f test(02-02): add failing audit integration tests
- Tests for AuditLogger.log_llm_call, log_tool_call, log_escalation
- Tests for audit_events immutability (UPDATE/DELETE rejection)
- Tests for RLS tenant isolation
2026-03-23 14:48:52 -06:00
6fea34db28 feat(02-03): WhatsApp adapter with business-function scoping and router registration
- Register whatsapp_router in gateway main.py (GET + POST /whatsapp/webhook)
- Implement is_clearly_off_topic() tier 1 keyword scoping gate
- Implement build_off_topic_reply() canned redirect message builder
- Full webhook handler: verify -> normalize -> tenant -> rate limit -> dedup -> scope -> media -> dispatch
- Outbound delivery via send_whatsapp_message() and send_whatsapp_media()
- Media download from Meta API and storage in MinIO with tenant-prefixed keys
- 14 new passing scoping tests
2026-03-23 14:43:04 -06:00
28a5ee996e feat(02-01): add two-layer memory system — Redis sliding window + pgvector long-term
- ConversationEmbedding ORM model with Vector(384) column (pgvector)
- memory_short_key, escalation_status_key, pending_tool_confirm_key in redis_keys.py
- orchestrator/memory/short_term.py: RPUSH/LTRIM sliding window (get_recent_messages, append_message)
- orchestrator/memory/long_term.py: pgvector HNSW cosine search (retrieve_relevant, store_embedding)
- Migration 002: conversation_embeddings table, HNSW index, RLS with FORCE, SELECT/INSERT only
- 10 unit tests (fakeredis), 6 integration tests (pgvector) — all passing
- Auto-fix [Rule 3]: postgres image updated to pgvector/pgvector:pg16 (extension required)
2026-03-23 14:41:57 -06:00
370a860622 feat(02-03): add MediaAttachment model, WhatsApp normalizer, and signature verification
- Add MediaType(StrEnum) and MediaAttachment(BaseModel) to shared/models/message.py
- Add media: list[MediaAttachment] field to MessageContent
- Add whatsapp_app_secret, whatsapp_verify_token, and MinIO settings to shared/config.py
- Add normalize_whatsapp_event() to gateway/normalize.py (text, image, document support)
- Create whatsapp.py adapter with verify_whatsapp_signature() and verify_hub_challenge()
- 30 new passing tests (signature verification + normalizer)
2026-03-23 14:41:48 -06:00
74326dfc3d feat(01-03): integration tests for Slack flow, rate limiting, and agent persona
- tests/unit/test_ratelimit.py: 11 tests for Redis token bucket (CHAN-05)
  - allows requests under limit, rejects 31st request
  - per-tenant isolation, per-channel isolation
  - TTL key expiry and window reset
- tests/integration/test_slack_flow.py: 15 tests for end-to-end Slack flow (CHAN-02)
  - normalization: bot token stripped, channel=slack, thread_id set
  - @mention: placeholder posted in-thread, Celery dispatched with placeholder_ts
  - DM flow: same pipeline triggered for channel_type=im
  - bot messages silently ignored (no infinite loop)
  - unknown workspace_id silently ignored
  - duplicate events (Slack retries) skipped via idempotency
- tests/integration/test_agent_persona.py: 15 tests for persona in prompts (AGNT-01)
  - system prompt contains name, role, persona, AI transparency clause
  - model_preference forwarded to LLM pool
  - full messages array: [system, user] structure verified
- tests/integration/test_ratelimit.py: 4 tests for rate limit integration
  - over-limit -> ephemeral rejection posted
  - over-limit -> Celery NOT dispatched, placeholder NOT posted
  - within-limit -> no rejection
  - ephemeral message includes actionable retry hint
All 45 tests pass
2026-03-23 10:32:48 -06:00
8257c554d7 feat(01-02): Celery orchestrator — handle_message task, system prompt builder, LLM pool runner
- Create orchestrator/main.py: Celery app with Redis broker/backend, task_acks_late=True, 10-min timeout
- Create orchestrator/tasks.py: SYNC def handle_message (critical pattern: asyncio.run for async work)
  - Deserializes KonstructMessage, sets RLS context, loads agent from DB, calls run_agent
  - Retries up to 3x on deserialization failure
- Create orchestrator/agents/builder.py: build_system_prompt assembles system_prompt + identity + persona + AI transparency clause
- Create orchestrator/agents/runner.py: run_agent posts to llm-pool /complete via httpx, returns polite fallback on error
- Add Celery[redis] dependency to orchestrator pyproject.toml
- Create tests/integration/test_llm_fallback.py: 7 tests for fallback routing and 503 on total failure (LLM-01)
- Create tests/integration/test_llm_providers.py: 12 tests verifying all three providers configured correctly (LLM-02)
- All 19 integration tests pass
2026-03-23 10:06:44 -06:00
7b348b97e9 feat(01-04): FastAPI portal API endpoints with tenant/agent CRUD and auth
- Add packages/shared/shared/api/portal.py with APIRouter at /api/portal
- POST /auth/verify validates bcrypt credentials against portal_users table
- POST /auth/register creates new portal users with hashed passwords
- Tenant CRUD: GET/POST /tenants, GET/PUT/DELETE /tenants/{id}
- Agent CRUD: full CRUD under /tenants/{tenant_id}/agents/{id}
- Agent endpoints set RLS current_tenant_id context for policy compliance
- Pydantic v2 schemas with slug validation (lowercase, hyphens, 2-50 chars)
- Add bcrypt>=4.0.0 dependency to konstruct-shared
- Integration tests: 38 tests covering all CRUD, validation, and isolation
2026-03-23 10:05:07 -06:00
47e78627fd feat(01-foundation-01): Alembic migrations with RLS and tenant isolation tests
- alembic.ini + migrations/env.py: async SQLAlchemy migration setup using asyncpg
- migrations/versions/001_initial_schema.py: creates tenants, agents, channel_connections, portal_users
  - ENABLE + FORCE ROW LEVEL SECURITY on agents and channel_connections
  - RLS policy: tenant_id = current_setting('app.current_tenant', TRUE)::uuid
  - konstruct_app role created with SELECT/INSERT/UPDATE/DELETE on all tables
- packages/shared/shared/rls.py: idempotent configure_rls_hook, UUID-sanitized SET LOCAL
- tests/conftest.py: test_db_name (session-scoped), db_engine + db_session as konstruct_app
- tests/unit/test_normalize.py: 11 tests for KonstructMessage Slack normalization (CHAN-01)
- tests/unit/test_tenant_resolution.py: 7 tests for workspace_id → tenant resolution (TNNT-02)
- tests/unit/test_redis_namespacing.py: 15 tests for Redis key namespace isolation (TNNT-03)
- tests/integration/test_tenant_isolation.py: 7 tests proving RLS tenant isolation (TNNT-01)
  - tenant_b cannot see tenant_a's agents or channel_connections
  - FORCE ROW LEVEL SECURITY verified via pg_class.relforcerowsecurity
2026-03-23 09:57:29 -06:00