Commit Graph

199 Commits

Author SHA1 Message Date
9c7686a7b4 feat(10-01): Celery ingestion task, executor injection, KB search wiring
- Add ingest_document Celery task (sync def + asyncio.run per arch constraint)
- Add ingest_document_pipeline: MinIO download, extract, chunk, embed, store
- Add chunk_text sliding window chunker (500 chars default, 50 overlap)
- Update execute_tool to inject tenant_id/agent_id into all tool handler kwargs
- Update web_search to use settings.brave_api_key (shared config) not os.getenv
- Unit tests: test_ingestion.py (9 tests) and test_executor_injection.py (5 tests) all pass
2026-03-26 09:09:36 -06:00
08572fcc40 feat(10-02): Google Calendar OAuth endpoints and per-tenant calendar tool
- Add calendar_auth.py: OAuth install/callback/status endpoints with HMAC-signed state
- Replace calendar_lookup.py service account stub with per-tenant OAuth token lookup
- Support list, check_availability, and create actions with natural language responses
- Token auto-refresh: write updated credentials back to channel_connections on refresh
- Add migration 013: add google_calendar to channel_type CHECK constraint
- Add unit tests: 16 tests covering all actions, not-connected path, token refresh write-back
2026-03-26 09:07:37 -06:00
e8d3e8a108 feat(10-01): KB ingestion pipeline - migration, extractors, API router
- Migration 014: add status/error_message/chunk_count to kb_documents, make agent_id nullable
- Add GOOGLE_CALENDAR to ChannelTypeEnum in tenant.py
- Add brave_api_key, firecrawl_api_key, google_client_id/secret, minio_kb_bucket to config
- Add text extractors for PDF, DOCX, PPTX, XLSX/XLS, CSV, TXT, MD
- Add KB management API router with upload, list, delete, URL ingest, reindex endpoints
- Install pypdf, python-docx, python-pptx, openpyxl, pandas, firecrawl-py, youtube-transcript-api
- Update .env.example with new env vars
- Unit tests: test_extractors.py (10 tests) and test_kb_upload.py (7 tests) all pass
2026-03-26 09:05:29 -06:00
eae4b0324d docs(10): create phase plan
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Portal E2E (push) Has been cancelled
2026-03-25 23:33:27 -06:00
95d05f5f88 docs(10): add research and validation strategy 2026-03-25 23:24:53 -06:00
9f70eede69 docs(10): research phase agent capabilities 2026-03-25 23:24:03 -06:00
003bebc39f docs(state): record phase 10 context session
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Portal E2E (push) Has been cancelled
2026-03-25 23:17:22 -06:00
63cc198ede docs(10): capture phase context 2026-03-25 23:17:22 -06:00
5847052ce4 docs: add Phase 10 — Agent Capabilities (real tool integrations)
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Portal E2E (push) Has been cancelled
2026-03-25 23:07:18 -06:00
46eece580d fix: /admin/users page crash — API response shape mismatch
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Portal E2E (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:01:41 -06:00
b441d7d8e9 docs(phase-9): complete Testing & QA phase execution
Some checks failed
CI / Backend Tests (push) Has been cancelled
CI / Portal E2E (push) Has been cancelled
2026-03-25 22:54:29 -06:00
58cf5811f5 chore: update portal submodule to 09-02 + lighthouserc fix commits
- Portal now at 067c08b (includes visual regression, a11y scans, fixed Lighthouse CI thresholds)
2026-03-25 22:53:48 -06:00
27146c621d docs(09-03): complete Gitea Actions CI pipeline plan
- 09-03-SUMMARY.md: CI pipeline with 2-job fail-fast backend+portal
- STATE.md: advanced to 09-03 complete, added CI decisions
- ROADMAP.md: Phase 9 marked 3/3 plans complete
- REQUIREMENTS.md: QA-07 marked complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 22:53:43 -06:00
24dfb033d7 docs(09-02): complete visual regression, a11y, and Lighthouse CI plan
- 09-02-SUMMARY.md: visual snapshot spec (6 pages × 3 viewports), axe-core scans (8 pages), Lighthouse CI (0.80 hard floor)
- STATE.md: advanced plan counter, added 3 decisions, updated session
- ROADMAP.md: Phase 9 marked complete (3/3 summaries)
- REQUIREMENTS.md: QA-02, QA-03, QA-04 marked complete
2026-03-25 22:53:34 -06:00
542ac51eba feat(09-03): add Gitea Actions CI pipeline with backend + portal jobs
- .gitea/workflows/ci.yml: 2-job pipeline (backend → portal fail-fast)
- backend job: ruff check, ruff format --check, pytest with JUnit XML artifact
- portal job: Next.js build, Playwright E2E (flows + accessibility), Lighthouse CI
- all test reports uploaded as artifacts (playwright-report, playwright-junit, lighthouse)
- credentials via secrets (AUTH_SECRET, E2E_*) — never hardcoded
- packages/portal/e2e/accessibility/axe.spec.ts: WCAG 2.1 AA axe-core tests
- packages/portal/e2e/lighthouse/lighthouserc.json: Lighthouse CI score assertions
2026-03-25 22:41:04 -06:00
86a81ceabb docs(09-01): complete E2E test infrastructure plan
- 09-01-SUMMARY.md: 29 tests across 7 flow specs, 3-browser coverage
- STATE.md: advanced to 94%, added 3 decisions, updated session
- ROADMAP.md: phase 9 in progress (1/3 summaries)
- REQUIREMENTS.md: marked QA-01, QA-05, QA-06 complete
2026-03-25 22:38:45 -06:00
e31690e37a docs(09-testing-qa): create phase plan 2026-03-25 22:26:03 -06:00
a46ff0a970 docs(09): add research and validation strategy 2026-03-25 22:20:19 -06:00
30c82a1754 docs(09): research phase Testing & QA 2026-03-25 22:19:32 -06:00
1db2e0c052 docs(state): record phase 9 context session 2026-03-25 22:11:53 -06:00
972ef9b1f7 docs(09): capture phase context 2026-03-25 22:11:53 -06:00
df6bce7289 docs: add Phase 9 — Testing & QA (E2E, Lighthouse, visual regression, a11y, cross-browser) 2026-03-25 22:09:30 -06:00
13dc55d59c fix: React #418 hydration error — suppressHydrationWarning on html tag
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 21:59:11 -06:00
c910230994 fix: PWA manifest auth bypass + i18n formatting error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 21:45:42 -06:00
a9077e3559 docs(phase-8): complete Mobile + PWA phase execution
Fixed uuid() recursion bug, updated MOB-02 requirement text.
All 8 phases complete.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 21:39:04 -06:00
66bc460a7a docs(08-mobile-pwa-04): complete mobile PWA verification plan — Phase 08 and v1.0 milestone complete 2026-03-25 21:33:45 -06:00
e4b6e8e09f docs(08-03): complete push notifications, offline queue, install prompt plan 2026-03-25 21:32:09 -06:00
81a2ce1498 feat(08-03): push subscription client, service worker handlers, install prompt, offline queue
- Service worker push/notificationclick handlers with conversation deep-link
- PushPermission component for opt-in UI in More sheet
- InstallPrompt component (second-visit, Android + iOS)
- IndexedDB message-queue for offline message persistence
- use-chat-socket.ts: drain queue on reconnect, enqueue when offline
2026-03-25 21:30:29 -06:00
7d3a393758 feat(08-03): push notification backend — DB model, migration, API router, VAPID setup
- Add PushSubscription ORM model with unique(user_id, endpoint) constraint
- Add Alembic migration 012 for push_subscriptions table
- Add push router (subscribe, unsubscribe, send) in shared/api/push.py
- Mount push router in gateway/main.py
- Add pywebpush to gateway dependencies for server-side VAPID delivery
- Wire push trigger into WebSocket handler (fires when client disconnects mid-stream)
- Add VAPID keys to .env / .env.example
- Add push/install i18n keys in en/es/pt message files
2026-03-25 21:26:51 -06:00
5c30651754 docs(08-01): complete mobile PWA foundation plan
- Add 08-01-SUMMARY.md: responsive tab bar + PWA infra with K monogram icons
- Update STATE.md: phase 8 plan 1 progress, decisions, metrics
- Update ROADMAP.md: phase 8 in progress (1/4 SUMMARY files)
- Mark requirements MOB-01, MOB-02, MOB-04 complete
- Update portal submodule pointer to acba978 (mobile nav + PWA commits)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 21:20:45 -06:00
21c91ea83f docs(08-02): complete mobile chat plan — SUMMARY, STATE, ROADMAP updated
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 21:19:16 -06:00
5b6cd348fa fix(08): add 08-02 dependency to 08-03 — shared use-chat-socket.ts file 2026-03-25 20:38:23 -06:00
d9b022bd4c docs(08-mobile-pwa): create phase plan 2026-03-25 20:34:24 -06:00
467a994d9f docs(08): add research and validation strategy 2026-03-25 20:26:37 -06:00
fafbcf742b docs(08): research mobile + PWA phase 2026-03-25 20:25:29 -06:00
238e7dd888 docs(state): record phase 8 context session 2026-03-25 20:08:35 -06:00
f005f4a3b4 docs(08): capture phase context 2026-03-25 20:08:35 -06:00
210be50321 docs: add Phase 8 — Mobile Layout + PWA 2026-03-25 19:29:24 -06:00
9759019262 feat: consolidate 3 finance templates into 1 Finance & Accounting Manager
SMBs don't need separate Financial Manager, Controller, and Accountant.
Merged into one versatile role that handles invoicing, AP/AR, expenses,
budgets, reporting, and cash flow. Full en/es/pt translations.

Templates: 8 → 6 (Customer Support Rep, Sales Assistant, Marketing
Manager, Office Manager, Project Coordinator, Finance & Accounting Manager)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 19:10:57 -06:00
fe3c5a7198 feat: add Marketing Manager template with en/es/pt translations
New AI employee template: Marketing & Growth Manager (category: marketing).
Creative and data-driven, handles campaign strategy, content briefs,
metrics analysis, social media calendars, and lead gen coordination.
Escalates brand-sensitive decisions and high-budget approvals.

Full translations for Spanish and Portuguese with native business
terminology.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 19:03:59 -06:00
17f6d7cb4b fix: streaming timeout + WebSocket close guard
- Streaming httpx client uses 300s read timeout (cloud LLMs can take
  30-60s for first token). Was using 120s general timeout.
- Guard all WebSocket sends with try/except for client disconnect.
  Prevents "Cannot send once close message has been sent" crash.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 18:39:32 -06:00
6c1086046f fix: add LLM_POOL_URL to gateway env — was using localhost instead of llm-pool
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 18:35:43 -06:00
dd80e2b822 perf: bypass Celery for web chat — stream LLM directly from WebSocket
Eliminates 5-10s of overhead by calling the LLM pool's streaming
endpoint directly from the WebSocket handler instead of going through
Celery queue → worker → asyncio.run() → Redis pub-sub → WebSocket.

New flow: WebSocket → agent lookup → memory → LLM stream → WebSocket
Old flow: WebSocket → Celery → worker → DB → memory → LLM → Redis → WebSocket

Memory still saved (Redis sliding window + fire-and-forget embedding).
Slack/WhatsApp still use Celery (async webhook pattern).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 18:32:16 -06:00
2116059157 fix: NullPool for Celery workers + skip pgvector on first message
- Celery workers use NullPool to avoid "Future attached to a different
  loop" errors from stale pooled async connections across asyncio.run()
  calls. FastAPI keeps regular pool (single event loop, safe to reuse).
- Skip pgvector similarity search when no conversation history exists
  (first message) — saves ~3s embedding + query overhead.
- Wrap pgvector retrieval in try/except to prevent DB errors from
  blocking the LLM response.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 18:21:19 -06:00
5e4d9ce144 chore: update portal submodule ref to include streaming changes 2026-03-25 17:57:23 -06:00
61b8762bac feat(streaming): update WebSocket handler to forward streaming chunks to browser
- Pub-sub loop now handles 'chunk' and 'done' message types (not just 'response')
- 'chunk' messages are forwarded immediately via websocket.send_json
- 'done' message breaks the loop and triggers DB persistence of full response
- Sends final 'done' JSON to browser to signal stream completion
- Legacy 'response' type no longer emitted from orchestrator (now unified as 'done')
2026-03-25 17:57:08 -06:00
5fb79beb76 feat(streaming): wire streaming path through orchestrator task pipeline
- _stream_agent_response_to_redis() publishes chunk/done messages to Redis pub-sub
- _process_message() uses streaming path for web channel with no tools registered
- Non-web channels (Slack, WhatsApp) and tool-enabled agents use non-streaming run_agent()
- streaming_delivered flag prevents double-publish when streaming path is active
- _send_response() web branch changed from 'response' to 'done' message type for consistency
2026-03-25 17:57:04 -06:00
9090b54f43 feat(streaming): add run_agent_streaming() to orchestrator runner
- run_agent_streaming() calls POST /complete/stream and yields token strings
- Reads NDJSON lines from the streaming response, yields on 'chunk' events
- On 'error' or connection failure, yields the fallback response string
- Tool calls are not supported in the streaming path
- Existing run_agent() (non-streaming, tool-call loop) is unchanged
2026-03-25 17:57:00 -06:00
f3e358b418 feat(streaming): add complete_stream() generator and POST /complete/stream NDJSON endpoint to llm-pool
- complete_stream() in router.py yields token strings via acompletion(stream=True)
- POST /complete/stream returns NDJSON: chunk lines then a done line
- Streaming path does not support tool calls (plain text only)
- Non-streaming POST /complete endpoint unchanged
2026-03-25 17:56:56 -06:00
b6c8da8cca fix: increase WebSocket pub-sub timeout from 60s to 180s
LLM responses can take >60s (especially with local models). The
WebSocket listener was timing out before the response arrived,
causing agent replies to appear in logs but not in the chat UI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 17:48:22 -06:00