---
phase: 10-agent-capabilities
verified: 2026-03-25T22:00:00Z
status: passed
score: 15/15 must-haves verified
re_verification: false
---

# Phase 10: Agent Capabilities Verification Report

**Phase Goal:** Connect the 4 built-in agent tools to real external services so AI Employees can actually search the web, query a knowledge base of uploaded documents, make HTTP API calls, and check calendar availability
**Verified:** 2026-03-25
**Status:** PASSED
**Re-verification:** No — initial verification

---

## Goal Achievement

### Observable Truths

All must-haves are drawn from plan frontmatter across plans 10-01, 10-02, and 10-03.

#### Plan 10-01 Truths

| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| 1 | Documents uploaded via API are saved to MinIO and a KbDocument row is created with status=processing | VERIFIED | `kb.py` L150-157: inserts `KnowledgeBaseDocument(status='processing')`, `L162-176`: uploads bytes to MinIO via boto3 |
| 2 | The Celery ingestion task extracts text from PDF, DOCX, PPTX, XLSX, CSV, TXT, and MD files | VERIFIED | `extractors.py`: real implementations for all 7 formats using pypdf, python-docx, python-pptx, pandas, UTF-8 decode |
| 3 | Extracted text is chunked (500 chars, 50 overlap) and embedded via all-MiniLM-L6-v2 into kb_chunks with tenant_id | VERIFIED | `ingest.py` L56-92: `chunk_text` sliding window; L174: `embed_texts(chunks)`; L186-202: raw SQL INSERT into kb_chunks with CAST vector |
| 4 | kb_search tool receives tenant_id injection from executor and returns matching chunks | VERIFIED | `executor.py` L126-127: `args["tenant_id"] = str(tenant_id)`; `kb_search.py` L24: accepts `tenant_id` kwarg, runs pgvector cosine similarity query |
| 5 | BRAVE_API_KEY and FIRECRAWL_API_KEY are platform-wide settings in shared config | VERIFIED | `config.py` L223-227: `brave_api_key` and `firecrawl_api_key` as Field entries |
| 6 | Tool executor injects tenant_id and agent_id into tool handler kwargs for context-aware tools | VERIFIED | `executor.py` L126-127: injection occurs after schema validation (L98-103), before handler call (L134) |

#### Plan 10-02 Truths

| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| 7 | Tenant admin can initiate Google Calendar OAuth from the portal and authorize calendar access | VERIFIED | `calendar_auth.py` L104-130: `GET /install` endpoint returns Google OAuth URL with HMAC-signed state, offline access, and consent prompt |
| 8 | Calendar OAuth callback exchanges code for tokens and stores them encrypted per tenant | VERIFIED | `calendar_auth.py` L175-235: httpx POST to Google token endpoint, Fernet encrypt, upsert ChannelConnection(channel_type=GOOGLE_CALENDAR) |
| 9 | Calendar tool reads per-tenant OAuth tokens from channel_connections and calls Google Calendar API | VERIFIED | `calendar_lookup.py` L137-147: SELECT ChannelConnection WHERE channel_type=GOOGLE_CALENDAR; L178: builds Google Credentials; L194-207: run_in_executor for API call |
| 10 | Calendar tool supports list events, check availability, and create event actions | VERIFIED | `calendar_lookup.py` L267-273: dispatches to `_action_list`, `_action_check_availability`, `_action_create`; all three fully implemented |
| 11 | Token auto-refresh works — expired access tokens are refreshed via stored refresh_token and written back to DB | VERIFIED | `calendar_lookup.py` L190: records `token_before`; L210-225: if `creds.token != token_before`, encrypts and commits updated token to DB |
| 12 | Tool results are formatted as natural language (no raw JSON) | VERIFIED | `builder.py` L180-181: system prompt appends "Never show raw data or JSON to the user"; all `calendar_lookup` actions return formatted strings, not dicts |

#### Plan 10-03 Truths

| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| 13 | Operators can see a Knowledge Base page in the portal navigation | VERIFIED | `nav.tsx` L49: `{ href: "/knowledge-base", label: t("knowledgeBase"), icon: BookOpen }`; i18n key present in en/es/pt message files |
| 14 | Operators can upload files via drag-and-drop or file picker dialog | VERIFIED | `upload-dialog.tsx` 249 lines: drag-and-drop zone, file picker input, sequential upload via `uploadKbDocument`; `api.ts` uses `new FormData()` |
| 15 | Uploaded documents show processing status with live polling | VERIFIED | `queries.ts` L518-521: `refetchInterval` returns 5000 when any doc has `status === "processing"`, false otherwise |

**Score:** 15/15 truths verified

---

### Required Artifacts

| Artifact | Status | Details |
|----------|--------|---------|
| `migrations/versions/014_kb_status.py` | VERIFIED | Adds status, error_message, chunk_count to kb_documents; makes agent_id nullable |
| `migrations/versions/013_google_calendar_channel.py` | VERIFIED | Adds google_calendar to channel_connections CHECK constraint |
| `packages/orchestrator/orchestrator/tools/extractors.py` | VERIFIED | 142 lines; real implementations for all 7 format families; exports `extract_text` |
| `packages/orchestrator/orchestrator/tools/ingest.py` | VERIFIED | 323 lines; exports `chunk_text` and `ingest_document_pipeline`; full pipeline with MinIO, YouTube, Firecrawl |
| `packages/shared/shared/api/kb.py` | VERIFIED | 377 lines; 5 endpoints; exports `kb_router` |
| `packages/orchestrator/orchestrator/tasks.py` | VERIFIED | `ingest_document` Celery task at L1008-1036; calls `asyncio.run(ingest_document_pipeline(...))` |
| `packages/orchestrator/orchestrator/tools/executor.py` | VERIFIED | Tenant/agent injection at L126-127, after schema validation, before handler call |
| `packages/shared/shared/api/calendar_auth.py` | VERIFIED | Full OAuth flow; exports `calendar_auth_router`; 3 endpoints |
| `packages/orchestrator/orchestrator/tools/builtins/calendar_lookup.py` | VERIFIED | Service account stub replaced; per-tenant OAuth; list/create/check_availability; token refresh write-back |
| `packages/orchestrator/orchestrator/tools/registry.py` | VERIFIED | All 4 tools in registry; calendar_lookup schema updated with action enum, event_summary, event_start, event_end |
| `packages/gateway/gateway/main.py` | VERIFIED | `kb_router` and `calendar_auth_router` mounted at L174-175 |
| `packages/portal/app/(dashboard)/knowledge-base/page.tsx` | VERIFIED | 88 lines; RBAC-conditional buttons; uses session for tenantId |
| `packages/portal/components/kb/document-list.tsx` | VERIFIED | 259 lines; status badges; delete confirm dialog; re-index; polling via `useKbDocuments` |
| `packages/portal/components/kb/upload-dialog.tsx` | VERIFIED | 249 lines; drag-and-drop; file picker; sequential upload with per-file progress |
| `packages/portal/components/kb/url-ingest-dialog.tsx` | VERIFIED | 162 lines; URL input; auto-YouTube detection; radio source type |
| `tests/unit/test_extractors.py` | VERIFIED | Exists on disk |
| `tests/unit/test_kb_upload.py` | VERIFIED | Exists on disk |
| `tests/unit/test_ingestion.py` | VERIFIED | Exists on disk |
| `tests/unit/test_executor_injection.py` | VERIFIED | Exists on disk |
| `tests/unit/test_calendar_lookup.py` | VERIFIED | Exists on disk |
| `tests/unit/test_calendar_auth.py` | VERIFIED | Exists on disk |

---

### Key Link Verification

| From | To | Via | Status | Details |
|------|----|-----|--------|---------|
| `shared/api/kb.py` | `orchestrator/tasks.py` | `ingest_document.delay(document_id, tenant_id)` | WIRED | L185-187 in kb.py: `_get_ingest_task().delay(str(doc_id), str(tenant_id))`; lazy import avoids circular dep |
| `orchestrator/tools/executor.py` | `tool.handler` | `tenant_id/agent_id` injected into kwargs | WIRED | L126-127: `args["tenant_id"] = str(tenant_id); args["agent_id"] = str(agent_id)` after schema validation |
| `shared/api/calendar_auth.py` | `channel_connections` table | Upsert with `channel_type='google_calendar'` and encrypted token | WIRED | L213-233: `enc_svc.encrypt(token_json)`, upsert `ChannelConnection(channel_type=GOOGLE_CALENDAR, config={"token": encrypted_token})` |
| `orchestrator/tools/builtins/calendar_lookup.py` | `channel_connections` table | Load encrypted token, decrypt, build Credentials | WIRED | L137-147: SELECT ChannelConnection; L167-172: `enc_svc.decrypt(encrypted_token)`; L76-83: `Credentials(refresh_token=...)` |
| `portal/components/kb/knowledge-base/page.tsx` | `/api/portal/kb/{tenant_id}/documents` | TanStack Query fetch + polling | WIRED | `document-list.tsx` L30: imports `useKbDocuments`; L111: `const { data } = useKbDocuments(tenantId)`; `queries.ts` L518-521: conditional `refetchInterval` |
| `portal/components/kb/upload-dialog.tsx` | `/api/portal/kb/{tenant_id}/documents` | FormData multipart POST | WIRED | L109: `await uploadKbDocument(tenantId, files[i].file, authHeaders)`; `api.ts` L378: `const formData = new FormData()` |
| `gateway/gateway/main.py` | `kb_router` + `calendar_auth_router` | `app.include_router(...)` | WIRED | L174-175: both routers mounted |

---

### Requirements Coverage

| Requirement | Source Plan | Description | Status | Evidence |
|-------------|-------------|-------------|--------|----------|
| CAP-01 | 10-01 | Web search tool returns real results from Brave Search | SATISFIED | `web_search.py` L23: `_BRAVE_API_URL = "https://api.search.brave.com/res/v1/web/search"`; L40: `settings.brave_api_key`; full httpx call with error handling |
| CAP-02 | 10-01 | KB tool searches tenant-scoped documents chunked and embedded in pgvector | SATISFIED | `kb_search.py`: pgvector cosine similarity query on kb_chunks; executor injects tenant_id; `ingest.py`: embed_texts + INSERT with CAST vector |
| CAP-03 | 10-01, 10-03 | Operators can upload documents (PDF, DOCX, TXT) via portal | SATISFIED | `kb.py`: upload endpoint + Celery dispatch; portal KB page with upload dialog, URL ingest, status polling, delete, reindex |
| CAP-04 | 10-02 (confirmed) | HTTP request tool can call operator-configured URLs with timeout | SATISFIED | `http_request.py`: full httpx implementation, 30s timeout, 1MB cap, in registry |
| CAP-05 | 10-02 | Calendar tool can check Google Calendar availability and create events | SATISFIED | `calendar_lookup.py`: per-tenant OAuth, list/check_availability/create actions; full Google Calendar API integration |
| CAP-06 | 10-02 | Tool results incorporated naturally — no raw JSON shown to users | SATISFIED | `builder.py` L180-181: system prompt instruction; all tool handlers return formatted strings |
| CAP-07 | 10-02 (confirmed) | All tool invocations logged in audit trail | SATISFIED | `executor.py` L137-145: `audit_logger.log_tool_call(...)` on every success; L153-161: logged on every error; L192: logged on validation failure |

**All 7 requirements satisfied. No orphaned requirements.**

---

### Anti-Patterns Found

None detected. Scanned all key backend and portal files for TODO, FIXME, placeholder, `return null`, `return {}`, `console.log` — none found.

---

### Human Verification Required

**1. Google Calendar OAuth end-to-end flow**
**Test:** With GOOGLE_CLIENT_ID/SECRET configured, navigate to portal settings, click "Connect Google Calendar", complete Google consent, verify redirect back with `?calendar=connected`
**Expected:** Token stored in channel_connections; subsequent agent messages can list/create calendar events
**Why human:** External OAuth redirect flow cannot be verified programmatically without real Google credentials and a live browser session

**2. Knowledge Base document ingestion end-to-end**
**Test:** Upload a PDF or DOCX via the portal KB page, wait for status to change from "Processing" to "Ready", then send a message to an agent with kb_search assigned that references the document content
**Expected:** Agent correctly cites information from the uploaded document
**Why human:** Requires live MinIO, Celery worker, pgvector DB, and LLM inference stack to be running

**3. Portal RBAC enforcement on KB page**
**Test:** Log in as a customer_operator user, navigate to /knowledge-base
**Expected:** Document list is visible; "Upload Files" and "Add URL" buttons are hidden; Delete and Re-index action buttons are hidden
**Why human:** RBAC conditional rendering requires live portal with a real operator session

**4. Web search returns real results**
**Test:** With BRAVE_API_KEY set, trigger an agent tool call to `web_search` with a current events query
**Expected:** Agent receives and summarizes real search results, not cached or static data
**Why human:** Requires live Brave API key and working agent inference loop

---

### Gaps Summary

No gaps. All 15 must-have truths verified, all 7 requirements satisfied (CAP-01 through CAP-07), all key links wired, no anti-patterns found, all artifacts are substantive implementations (not stubs).

Notable: The portal KB implementation (Plan 10-03) is in a git submodule at `packages/portal`. The commit `c525c02` exists in the submodule log but is not surfaced in the parent repo's git log — this is expected submodule behavior. The files exist on disk and are substantive.

---

_Verified: 2026-03-25_
_Verifier: Claude (gsd-verifier)_