---
phase: 02-agent-features
plan: "02"
subsystem: agent-tools
tags: [tool-framework, audit-logging, jsonschema, pgvector, litellm, celery, redis, postgresql-rls]

# Dependency graph
requires:
  - phase: 02-agent-features/02-01
    provides: "Memory pipeline (conversation embeddings, Redis sliding window, pgvector retrieval)"
  - phase: 01-foundation
    provides: "PostgreSQL RLS infrastructure, agent runner, LLM pool, Celery tasks"

provides:
  - "Tool registry with 4 built-in tools (web_search, kb_search, http_request, calendar_lookup)"
  - "Schema-validated tool executor using jsonschema — rejects invalid LLM-generated args"
  - "Multi-turn tool-call loop in agent runner (up to 5 iterations)"
  - "Confirmation gate: tools with requires_confirmation=True pause for user approval via Redis"
  - "Immutable audit trail (audit_events) — REVOKE UPDATE/DELETE enforced at DB level"
  - "AuditLogger: log_llm_call, log_tool_call, log_escalation methods"
  - "KB document and chunk ORM models for knowledge base ingestion"
  - "Migration 004: audit_events, kb_documents, kb_chunks with HNSW index and RLS"
  - "llm-pool /complete endpoint now accepts tools parameter and returns tool_calls"

affects:
  - 02-agent-features/02-03
  - 02-agent-features/02-04
  - 03-operator-experience

# Tech tracking
tech-stack:
  added:
    - "jsonschema (orchestrator) — JSON Schema validation of LLM-generated tool args"
  patterns:
    - "Tool registry pattern: ToolDefinition model with name/description/parameters/requires_confirmation/handler"
    - "Schema-validate-before-execute: all tool args validated against JSON Schema before handler call"
    - "Tool-call loop: LLM -> tool_calls -> execute -> tool result message -> re-call LLM"
    - "Confirmation gate: tools with side effects require user yes/no before execution"
    - "Audit events are append-only: REVOKE UPDATE/DELETE from konstruct_app in migration"
    - "AuditLogger uses raw INSERT (not ORM) to prevent accidental UPDATE/DELETE via ORM session"
    - "LLMResponse model wraps content + tool_calls from litellm.acompletion()"

key-files:
  created:
    - "packages/shared/shared/models/audit.py — AuditEvent ORM model"
    - "packages/shared/shared/models/kb.py — KnowledgeBaseDocument and KBChunk ORM models"
    - "packages/orchestrator/orchestrator/audit/__init__.py"
    - "packages/orchestrator/orchestrator/audit/logger.py — AuditLogger class"
    - "packages/orchestrator/orchestrator/tools/__init__.py"
    - "packages/orchestrator/orchestrator/tools/registry.py — ToolDefinition + BUILTIN_TOOLS"
    - "packages/orchestrator/orchestrator/tools/executor.py — execute_tool with schema validation"
    - "packages/orchestrator/orchestrator/tools/builtins/web_search.py — Brave Search API"
    - "packages/orchestrator/orchestrator/tools/builtins/kb_search.py — pgvector KB search"
    - "packages/orchestrator/orchestrator/tools/builtins/http_request.py — outbound HTTP"
    - "packages/orchestrator/orchestrator/tools/builtins/calendar_lookup.py — Google Calendar"
    - "migrations/versions/004_phase2_audit_kb.py — audit_events, kb_documents, kb_chunks"
    - "tests/unit/test_tool_registry.py — 14 unit tests"
    - "tests/unit/test_tool_executor.py — 8 unit tests"
    - "tests/integration/test_audit.py — 6 integration tests"
  modified:
    - "packages/orchestrator/orchestrator/agents/runner.py — tool-call loop + audit logging"
    - "packages/orchestrator/orchestrator/tasks.py — AuditLogger init, tool registry, confirmation flow"
    - "packages/llm-pool/llm_pool/main.py — tools param in request, tool_calls in response"
    - "packages/llm-pool/llm_pool/router.py — LLMResponse model, tools forwarded to litellm"
    - "packages/orchestrator/pyproject.toml — jsonschema dependency added"

key-decisions:
  - "CAST(:metadata AS jsonb) used instead of :metadata::jsonb — asyncpg doesn't support :: cast syntax in parameterized queries"
  - "Migration renamed 003 -> 004 to avoid duplicate revision ID conflict with existing 003_escalation_fields.py"
  - "AuditLogger uses raw INSERT text() not ORM model — prevents accidental ORM-managed UPDATE/DELETE"
  - "Confirmation message detection via startswith prefix string matching — simple and reliable without adding state to runner"
  - "Tool confirmation stores minimal JSON in Redis (tool_name + message, not full tool_call) — full re-execution deferred to Phase 3"
  - "Memory persistence skipped for confirmation request responses — only real LLM turns get embedded"

patterns-established:
  - "Tool-call loop pattern: check response.tool_calls, execute each, append tool role message, re-call LLM"
  - "Confirmation gate pattern: check requires_confirmation before executing, store pending in Redis, resolve on next turn"
  - "Audit write pattern: fresh session per write (not shared with caller session) to prevent ORM tracking"
  - "Schema validation first: jsonschema.validate() called before any tool handler — untrusted LLM args always validated"

requirements-completed:
  - AGNT-04
  - AGNT-06

# Metrics
duration: 12min 22s
completed: 2026-03-23
---

# Phase 2 Plan 02: Tool Framework and Audit Logging Summary

**JSON Schema-validated tool registry with 4 built-in tools, multi-turn LLM tool-call loop with confirmation gate, and immutable tenant-scoped audit trail enforced at the PostgreSQL REVOKE level**

## Performance

- **Duration:** 12min 22s
- **Started:** 2026-03-23T20:48:09Z
- **Completed:** 2026-03-23T21:00:31Z
- **Tasks:** 3
- **Files modified:** 19

## Accomplishments

- Tool framework with schema-validated execution, confirmation gates, and 4 built-in tools
- Immutable audit trail (audit_events) with REVOKE UPDATE/DELETE from konstruct_app — tamper-proof at DB level
- Multi-turn tool-call loop in agent runner (5-iteration max guard, reason→tool→observe→respond pattern)
- llm-pool updated to forward tools to LiteLLM and return tool_calls in response
- 28 new tests (14 unit + 6 unit + 6 integration + 2 extra for TDD cycle) — 258 total passing

## Task Commits

1. **Task 1: Audit model, KB model, migration, and audit logger** (TDD)
   - `df7a5a9` test(02-02): add failing audit integration tests
   - `30b9f60` feat(02-02): audit model, KB model, migration, and audit logger

2. **Task 2: Tool registry, executor, and 4 built-in tools** (TDD)
   - `420294b` test(02-02): add failing tool registry and executor unit tests
   - `f499278` feat(02-02): tool registry, executor, and 4 built-in tools

3. **Task 3: Wire tool-call loop into agent runner and orchestrator pipeline**
   - `44fa7e6` feat(02-02): wire tool-call loop into agent runner and orchestrator pipeline

## Files Created/Modified

- `packages/shared/shared/models/audit.py` — AuditEvent ORM (append-only, RLS-scoped)
- `packages/shared/shared/models/kb.py` — KnowledgeBaseDocument and KBChunk ORM models
- `packages/orchestrator/orchestrator/audit/logger.py` — AuditLogger with 3 log methods
- `packages/orchestrator/orchestrator/tools/registry.py` — ToolDefinition model + BUILTIN_TOOLS + to_litellm_format
- `packages/orchestrator/orchestrator/tools/executor.py` — Schema-validated execute_tool
- `packages/orchestrator/orchestrator/tools/builtins/web_search.py` — Brave Search API
- `packages/orchestrator/orchestrator/tools/builtins/kb_search.py` — pgvector KB search
- `packages/orchestrator/orchestrator/tools/builtins/http_request.py` — Outbound HTTP (30s timeout, 1MB cap)
- `packages/orchestrator/orchestrator/tools/builtins/calendar_lookup.py` — Google Calendar read-only
- `migrations/versions/004_phase2_audit_kb.py` — audit_events (immutable) + kb tables with HNSW index
- `packages/orchestrator/orchestrator/agents/runner.py` — Tool-call loop, audit logging per LLM call
- `packages/orchestrator/orchestrator/tasks.py` — AuditLogger init, tool registry, confirmation Redis flow
- `packages/llm-pool/llm_pool/main.py` — tools in request, tool_calls in response
- `packages/llm-pool/llm_pool/router.py` — LLMResponse model, tools forwarded to litellm

## Decisions Made

- **CAST(:metadata AS jsonb)** — asyncpg does not support PostgreSQL-style `::` cast syntax in parameterized queries; must use SQL CAST()
- **Migration 004** — existing Plan 02-04 had already created a revision 003 (escalation fields); renamed to 004 to maintain linear history
- **AuditLogger raw INSERT** — uses `text()` raw SQL rather than ORM model to prevent SQLAlchemy session from accidentally tracking the row for update
- **Confirmation string matching** — detection of confirmation messages via `startswith("This action requires your approval")` is simple and doesn't require additional state in the runner
- **Redis confirmation TTL = 10 minutes** — reasonable window for a human to respond in a chat context without the pending state lingering indefinitely

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 1 - Bug] Fixed CAST syntax for JSONB parameter in asyncpg**
- **Found during:** Task 1 (AuditLogger GREEN phase)
- **Issue:** `INSERT ... VALUES (:metadata::jsonb)` raises PostgresSyntaxError — asyncpg converts named params to $1 placeholders before PostgreSQL sees the `::` cast
- **Fix:** Changed to `CAST(:metadata AS jsonb)` which is standard SQL and works with asyncpg
- **Files modified:** `packages/orchestrator/orchestrator/audit/logger.py`
- **Verification:** All 6 integration tests pass
- **Committed in:** 30b9f60 (Task 1 feat commit)

**2. [Rule 1 - Bug] Added missing execute_tool import in test file**
- **Found during:** Task 2 (tool executor GREEN phase)
- **Issue:** `test_confirmation_required_does_not_call_handler` test referenced `execute_tool` without importing it
- **Fix:** Added `from orchestrator.tools.executor import execute_tool` to the test method
- **Files modified:** `tests/unit/test_tool_executor.py`
- **Verification:** All 22 unit tests pass
- **Committed in:** f499278 (Task 2 feat commit)

**3. [Rule 1 - Bug] Renamed migration 003 -> 004 to fix duplicate revision ID**
- **Found during:** Task 3 (overall verification after wiring runner)
- **Issue:** Alembic reported "Multiple head revisions" — 003_escalation_fields.py (from Plan 02-04) already claimed revision ID "003"
- **Fix:** Renamed file to 004_phase2_audit_kb.py, updated revision="004" and down_revision="003"
- **Files modified:** `migrations/versions/004_phase2_audit_kb.py`
- **Verification:** `alembic history` shows clean linear chain 001→002→003→004
- **Committed in:** 44fa7e6 (Task 3 feat commit)

---

**Total deviations:** 3 auto-fixed (all Rule 1 - Bug)
**Impact on plan:** All necessary for correctness — CAST syntax fix for DB compatibility, import fix for test correctness, migration rename for clean history. No scope creep.

## Issues Encountered

None beyond the auto-fixed deviations above.

## User Setup Required

The following environment variables enable the built-in tools:

- `BRAVE_API_KEY` — enables web_search tool (Brave Search API)
- `GOOGLE_SERVICE_ACCOUNT_KEY` — enables calendar_lookup tool (JSON key for Google Calendar read-only access)

Without these vars the tools degrade gracefully (return informative messages rather than errors).

## Next Phase Readiness

- Tool framework is complete: agents can reason, call tools, observe results, and respond
- Audit trail is operational: every LLM call and tool invocation is logged with tenant isolation
- KB tables are ready: kb_documents and kb_chunks are in the DB, knowledge base ingestion pipeline can be built
- Confirmation flow works end-to-end via Redis: http_request tool will pause and ask before executing
- Ready for Phase 2 Plan 03 (multi-agent teams with coordinator pattern)

---
*Phase: 02-agent-features*
*Completed: 2026-03-23*

## Self-Check: PASSED

All 15 created/modified files verified present on disk.
All 5 task commits (df7a5a9, 30b9f60, 420294b, f499278, 44fa7e6) verified in git history.