docs(01-foundation): create phase plan
This commit is contained in:
264
.planning/phases/01-foundation/01-01-PLAN.md
Normal file
264
.planning/phases/01-foundation/01-01-PLAN.md
Normal file
@@ -0,0 +1,264 @@
|
||||
---
|
||||
phase: 01-foundation
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- pyproject.toml
|
||||
- docker-compose.yml
|
||||
- .env.example
|
||||
- packages/shared/models/message.py
|
||||
- packages/shared/models/tenant.py
|
||||
- packages/shared/models/auth.py
|
||||
- packages/shared/db.py
|
||||
- packages/shared/rls.py
|
||||
- packages/shared/config.py
|
||||
- packages/shared/redis_keys.py
|
||||
- packages/shared/__init__.py
|
||||
- packages/shared/models/__init__.py
|
||||
- migrations/env.py
|
||||
- migrations/versions/001_initial_schema.py
|
||||
- tests/conftest.py
|
||||
- tests/unit/test_normalize.py
|
||||
- tests/unit/test_tenant_resolution.py
|
||||
- tests/unit/test_redis_namespacing.py
|
||||
- tests/integration/test_tenant_isolation.py
|
||||
autonomous: true
|
||||
requirements:
|
||||
- CHAN-01
|
||||
- TNNT-01
|
||||
- TNNT-02
|
||||
- TNNT-03
|
||||
- TNNT-04
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "KonstructMessage Pydantic model validates and normalizes a Slack event payload into the unified internal format"
|
||||
- "Tenant A cannot query Tenant B's rows from the agents or channel_connections tables — enforced at the PostgreSQL layer via RLS"
|
||||
- "A channel workspace ID resolves to the correct Konstruct tenant ID via the channel_connections table"
|
||||
- "All Redis keys include {tenant_id}: prefix — no bare keys are possible through the shared utility"
|
||||
- "PostgreSQL and Redis are reachable via Docker Compose with TLS-ready configuration"
|
||||
artifacts:
|
||||
- path: "packages/shared/models/message.py"
|
||||
provides: "KonstructMessage, ChannelType, SenderInfo, MessageContent Pydantic models"
|
||||
exports: ["KonstructMessage", "ChannelType", "SenderInfo", "MessageContent"]
|
||||
- path: "packages/shared/models/tenant.py"
|
||||
provides: "Tenant, Agent, ChannelConnection SQLAlchemy models with RLS"
|
||||
exports: ["Tenant", "Agent", "ChannelConnection"]
|
||||
- path: "packages/shared/db.py"
|
||||
provides: "Async SQLAlchemy engine, session factory, get_session dependency"
|
||||
exports: ["engine", "async_session_factory", "get_session"]
|
||||
- path: "packages/shared/rls.py"
|
||||
provides: "current_tenant_id ContextVar and SQLAlchemy event hook for SET LOCAL"
|
||||
exports: ["current_tenant_id", "configure_rls_hook"]
|
||||
- path: "packages/shared/redis_keys.py"
|
||||
provides: "Namespaced Redis key constructors"
|
||||
exports: ["rate_limit_key", "idempotency_key", "session_key"]
|
||||
- path: "migrations/versions/001_initial_schema.py"
|
||||
provides: "Initial DB schema with RLS policies and FORCE ROW LEVEL SECURITY"
|
||||
contains: "FORCE ROW LEVEL SECURITY"
|
||||
- path: "tests/integration/test_tenant_isolation.py"
|
||||
provides: "Two-tenant RLS isolation test"
|
||||
contains: "tenant_a.*tenant_b"
|
||||
key_links:
|
||||
- from: "packages/shared/rls.py"
|
||||
to: "packages/shared/db.py"
|
||||
via: "SQLAlchemy before_cursor_execute event hook on engine"
|
||||
pattern: "event\\.listens_for.*before_cursor_execute"
|
||||
- from: "migrations/versions/001_initial_schema.py"
|
||||
to: "packages/shared/models/tenant.py"
|
||||
via: "Schema must match SQLAlchemy model definitions"
|
||||
pattern: "CREATE TABLE (tenants|agents|channel_connections)"
|
||||
- from: "packages/shared/redis_keys.py"
|
||||
to: "Redis"
|
||||
via: "All key functions prepend tenant_id"
|
||||
pattern: "f\"{tenant_id}:"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Scaffold the Python monorepo, Docker Compose dev environment, shared Pydantic/SQLAlchemy models, PostgreSQL schema with RLS tenant isolation, Redis namespacing utilities, and the foundational test infrastructure.
|
||||
|
||||
Purpose: Establish the secure multi-tenant data layer that every subsequent plan builds on. Tenant isolation is the most dangerous failure mode in Phase 1 — it must be proven correct before any channel or LLM code exists.
|
||||
|
||||
Output: Working monorepo with `uv` workspaces, Docker Compose running PostgreSQL 16 + Redis 7 + Ollama, shared data models (KonstructMessage, Tenant, Agent, ChannelConnection), Alembic migrations with RLS, Redis key namespacing, and green isolation tests.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@/home/adelorenzo/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@/home/adelorenzo/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/01-foundation/01-CONTEXT.md
|
||||
@.planning/phases/01-foundation/01-RESEARCH.md
|
||||
@CLAUDE.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Monorepo scaffolding, Docker Compose, and shared data models</name>
|
||||
<files>
|
||||
pyproject.toml,
|
||||
docker-compose.yml,
|
||||
.env.example,
|
||||
packages/shared/__init__.py,
|
||||
packages/shared/models/__init__.py,
|
||||
packages/shared/models/message.py,
|
||||
packages/shared/models/tenant.py,
|
||||
packages/shared/models/auth.py,
|
||||
packages/shared/config.py,
|
||||
packages/shared/db.py,
|
||||
packages/shared/rls.py,
|
||||
packages/shared/redis_keys.py
|
||||
</files>
|
||||
<action>
|
||||
1. Initialize the Python monorepo with `uv init` and configure `pyproject.toml` with workspace members for packages/shared, packages/gateway, packages/router, packages/orchestrator, packages/llm-pool. Add core dependencies: fastapi[standard], pydantic[email], sqlalchemy[asyncio], asyncpg, alembic, litellm, redis, celery[redis], slack-bolt, httpx, slowapi. Add dev dependencies: ruff, mypy, pytest, pytest-asyncio, pytest-httpx. Configure `[tool.pytest.ini_options]` with `asyncio_mode = "auto"` and `testpaths = ["tests"]`. Configure `[tool.ruff]` with line-length=120 and basic rules.
|
||||
|
||||
2. Create `docker-compose.yml` with services:
|
||||
- `postgres`: PostgreSQL 16 with `POSTGRES_DB=konstruct`, creates `konstruct_app` role via init script, port 5432
|
||||
- `redis`: Redis 7, port 6379
|
||||
- `ollama`: Ollama with GPU optional (deploy.resources.reservations.devices with count:all, but service starts regardless), port 11434
|
||||
- Shared `konstruct-net` bridge network
|
||||
|
||||
3. Create `.env.example` with all required environment variables: DATABASE_URL (using konstruct_app role, not postgres superuser), REDIS_URL, SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET, ANTHROPIC_API_KEY, OPENAI_API_KEY, OLLAMA_BASE_URL, AUTH_SECRET.
|
||||
|
||||
4. Create `packages/shared/config.py` using Pydantic Settings to load all env vars with sensible defaults for local dev.
|
||||
|
||||
5. Create `packages/shared/models/message.py` with the KonstructMessage Pydantic model exactly as specified in RESEARCH.md: ChannelType (StrEnum: slack, whatsapp, mattermost), SenderInfo, MessageContent, KonstructMessage. The `tenant_id` field is `str | None = None` (populated by Router after tenant resolution).
|
||||
|
||||
6. Create `packages/shared/models/tenant.py` with SQLAlchemy 2.0 async models:
|
||||
- `Tenant`: id (UUID PK), name (str, unique), slug (str, unique), settings (JSON), created_at, updated_at
|
||||
- `Agent`: id (UUID PK), tenant_id (FK to Tenant, NOT NULL), name (str), role (str), persona (text), system_prompt (text), model_preference (str, default "quality"), tool_assignments (JSON, default []), escalation_rules (JSON, default []), is_active (bool, default True), created_at, updated_at
|
||||
- `ChannelConnection`: id (UUID PK), tenant_id (FK to Tenant, NOT NULL), channel_type (ChannelType enum), workspace_id (str, unique for the channel_type), config (JSON — stores bot tokens, channel IDs per tenant), created_at
|
||||
Use SQLAlchemy 2.0 `Mapped[]` and `mapped_column()` style — never 1.x `Column()` style.
|
||||
|
||||
7. Create `packages/shared/models/auth.py` with a `PortalUser` SQLAlchemy model: id (UUID PK), email (str, unique), hashed_password (str), name (str), is_admin (bool, default False), created_at, updated_at. This is for portal authentication (Auth.js v5 will validate against this via a FastAPI endpoint).
|
||||
|
||||
8. Create `packages/shared/db.py` with async SQLAlchemy engine (asyncpg driver) and session factory. Use `create_async_engine` with `DATABASE_URL` from config. Export `get_session` as an async generator for FastAPI dependency injection.
|
||||
|
||||
9. Create `packages/shared/rls.py` with `current_tenant_id` ContextVar and a `configure_rls_hook(engine)` function that registers a `before_cursor_execute` event listener to `SET LOCAL app.current_tenant = '{tenant_id}'` when `current_tenant_id` is set. CRITICAL: Use parameterized query for the SET LOCAL to prevent SQL injection — use `cursor.execute("SET LOCAL app.current_tenant = %s", (tenant_id,))`.
|
||||
|
||||
10. Create `packages/shared/redis_keys.py` with typed key constructor functions: `rate_limit_key(tenant_id, channel)`, `idempotency_key(tenant_id, message_id)`, `session_key(tenant_id, thread_id)`, `engaged_thread_key(tenant_id, thread_id)`. Every function prepends `{tenant_id}:`. No Redis key should ever be constructable without a tenant_id.
|
||||
|
||||
11. Create minimal `__init__.py` files for packages/shared and packages/shared/models with appropriate re-exports.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/adelorenzo/repos/konstruct && uv sync && python -c "from packages.shared.models.message import KonstructMessage; from packages.shared.models.tenant import Tenant, Agent, ChannelConnection; from packages.shared.redis_keys import rate_limit_key; print('imports OK')"</automated>
|
||||
</verify>
|
||||
<done>
|
||||
- pyproject.toml configures uv workspaces with all dependencies
|
||||
- docker-compose.yml defines PostgreSQL 16, Redis 7, Ollama services
|
||||
- KonstructMessage, Tenant, Agent, ChannelConnection, PortalUser models importable
|
||||
- RLS hook configurable on engine
|
||||
- All Redis key functions require tenant_id parameter
|
||||
- .env.example documents all required env vars with konstruct_app role (not postgres superuser)
|
||||
</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Alembic migrations with RLS and tenant isolation tests</name>
|
||||
<files>
|
||||
migrations/env.py,
|
||||
migrations/script.py.mako,
|
||||
migrations/versions/001_initial_schema.py,
|
||||
tests/conftest.py,
|
||||
tests/unit/__init__.py,
|
||||
tests/unit/test_normalize.py,
|
||||
tests/unit/test_tenant_resolution.py,
|
||||
tests/unit/test_redis_namespacing.py,
|
||||
tests/integration/__init__.py,
|
||||
tests/integration/test_tenant_isolation.py
|
||||
</files>
|
||||
<action>
|
||||
1. Initialize Alembic with `alembic init migrations`. Modify `migrations/env.py` to use async engine (asyncpg) — follow the SQLAlchemy 2.0 async Alembic pattern with `run_async_migrations()`. Import the SQLAlchemy Base metadata from `packages/shared/models/tenant.py` so autogenerate works.
|
||||
|
||||
2. Create `migrations/versions/001_initial_schema.py`:
|
||||
- Create `konstruct_app` role: `CREATE ROLE konstruct_app WITH LOGIN PASSWORD 'konstruct_dev'` (dev password, .env overrides in prod)
|
||||
- Create tables: tenants, agents, channel_connections, portal_users — matching the SQLAlchemy models from Task 1
|
||||
- Apply RLS to tenant-scoped tables (agents, channel_connections):
|
||||
```sql
|
||||
ALTER TABLE agents ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE agents FORCE ROW LEVEL SECURITY;
|
||||
CREATE POLICY tenant_isolation ON agents USING (tenant_id = current_setting('app.current_tenant')::uuid);
|
||||
```
|
||||
Same pattern for channel_connections.
|
||||
- Do NOT apply RLS to `tenants` table itself (platform admin needs to list all tenants) or `portal_users` table.
|
||||
- GRANT SELECT, INSERT, UPDATE, DELETE on all tables to `konstruct_app`.
|
||||
- GRANT USAGE ON SCHEMA public TO `konstruct_app`.
|
||||
|
||||
3. Create `tests/conftest.py` with shared fixtures:
|
||||
- `db_engine`: Creates a test PostgreSQL database, runs migrations, yields async engine connected as `konstruct_app` (not postgres superuser), drops test DB after
|
||||
- `db_session`: Async session from the engine with RLS hook configured
|
||||
- `tenant_a` / `tenant_b`: Two-tenant fixture — creates two tenants, yields their IDs
|
||||
- `redis_client`: Creates a real Redis connection (or fakeredis if Docker not available) scoped to test prefix
|
||||
- Use `pytest.mark.asyncio` for async tests (auto mode from pyproject.toml config)
|
||||
- IMPORTANT: The test DB connection MUST use the `konstruct_app` role to actually test RLS. If using postgres superuser, RLS is bypassed and tests are worthless.
|
||||
|
||||
4. Create `tests/unit/test_normalize.py` (CHAN-01):
|
||||
- Test that a raw Slack `message` event payload normalizes to a valid KonstructMessage
|
||||
- Test ChannelType is set to "slack"
|
||||
- Test sender info is extracted correctly
|
||||
- Test thread_id is populated from Slack's `thread_ts`
|
||||
- Test channel_metadata contains workspace_id
|
||||
|
||||
5. Create `tests/unit/test_tenant_resolution.py` (TNNT-02):
|
||||
- Test that given a workspace_id and channel_type, the correct tenant_id is returned from a mock channel_connections lookup
|
||||
- Test that unknown workspace_id returns None
|
||||
- Test that workspace_id from wrong channel_type doesn't match
|
||||
|
||||
6. Create `tests/unit/test_redis_namespacing.py` (TNNT-03):
|
||||
- Test that `rate_limit_key("tenant-a", "slack")` returns `"tenant-a:ratelimit:slack"`
|
||||
- Test that `idempotency_key("tenant-a", "msg-123")` returns `"tenant-a:dedup:msg-123"`
|
||||
- Test that all key functions include tenant_id prefix
|
||||
- Test that no function can produce a key without tenant_id
|
||||
|
||||
7. Create `tests/integration/test_tenant_isolation.py` (TNNT-01) — THIS IS THE MOST CRITICAL TEST IN PHASE 1:
|
||||
- Uses the two-tenant fixture (tenant_a, tenant_b)
|
||||
- Creates an agent for tenant_a
|
||||
- Sets current_tenant_id to tenant_b
|
||||
- Queries agents table — MUST return zero rows (tenant_b cannot see tenant_a's agent)
|
||||
- Sets current_tenant_id to tenant_a
|
||||
- Queries agents table — MUST return one row
|
||||
- Repeat for channel_connections table
|
||||
- Verify with `SELECT relforcerowsecurity FROM pg_class WHERE relname = 'agents'` — must be True
|
||||
|
||||
8. Create empty `__init__.py` for tests/unit/ and tests/integration/.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/adelorenzo/repos/konstruct && docker compose up -d postgres redis && sleep 3 && alembic upgrade head && pytest tests/unit -x -q && pytest tests/integration/test_tenant_isolation.py -x -q</automated>
|
||||
</verify>
|
||||
<done>
|
||||
- Alembic migration creates all tables with RLS policies and FORCE ROW LEVEL SECURITY
|
||||
- konstruct_app role exists and is used by all application connections
|
||||
- Unit tests pass for KonstructMessage normalization, tenant resolution logic, and Redis namespacing
|
||||
- Integration test proves tenant_a cannot see tenant_b's data through PostgreSQL RLS
|
||||
- `relforcerowsecurity` is True for agents and channel_connections tables
|
||||
</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- `docker compose up -d` starts PostgreSQL 16, Redis 7, and Ollama without errors
|
||||
- `alembic upgrade head` applies the initial schema with RLS
|
||||
- `pytest tests/unit -x -q` passes all unit tests (normalize, tenant resolution, redis namespacing)
|
||||
- `pytest tests/integration/test_tenant_isolation.py -x -q` proves RLS isolation
|
||||
- All imports from packages/shared work correctly
|
||||
- No Redis key can be constructed without a tenant_id
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Green test suite proving tenant A cannot access tenant B's data
|
||||
- KonstructMessage model validates Slack event payloads
|
||||
- Docker Compose dev environment boots cleanly
|
||||
- All subsequent plans can import from packages/shared without modification
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/01-foundation/01-01-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user