talk2me/SECRETS_MANAGEMENT.md

# Secrets Management Documentation

This document describes the secure secrets management system implemented in Talk2Me.

## Overview

Talk2Me uses a comprehensive secrets management system that provides:
- Encrypted storage of sensitive configuration
- Secret rotation capabilities
- Audit logging
- Integrity verification
- CLI management tools
- Environment variable integration

## Architecture

### Components

1. **SecretsManager** (`secrets_manager.py`)
   - Handles encryption/decryption using Fernet (AES-128)
   - Manages secret lifecycle (create, read, update, delete)
   - Provides audit logging
   - Supports secret rotation

2. **Configuration System** (`config.py`)
   - Integrates secrets with Flask configuration
   - Environment-specific configurations
   - Validation and sanitization

3. **CLI Tool** (`manage_secrets.py`)
   - Command-line interface for secret management
   - Interactive and scriptable

### Security Features

- **Encryption**: AES-128 encryption using cryptography.fernet
- **Key Derivation**: PBKDF2 with SHA256 (100,000 iterations)
- **Master Key**: Stored separately with restricted permissions
- **Audit Trail**: All access and modifications logged
- **Integrity Checks**: Verify secrets haven't been tampered with

## Quick Start

### 1. Initialize Secrets

```bash
python manage_secrets.py init
```

This will:
- Generate a master encryption key
- Create initial secrets (Flask secret key, admin token)
- Prompt for required secrets (TTS API key)

### 2. Set a Secret

```bash
# Interactive (hidden input)
python manage_secrets.py set TTS_API_KEY

# Direct (be careful with shell history)
python manage_secrets.py set TTS_API_KEY --value "your-api-key"

# With metadata
python manage_secrets.py set API_KEY --value "key" --metadata '{"service": "external-api"}'
```

### 3. List Secrets

```bash
python manage_secrets.py list
```

Output:
```
Key                            Created             Last Rotated         Has Value
-------------------------------------------------------------------------------------
FLASK_SECRET_KEY              2024-01-15          2024-01-20          ✓
TTS_API_KEY                   2024-01-15          Never               ✓
ADMIN_TOKEN                   2024-01-15          2024-01-18          ✓
```

### 4. Rotate Secrets

```bash
# Rotate a specific secret
python manage_secrets.py rotate ADMIN_TOKEN

# Check which secrets need rotation
python manage_secrets.py check-rotation

# Schedule automatic rotation
python manage_secrets.py schedule-rotation API_KEY 30  # Every 30 days
```

## Configuration

### Environment Variables

The secrets manager checks these locations in order:
1. Encrypted secrets storage (`.secrets.json`)
2. `SECRET_<KEY>` environment variable
3. `<KEY>` environment variable
4. Default value

### Master Key

The master encryption key is loaded from:
1. `MASTER_KEY` environment variable
2. `.master_key` file (default)
3. Auto-generated if neither exists

**Important**: Protect the master key!
- Set file permissions: `chmod 600 .master_key`
- Back it up securely
- Never commit to version control

### Flask Integration

Secrets are automatically loaded into Flask configuration:

```python
# In app.py
from config import init_app as init_config
from secrets_manager import init_app as init_secrets

app = Flask(__name__)
init_config(app)
init_secrets(app)

# Access secrets
api_key = app.config['TTS_API_KEY']
```

## CLI Commands

### Basic Operations

```bash
# List all secrets
python manage_secrets.py list

# Get a secret value (requires confirmation)
python manage_secrets.py get TTS_API_KEY

# Set a secret
python manage_secrets.py set DATABASE_URL

# Delete a secret
python manage_secrets.py delete OLD_API_KEY

# Rotate a secret
python manage_secrets.py rotate ADMIN_TOKEN
```

### Advanced Operations

```bash
# Verify integrity of all secrets
python manage_secrets.py verify

# Migrate from environment variables
python manage_secrets.py migrate

# View audit log
python manage_secrets.py audit
python manage_secrets.py audit TTS_API_KEY --limit 50

# Schedule rotation
python manage_secrets.py schedule-rotation API_KEY 90
```

## Security Best Practices

### 1. File Permissions

```bash
# Secure the secrets files
chmod 600 .secrets.json
chmod 600 .master_key
```

### 2. Backup Strategy

- Back up `.master_key` separately from `.secrets.json`
- Store backups in different secure locations
- Test restore procedures regularly

### 3. Rotation Policy

Recommended rotation intervals:
- API Keys: 90 days
- Admin Tokens: 30 days
- Database Passwords: 180 days
- Encryption Keys: 365 days

### 4. Access Control

- Use environment-specific secrets
- Implement least privilege access
- Audit secret access regularly

### 5. Git Security

Ensure these files are in `.gitignore`:
```
.secrets.json
.master_key
secrets.db
*.key
```

## Deployment

### Development

```bash
# Use .env file for convenience
cp .env.example .env
# Edit .env with development values

# Initialize secrets
python manage_secrets.py init
```

### Production

```bash
# Set master key via environment
export MASTER_KEY="your-production-master-key"

# Or use a key management service
export MASTER_KEY_FILE="/secure/path/to/master.key"

# Load secrets from secure storage
python manage_secrets.py set TTS_API_KEY --value "$TTS_API_KEY"
python manage_secrets.py set ADMIN_TOKEN --value "$ADMIN_TOKEN"
```

### Docker

```dockerfile
# Dockerfile
FROM python:3.9

# Copy encrypted secrets (not the master key!)
COPY .secrets.json /app/.secrets.json

# Master key provided at runtime
ENV MASTER_KEY=""

# Run with:
# docker run -e MASTER_KEY="$MASTER_KEY" myapp
```

### Kubernetes

```yaml
# secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: talk2me-master-key
type: Opaque
stringData:
  master-key: "your-master-key"

---
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
      - name: talk2me
        env:
        - name: MASTER_KEY
          valueFrom:
            secretKeyRef:
              name: talk2me-master-key
              key: master-key
```

## Troubleshooting

### Lost Master Key

If you lose the master key:
1. You'll need to recreate all secrets
2. Generate new master key: `python manage_secrets.py init`
3. Re-enter all secret values

### Corrupted Secrets File

```bash
# Check integrity
python manage_secrets.py verify

# If corrupted, restore from backup or reinitialize
```

### Permission Errors

```bash
# Fix file permissions
chmod 600 .secrets.json .master_key
chown $USER:$USER .secrets.json .master_key
```

## Monitoring

### Audit Logs

Review secret access patterns:
```bash
# View all audit entries
python manage_secrets.py audit

# Check specific secret
python manage_secrets.py audit TTS_API_KEY

# Export for analysis
python manage_secrets.py audit > audit.log
```

### Rotation Monitoring

```bash
# Check rotation status
python manage_secrets.py check-rotation

# Set up cron job for automatic checks
0 0 * * * /path/to/python /path/to/manage_secrets.py check-rotation
```

## Migration Guide

### From Environment Variables

```bash
# Automatic migration
python manage_secrets.py migrate

# Manual migration
export OLD_API_KEY="your-key"
python manage_secrets.py set API_KEY --value "$OLD_API_KEY"
unset OLD_API_KEY
```

### From .env Files

```python
# migrate_env.py
from dotenv import dotenv_values
from secrets_manager import get_secrets_manager

env_values = dotenv_values('.env')
manager = get_secrets_manager()

for key, value in env_values.items():
    if key.endswith('_KEY') or key.endswith('_TOKEN'):
        manager.set(key, value, {'migrated_from': '.env'})
```

## API Reference

### Python API

```python
from secrets_manager import get_secret, set_secret

# Get a secret
api_key = get_secret('TTS_API_KEY', default='')

# Set a secret
set_secret('NEW_API_KEY', 'value', metadata={'service': 'external'})

# Advanced usage
from secrets_manager import get_secrets_manager

manager = get_secrets_manager()
manager.rotate('API_KEY')
manager.schedule_rotation('TOKEN', days=30)
```

### Flask CLI

```bash
# Via Flask CLI
flask secrets-list
flask secrets-set
flask secrets-rotate
flask secrets-check-rotation
```

## Security Considerations

1. **Never log secret values**
2. **Use secure random generation for new secrets**
3. **Implement proper access controls**
4. **Regular security audits**
5. **Incident response plan for compromised secrets**

## Future Enhancements

- Integration with cloud KMS (AWS, Azure, GCP)
- Hardware security module (HSM) support
- Secret sharing (Shamir's Secret Sharing)
- Time-based access controls
- Automated compliance reporting