talk2me/SECRETS_MANAGEMENT.md

Secrets Management Documentation

This document describes the secure secrets management system implemented in Talk2Me.

Overview

Talk2Me uses a comprehensive secrets management system that provides:

• Encrypted storage of sensitive configuration
• Secret rotation capabilities
• Audit logging
• Integrity verification
• CLI management tools
• Environment variable integration

Architecture

Components

1. SecretsManager (secrets_manager.py)

   • Handles encryption/decryption using Fernet (AES-128)
   • Manages secret lifecycle (create, read, update, delete)
   • Provides audit logging
   • Supports secret rotation

2. Configuration System (config.py)

   • Integrates secrets with Flask configuration
   • Environment-specific configurations
   • Validation and sanitization

3. CLI Tool (manage_secrets.py)

   • Command-line interface for secret management
   • Interactive and scriptable

Security Features

• Encryption: AES-128 encryption using cryptography.fernet
• Key Derivation: PBKDF2 with SHA256 (100,000 iterations)
• Master Key: Stored separately with restricted permissions
• Audit Trail: All access and modifications logged
• Integrity Checks: Verify secrets haven't been tampered with

Quick Start

1. Initialize Secrets

python manage_secrets.py init

This will:

• Generate a master encryption key
• Create initial secrets (Flask secret key, admin token)
• Prompt for required secrets (TTS API key)

2. Set a Secret

# Interactive (hidden input)
python manage_secrets.py set TTS_API_KEY

# Direct (be careful with shell history)
python manage_secrets.py set TTS_API_KEY --value "your-api-key"

# With metadata
python manage_secrets.py set API_KEY --value "key" --metadata '{"service": "external-api"}'

3. List Secrets

python manage_secrets.py list

Output:

Key                            Created             Last Rotated         Has Value
-------------------------------------------------------------------------------------
FLASK_SECRET_KEY              2024-01-15          2024-01-20          ✓
TTS_API_KEY                   2024-01-15          Never               ✓
ADMIN_TOKEN                   2024-01-15          2024-01-18          ✓

4. Rotate Secrets

# Rotate a specific secret
python manage_secrets.py rotate ADMIN_TOKEN

# Check which secrets need rotation
python manage_secrets.py check-rotation

# Schedule automatic rotation
python manage_secrets.py schedule-rotation API_KEY 30  # Every 30 days

Configuration

Environment Variables

The secrets manager checks these locations in order:

1. Encrypted secrets storage (.secrets.json)
2. SECRET_<KEY> environment variable
3. <KEY> environment variable
4. Default value

Master Key

The master encryption key is loaded from:

1. MASTER_KEY environment variable
2. .master_key file (default)
3. Auto-generated if neither exists

Important: Protect the master key!

• Set file permissions: chmod 600 .master_key
• Back it up securely
• Never commit to version control

Flask Integration

Secrets are automatically loaded into Flask configuration:

# In app.py
from config import init_app as init_config
from secrets_manager import init_app as init_secrets

app = Flask(__name__)
init_config(app)
init_secrets(app)

# Access secrets
api_key = app.config['TTS_API_KEY']

CLI Commands

Basic Operations

# List all secrets
python manage_secrets.py list

# Get a secret value (requires confirmation)
python manage_secrets.py get TTS_API_KEY

# Set a secret
python manage_secrets.py set DATABASE_URL

# Delete a secret
python manage_secrets.py delete OLD_API_KEY

# Rotate a secret
python manage_secrets.py rotate ADMIN_TOKEN

Advanced Operations

# Verify integrity of all secrets
python manage_secrets.py verify

# Migrate from environment variables
python manage_secrets.py migrate

# View audit log
python manage_secrets.py audit
python manage_secrets.py audit TTS_API_KEY --limit 50

# Schedule rotation
python manage_secrets.py schedule-rotation API_KEY 90

Security Best Practices

1. File Permissions

# Secure the secrets files
chmod 600 .secrets.json
chmod 600 .master_key

2. Backup Strategy

• Back up .master_key separately from .secrets.json
• Store backups in different secure locations
• Test restore procedures regularly

3. Rotation Policy

Recommended rotation intervals:

• API Keys: 90 days
• Admin Tokens: 30 days
• Database Passwords: 180 days
• Encryption Keys: 365 days

4. Access Control

• Use environment-specific secrets
• Implement least privilege access
• Audit secret access regularly

5. Git Security

Ensure these files are in .gitignore:

.secrets.json
.master_key
secrets.db
*.key

Deployment

Development

# Use .env file for convenience
cp .env.example .env
# Edit .env with development values

# Initialize secrets
python manage_secrets.py init

Production

# Set master key via environment
export MASTER_KEY="your-production-master-key"

# Or use a key management service
export MASTER_KEY_FILE="/secure/path/to/master.key"

# Load secrets from secure storage
python manage_secrets.py set TTS_API_KEY --value "$TTS_API_KEY"
python manage_secrets.py set ADMIN_TOKEN --value "$ADMIN_TOKEN"

Docker

# Dockerfile
FROM python:3.9

# Copy encrypted secrets (not the master key!)
COPY .secrets.json /app/.secrets.json

# Master key provided at runtime
ENV MASTER_KEY=""

# Run with:
# docker run -e MASTER_KEY="$MASTER_KEY" myapp

Kubernetes

# secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: talk2me-master-key
type: Opaque
stringData:
  master-key: "your-master-key"

---
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
      - name: talk2me
        env:
        - name: MASTER_KEY
          valueFrom:
            secretKeyRef:
              name: talk2me-master-key
              key: master-key

Troubleshooting

Lost Master Key

If you lose the master key:

1. You'll need to recreate all secrets
2. Generate new master key: python manage_secrets.py init
3. Re-enter all secret values

Corrupted Secrets File

# Check integrity
python manage_secrets.py verify

# If corrupted, restore from backup or reinitialize

Permission Errors

# Fix file permissions
chmod 600 .secrets.json .master_key
chown $USER:$USER .secrets.json .master_key

Monitoring

Audit Logs

Review secret access patterns:

# View all audit entries
python manage_secrets.py audit

# Check specific secret
python manage_secrets.py audit TTS_API_KEY

# Export for analysis
python manage_secrets.py audit > audit.log

Rotation Monitoring

# Check rotation status
python manage_secrets.py check-rotation

# Set up cron job for automatic checks
0 0 * * * /path/to/python /path/to/manage_secrets.py check-rotation

Migration Guide

From Environment Variables

# Automatic migration
python manage_secrets.py migrate

# Manual migration
export OLD_API_KEY="your-key"
python manage_secrets.py set API_KEY --value "$OLD_API_KEY"
unset OLD_API_KEY

From .env Files

# migrate_env.py
from dotenv import dotenv_values
from secrets_manager import get_secrets_manager

env_values = dotenv_values('.env')
manager = get_secrets_manager()

for key, value in env_values.items():
    if key.endswith('_KEY') or key.endswith('_TOKEN'):
        manager.set(key, value, {'migrated_from': '.env'})

API Reference

Python API

from secrets_manager import get_secret, set_secret

# Get a secret
api_key = get_secret('TTS_API_KEY', default='')

# Set a secret
set_secret('NEW_API_KEY', 'value', metadata={'service': 'external'})

# Advanced usage
from secrets_manager import get_secrets_manager

manager = get_secrets_manager()
manager.rotate('API_KEY')
manager.schedule_rotation('TOKEN', days=30)

Flask CLI

# Via Flask CLI
flask secrets-list
flask secrets-set
flask secrets-rotate
flask secrets-check-rotation

Security Considerations

1. Never log secret values
2. Use secure random generation for new secrets
3. Implement proper access controls
4. Regular security audits
5. Incident response plan for compromised secrets

Future Enhancements

• Integration with cloud KMS (AWS, Azure, GCP)
• Hardware security module (HSM) support
• Secret sharing (Shamir's Secret Sharing)
• Time-based access controls
• Automated compliance reporting