Consolidate encryption keys in platform API

.github/workflows/ai-workspace-pr-check.yml

@@ -17,7 +17,8 @@ jobs:
   pr-check:
     runs-on: ubuntu-24.04
     env:
-      AUTH_JWT_SECRET_KEY: ai-workspace-pr-check-secret-key
+      ENCRYPTION_KEY: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+      AUTH_JWT_SECRET_KEY: "fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210"
     steps:
       - name: Checkout code
         uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1

distribution/all-in-one/docker-compose.yaml

@@ -40,6 +40,8 @@ services:
       context: ../../portals/developer-portal
       dockerfile: Dockerfile
     container_name: devportal
+    environment:
+      - APIP_DP_PLATFORMAPI_JWTSECRET:${AUTH_JWT_SECRET_KEY:-}
     ports:
       - "3001:3001"
     volumes:
@@ -73,7 +75,8 @@ services:
       - DATABASE_MAX_IDLE_CONNS=10
       - DATABASE_CONN_MAX_LIFETIME=300
       - DATABASE_EXECUTE_SCHEMA_DDL=true
-      - DATABASE_SUBSCRIPTION_TOKEN_ENCRYPTION_KEY=${DATABASE_SUBSCRIPTION_TOKEN_ENCRYPTION_KEY}
+      - AUTH_JWT_SECRET_KEY=${AUTH_JWT_SECRET_KEY:-}
+      - ENCRYPTION_KEY=${ENCRYPTION_KEY:-}
     depends_on:
       postgres:
         condition: service_healthy

docs/ai-workspace/configuration.md

@@ -92,6 +92,7 @@ Sensitive values (JWT signing key, database password) must be passed as environm
 | Platform API env variable | Description |
 |--------------------------|-------------|
 | `AUTH_JWT_SECRET_KEY` | JWT signing key (required when `auth.jwt.enabled = true`) |
+| `ENCRYPTION_KEY` | 32-byte key (64 hex / base64) — encrypts secrets & subscription tokens (required) |
 | `DATABASE_PASSWORD` | Database password |
 
 ## Environment Variable Override

docs/ai-workspace/features/secrets-management.md

@@ -337,7 +337,7 @@ The following environment variable controls encryption for the Platform API:
 
 | Env Var | Description |
 |---------|-------------|
-| `PLATFORM_SECRET_ENCRYPTION_KEY` | 32-byte AES-256 key as 64 hex characters or base64. If unset, a random ephemeral key is auto-generated at startup — secrets stored in that session will be unreadable after a restart. |
+| `ENCRYPTION_KEY` | 32-byte AES-256 key as 64 hex characters or base64. **Required in every mode and never auto-generated** — the Platform API fails to start if it is missing or malformed. |
 
 Generate a stable key with:
 
@@ -355,7 +355,7 @@ openssl rand -hex 32
 Then copy the output value into your `.env` file:
 
 ```sh
-PLATFORM_SECRET_ENCRYPTION_KEY=a3f1e2d4b5c6...
+ENCRYPTION_KEY=a3f1e2d4b5c6...
 ```
 
-> **Warning:** Always set a stable `PLATFORM_SECRET_ENCRYPTION_KEY` in any environment where secrets must persist across restarts or across multiple replicas. An ephemeral auto-generated key will make existing encrypted secrets unreadable after a restart.
+> **Warning:** Use the **same** stable `ENCRYPTION_KEY` across restarts and across all replicas. Changing it makes existing encrypted secrets unreadable.

platform-api/README.md

@@ -241,21 +241,19 @@ AUTH_IDP_ENABLED=true             →  IDP mode        (JWKS-based verification)
 ```
 
 > **Demo mode (`APIP_DEMO_MODE`).** Defaults to `true`; an explicit `false`/`0` opts into
-> production-grade startup checks. With demo mode off, the server will not fall back to an
-> ephemeral secret encryption key (set `PLATFORM_SECRET_ENCRYPTION_KEY` or
-> `DATABASE_ENCRYPTION_KEY`) and warns loudly if `AUTH_JWT_SKIP_VALIDATION=true`.
+> production-grade startup checks. Note that `ENCRYPTION_KEY` and `AUTH_JWT_SECRET_KEY` are **required**.
 
 ---
 
 #### Local JWT Mode (default)
 
-The server validates HMAC-signed tokens using `AUTH_JWT_SECRET_KEY`. Set `AUTH_JWT_SKIP_VALIDATION=true` only in local development environments where you do not have a token issuer available — all bearer values will be accepted without any signature check.
+The server signs and validates HMAC login tokens using `AUTH_JWT_SECRET_KEY` — a 32-byte key (64 hex chars or base64). Set `AUTH_JWT_SKIP_VALIDATION=true` only in local development environments where you do not have a token issuer available — all bearer values will be accepted without any signature check.
 
-| Variable | Default | Description |
-|---|---|---|
-| `AUTH_JWT_SECRET_KEY` | `your-secret-key-change-in-production` | HMAC signing key for token verification |
-| `AUTH_JWT_ISSUER` | `platform-api` | Expected `iss` claim value |
-| `AUTH_JWT_SKIP_VALIDATION` | `false` | Skip signature verification — **development only** |
+| Variable | Default | Description                                                         |
+|---|---|---------------------------------------------------------------------|
+| `AUTH_JWT_SECRET_KEY` | _(empty)_ | HMAC key for signing/verifying login JWTs — 32-byte value (64 hex or base64; `openssl rand -hex 32`) |
+| `AUTH_JWT_ISSUER` | `platform-api` | Expected `iss` claim value                                          |
+| `AUTH_JWT_SKIP_VALIDATION` | `false` | Skip signature verification — **development only**                  |
 | `DEV_MODE` | `false` | Suppresses the startup warning when `AUTH_JWT_SKIP_VALIDATION=true` |
 
 Local development with no token issuer:
@@ -369,7 +367,16 @@ In **IDP mode with `AUTH_IDP_VALIDATION_MODE=role`**, IDP roles are resolved fro
 | `DATABASE_PASSWORD` | _(empty)_ | Postgres password |
 | `DATABASE_SSL_MODE` | `disable` | Postgres SSL mode (`disable`, `require`, `verify-full`) |
 | `DATABASE_EXECUTE_SCHEMA_DDL` | `true` | Set to `false` when the DB user lacks DDL privileges |
-| `DATABASE_SUBSCRIPTION_TOKEN_ENCRYPTION_KEY` | _(empty)_ | 32-byte key (64 hex or 44 base64 chars) for AES-256-GCM token encryption. |
+
+---
+
+### Encryption
+
+`ENCRYPTION_KEY` protects all at-rest encryption (secrets, subscription tokens, WebSub HMAC secrets). It is **never auto-generated** — the operator must provide it.
+
+| Variable | Default | Description |
+|---|---|---|
+| `ENCRYPTION_KEY` | _(empty)_ | **Required.** 32-byte AES-256 key as 64 hex chars or base64 (32 bytes). Generate with `openssl rand -hex 32`. Startup fails if missing or malformed. |
 
 ---
 

platform-api/config/config.go

@@ -18,14 +18,11 @@
 package config
 
 import (
-	"crypto/rand"
+	"encoding/base64"
 	"encoding/hex"
 	"encoding/json"
-	"errors"
 	"fmt"
 	"log/slog"
-	"os"
-	"path/filepath"
 	"reflect"
 	"strings"
 	"sync"
@@ -85,6 +82,8 @@ type Server struct {
 	OpenAPISpecPath            string `koanf:"openapi_spec_path"`
 	LLMTemplateDefinitionsPath string `koanf:"llm_template_definitions_path"`
 
+	EncryptionKey string `koanf:"encryption_key"`
+
 	Database         Database         `koanf:"database"`
 	Auth             Auth             `koanf:"auth"`
 	WebSocket        WebSocket        `koanf:"websocket"`
@@ -214,16 +213,6 @@ type Database struct {
 	MaxOpenConns    int    `koanf:"max_open_conns"`
 	MaxIdleConns    int    `koanf:"max_idle_conns"`
 	ConnMaxLifetime int    `koanf:"conn_max_lifetime"`
-
-	EncryptionKey                  string `koanf:"encryption_key"`
-	SubscriptionTokenEncryptionKey string `koanf:"subscription_token_encryption_key"`
-	SecretEncryptionKey            string `koanf:"secret_encryption_key"`
-	// SecretEncryptionKeyFile is the path to a 32-byte binary key file used for secret encryption.
-	// Honoured in both demo and non-demo mode when neither SecretEncryptionKey nor
-	// EncryptionKey is set. In demo mode the file is auto-generated on first startup and
-	// reused on subsequent restarts; in non-demo mode the file must already exist (a missing
-	// or unreadable file is fatal). Matches the gateway controller key-management pattern.
-	SecretEncryptionKeyFile string `koanf:"secret_encryption_key_file"`
 }
 
 // DefaultDevPortal holds default DevPortal configuration for new organizations.
@@ -369,171 +358,40 @@ func LoadConfig(configPath string) (*Server, error) {
 		return nil, err
 	}
 
-	if cfg.Auth.JWT.Enabled && cfg.Auth.JWT.SecretKey == "" {
-		if !demoMode() {
-			return nil, fmt.Errorf(
-				"AUTH_JWT_SECRET_KEY must be configured when APIP_DEMO_MODE=false and JWT authentication is enabled; " +
-					"generate a secret with: openssl rand -hex 32",
-			)
+	if cfg.Auth.JWT.Enabled {
+		if cfg.Auth.JWT.SecretKey == "" {
+			return nil, fmt.Errorf("AUTH_JWT_SECRET_KEY is required when JWT authentication is enabled; " +
+				"generate one with: openssl rand -hex 32")
 		}
-		key, err := generateRandomSecret()
-		if err != nil {
-			return nil, fmt.Errorf("failed to generate JWT secret key: %w", err)
-		}
-		cfg.Auth.JWT.SecretKey = key
-		slog.Warn("AUTH_JWT_SECRET_KEY not set — generated an ephemeral demo key (restart will invalidate all sessions)",
-			slog.String("AUTH_JWT_SECRET_KEY", key))
-	}
-
-	// Resolve the secret key file path: explicit config → default alongside the DB file.
-	if cfg.Database.SecretEncryptionKeyFile == "" && cfg.Database.Path != "" {
-		cfg.Database.SecretEncryptionKeyFile = filepath.Join(filepath.Dir(cfg.Database.Path), "secret-encryption.key")
-	}
-
-	// SecretEncryptionKey is optional when the shared DATABASE_ENCRYPTION_KEY is configured;
-	// server.go resolves the final key via: SecretEncryptionKey → EncryptionKey.
-	// Only fail (or auto-generate in demo mode) when no key source is available at all.
-	if cfg.Database.SecretEncryptionKey == "" && cfg.Database.EncryptionKey == "" {
-		if cfg.Database.SecretEncryptionKeyFile != "" {
-			demoMode := strings.ToLower(strings.TrimSpace(os.Getenv("APIP_DEMO_MODE")))
-			isDemoMode := demoMode != "false" && demoMode != "0"
-			if isDemoMode {
-				// Demo mode: auto-generate the key file on first start, reload on subsequent starts.
-				hexKey, err := loadOrGenerateSecretKeyFile(cfg.Database.SecretEncryptionKeyFile)
-				if err == nil {
-					cfg.Database.SecretEncryptionKey = hexKey
-				} else {
-					slog.Warn("APIP_DEMO_MODE: could not initialise secret key file, falling back to ephemeral key",
-						slog.String("path", cfg.Database.SecretEncryptionKeyFile), slog.Any("err", err))
-				}
-			} else {
-				// Non-demo mode: the key file must already exist — never auto-generate.
-				hexKey, err := loadSecretKeyFile(cfg.Database.SecretEncryptionKeyFile)
-				if err != nil {
-					return nil, fmt.Errorf("failed to load secret key file: %w", err)
-				}
-				cfg.Database.SecretEncryptionKey = hexKey
-			}
+		if !valid32ByteKey(cfg.Auth.JWT.SecretKey) {
+			return nil, fmt.Errorf("invalid AUTH_JWT_SECRET_KEY: must be 64 hex characters or " +
+				"base64 decoding to 32 bytes (generate one with: openssl rand -hex 32)")
 		}
 	}
 
-	if cfg.Database.SecretEncryptionKey == "" && cfg.Database.EncryptionKey == "" {
-		// APIP_DEMO_MODE defaults to enabled when unset; only an explicit
-		// "false"/"0" opts out and requires a configured encryption key.
-		demoMode := strings.ToLower(strings.TrimSpace(os.Getenv("APIP_DEMO_MODE")))
-		if demoMode == "false" || demoMode == "0" {
-			return nil, fmt.Errorf("no encryption key configured for secrets management. " +
-				"Set PLATFORM_SECRET_ENCRYPTION_KEY (secret-specific), DATABASE_ENCRYPTION_KEY (shared), " +
-				"or DATABASE_SECRET_ENCRYPTION_KEY_FILE (key file). " +
-				"Generate one with: openssl rand -hex 32. " +
-				"To allow an ephemeral key in a single-node dev environment, set APIP_DEMO_MODE=true")
-		}
-
-		// Demo mode with no usable key file — fall back to an ephemeral key.
-		// Secrets will not survive restarts.
-		key, err := generateRandomSecret()
-		if err != nil {
-			return nil, fmt.Errorf("failed to generate secret encryption key: %w", err)
-		}
-		cfg.Database.SecretEncryptionKey = key
-		slog.Warn("APIP_DEMO_MODE: using an ephemeral random key — encrypted secrets will be unreadable after restart. " +
-			"Set DATABASE_SECRET_ENCRYPTION_KEY_FILE, PLATFORM_SECRET_ENCRYPTION_KEY, or DATABASE_ENCRYPTION_KEY.")
+	if cfg.EncryptionKey == "" {
+		return nil, fmt.Errorf("ENCRYPTION_KEY is required; generate one with: openssl rand -hex 32")
+	}
+	if !valid32ByteKey(cfg.EncryptionKey) {
+		return nil, fmt.Errorf("invalid ENCRYPTION_KEY: must be 64 hex characters or " +
+			"base64 decoding to 32 bytes (generate one with: openssl rand -hex 32)")
 	}
 
 	return cfg, nil
 }
 
-func generateRandomSecret() (string, error) {
-	b := make([]byte, 32)
-	if _, err := rand.Read(b); err != nil {
-		return "", err
+// valid32ByteKey reports whether keyStr is a 32-byte key encoded as 64 hex characters
+// or base64 decoding to 32 bytes — matching utils.DeriveEncryptionKey's acceptance.
+func valid32ByteKey(keyStr string) bool {
+	if len(keyStr) == 64 {
+		if k, err := hex.DecodeString(keyStr); err == nil && len(k) == 32 {
+			return true
+		}
 	}
-	return hex.EncodeToString(b), nil
-}
-
-// demoMode reports whether APIP_DEMO_MODE is enabled.
-// Defaults to true when the variable is unset.
-func demoMode() bool {
-	v := strings.ToLower(strings.TrimSpace(os.Getenv("APIP_DEMO_MODE")))
-	if v == "" {
+	if k, err := base64.StdEncoding.DecodeString(keyStr); err == nil && len(k) == 32 {
 		return true
 	}
-	return v == "true" || v == "1"
-}
-
-const secretKeySize = 32 // AES-256
-
-// loadOrGenerateSecretKeyFile loads a 32-byte binary key file from filePath, creating it
-// (and any missing parent directories) on first run. This mirrors the gateway controller's
-// KeyManager pattern: raw binary key file, 0600 permissions, validate size on load.
-// Returns the key as a 64-char hex string for use with DeriveEncryptionKey.
-//
-// Concurrent first-time callers are safe: generateSecretKeyFile uses O_CREATE|O_EXCL so
-// only one writer succeeds; others see os.ErrExist and fall through to loadSecretKeyFile.
-func loadOrGenerateSecretKeyFile(filePath string) (string, error) {
-	err := generateSecretKeyFile(filePath)
-	switch {
-	case err == nil:
-		slog.Info("APIP_DEMO_MODE: generated and persisted secret encryption key — encrypted secrets will survive restarts",
-			slog.String("path", filePath),
-			slog.String("hint", "Set PLATFORM_SECRET_ENCRYPTION_KEY or DATABASE_ENCRYPTION_KEY for production or multi-replica deployments"))
-	case errors.Is(err, os.ErrExist):
-		// Another initializer already created the file — load the winner's key.
-	default:
-		return "", err
-	}
-	return loadSecretKeyFile(filePath)
-}
-
-// generateSecretKeyFile creates parent directories and writes 32 cryptographically
-// random bytes to filePath with permissions 0600. Uses O_CREATE|O_EXCL so concurrent
-// first-time callers are safe: only one writer succeeds, others get os.ErrExist.
-// Mirrors gateway-controller's generateKeyFile.
-func generateSecretKeyFile(filePath string) error {
-	if err := os.MkdirAll(filepath.Dir(filePath), 0700); err != nil {
-		return fmt.Errorf("failed to create key directory: %w", err)
-	}
-	key := make([]byte, secretKeySize)
-	if _, err := rand.Read(key); err != nil {
-		return fmt.Errorf("failed to generate random key: %w", err)
-	}
-	f, err := os.OpenFile(filePath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600)
-	if err != nil {
-		// Propagate os.ErrExist so the caller can distinguish "already created" from other errors.
-		return err
-	}
-	_, err = f.Write(key)
-	if closeErr := f.Close(); closeErr != nil && err == nil {
-		err = closeErr
-	}
-	if err != nil {
-		os.Remove(filePath) // best-effort cleanup of a partial write
-		return fmt.Errorf("failed to write key file %s: %w", filePath, err)
-	}
-	return nil
-}
-
-// loadSecretKeyFile reads the key file, validates its size, warns if world-readable,
-// and returns the key as a 64-char hex string.
-func loadSecretKeyFile(filePath string) (string, error) {
-	info, err := os.Stat(filePath)
-	if err != nil {
-		return "", fmt.Errorf("failed to stat secret key file %s: %w", filePath, err)
-	}
-	if info.Mode().Perm()&0004 != 0 {
-		slog.Warn("Secret encryption key file is world-readable — consider restricting permissions to 0600",
-			slog.String("path", filePath),
-			slog.String("permissions", info.Mode().Perm().String()))
-	}
-	data, err := os.ReadFile(filePath)
-	if err != nil {
-		return "", fmt.Errorf("failed to read secret key file %s: %w", filePath, err)
-	}
-	if len(data) != secretKeySize {
-		return "", fmt.Errorf("secret key file %s has wrong size: expected %d bytes, got %d", filePath, secretKeySize, len(data))
-	}
-	slog.Info("APIP_DEMO_MODE: loaded persisted secret encryption key", slog.String("path", filePath))
-	return hex.EncodeToString(data), nil
+	return false
 }
 
 // envToKoanfKey maps a lowercased environment variable name to its koanf dot-notation key.
@@ -557,6 +415,8 @@ func envToKoanfKey(s string) string {
 		return "llm_template_definitions_path"
 	case "enable_scope_validation":
 		return "enable_scope_validation"
+	case "encryption_key":
+		return "encryption_key"
 
 	// Database
 	case "database_driver":
@@ -581,14 +441,6 @@ func envToKoanfKey(s string) string {
 		return "database.max_idle_conns"
 	case "database_conn_max_lifetime":
 		return "database.conn_max_lifetime"
-	case "database_encryption_key":
-		return "database.encryption_key"
-	case "database_subscription_token_encryption_key":
-		return "database.subscription_token_encryption_key"
-	case "platform_secret_encryption_key":
-		return "database.secret_encryption_key"
-	case "database_secret_encryption_key_file":
-		return "database.secret_encryption_key_file"
 
 	// Auth
 	case "auth_skip_paths":

platform-api/config/config.toml

@@ -24,6 +24,14 @@ log_level  = "INFO"   # DEBUG | INFO | WARN | ERROR
 log_format = "text"   # text | json
 port       = "9243"
 
+# ---------------------------------------------------------------------------
+# Encryption
+# ---------------------------------------------------------------------------
+# Single 32-byte key (64 hex chars or base64) used for ALL at-rest encryption
+# (secrets, subscription tokens, WebSub HMAC secrets)
+# Env: ENCRYPTION_KEY. Generate with: openssl rand -hex 32.
+# encryption_key = ""
+
 # ---------------------------------------------------------------------------
 # Database
 # ---------------------------------------------------------------------------
@@ -44,8 +52,8 @@ driver = "sqlite3"          # "sqlite3" or "postgres"
 # ---------------------------------------------------------------------------
 
 # JWT (local HMAC) — issues signed tokens after file-based login.
-# secret_key is auto-generated at startup when not set (ephemeral — sessions reset on restart).
-# Set AUTH_JWT_SECRET_KEY env var or secret_key here to persist sessions across restarts.
+# secret_key is a 32-byte key (64 hex chars or base64; openssl rand -hex 32)
+# Env: AUTH_JWT_SECRET_KEY. Generate with: openssl rand -hex 32.
 [auth.jwt]
 enabled = true
 issuer  = "platform-api"