Disclaimer: This blog post is automatically generated from project documentation and technical proposals using AI assistance. The content represents our development journey and architectural decisions. Code examples are simplified illustrations and may not reflect the exact production implementation.

Table of Contents

The Session Security Challenge

When Caroline and I implemented secure sessions for the UI service, we knew session keys needed rotation for security. But we had a problem: how do you rotate keys without invalidating every user’s session?

The naive approach:

  1. Generate new key
  2. Restart service
  3. All users logged out
  4. Users frustrated

There had to be a better way.

Multi-Key Strategy

The solution is elegant: support multiple session keys simultaneously. New sessions use the newest key, but old sessions can be validated with any key in the array.

graph TB
    subgraph "Key Array: [newKey, oldKey]"
        K1[Key 1: Newest]
        K2[Key 2: Old]
    end

    subgraph "New Session"
        NS[Create Session] --> SN[Sign with Key 1]
    end

    subgraph "Old Session"
        OS[Receive Session] --> VER[Verify with Key 2]
        VER --> RS[Re-sign with Key 1]
    end

    K1 --> SN
    K1 --> RS
    K2 --> VER

    style SN fill:#c8e6c9
    style RS fill:#fff9c4
    style VER fill:#c8e6c9

Here’s the magic:

  • Signing: Always uses the first key (newest)
  • Verification: Tries all keys in the array
  • Re-signing: When an old session is decoded, it’s automatically re-signed with the new key

Over time, all sessions migrate to the new key naturally.

Implementation with Fastify

We use @fastify/secure-session which supports key arrays out of the box.

Key Generation

First, generate cryptographically secure keys using the official tool:

npx @fastify/secure-session > session-key1
npx @fastify/secure-session > session-key2

Store as Docker Secret

Store keys as hex-encoded JSON to avoid encoding issues:

[
  "0af3686875d05bddf5618b10930a82b6da3683985a294088683eeaff8c54f5cc",
  "b170bba371a7ce808162e9cfb9449e928ec085ec050a8bbf24c8"
]

Save to deployments/docker/secrets/session_keys.json.

Load Keys in Application

The UI service loads keys asynchronously using Promise.allSettled:

import { readFile } from 'node:fs/promises';

const sessionKeysPath = '/run/secrets/session_keys';

// Load keys from Docker secret
const keysResult = await Promise.allSettled([
  readFile(sessionKeysPath, 'utf8'),
]);

if (keysResult[0].status === 'rejected') {
  fastify.log.error(
    { error: keysResult[0].reason },
    'Failed to load session keys'
  );
  throw new Error('Session keys are required');
}

// Parse JSON and convert hex to Buffer
const keysJson = JSON.parse(keysResult[0].value);
const sessionKeys: Buffer[] = keysJson.map((key: string) =>
  Buffer.from(key, 'hex')
);

fastify.log.info(
  { keyCount: sessionKeys.length },
  'Loaded session keys for rotation'
);

Register with Fastify

Pass the key array to secure-session:

fastify.register(fastifySecureSession, {
  sessionName: 'session',
  key: sessionKeys, // Array enables rotation!
  cookie: {
    path: '/',
    httpOnly: true,
    secure: true,
    sameSite: 'lax',
    maxAge: 60 * 60 * 24, // 24 hours
  },
});

That’s it. Fastify handles the rest:

  • Signs new sessions with sessionKeys[0]
  • Verifies sessions against all keys in the array
  • Automatically re-signs old sessions

Rotation Workflow

Here’s how key rotation works in practice:

timeline
    title Key Rotation Timeline
    Week 1 : Old Key Only : [oldKey] : All sessions use oldKey
    Week 2 : Add New Key : [newKey, oldKey] : New sessions use newKey : Old sessions still valid
    Week 3 : Grace Period : Old sessions re-signed : Most sessions migrated to newKey
    Week 4 : Remove Old Key : [newKey] : Clean state : All sessions using newKey

Step-by-Step Process

1. Generate new key:

npx @fastify/secure-session > session-key-new

2. Update JSON (new key first):

["NEW_KEY_HEX_HERE", "OLD_KEY_HEX_HERE"]

3. Restart UI service:

docker-compose restart ui

At this point:

  • New logins get sessions signed with new key
  • Existing sessions still work (verified with old key)
  • On next request, old sessions re-signed with new key

4. Wait for grace period:

Most sessions will re-sign within the cookie maxAge window (24 hours in our case). Wait longer for safety—we use 1 week.

5. Remove old key:

["NEW_KEY_HEX_HERE"]

6. Restart again:

docker-compose restart ui

Now only the new key is active. Any remaining old sessions are invalidated (users need to re-login), but most users already migrated.

Makefile Automation

Caroline suggested automating the rotation process with Make targets. We created several utilities:

Generate Initial Keys

session-keys-generate:
\t@echo "Generating session keys..."
\t@npx @fastify/secure-session | tr -d '\\n' > session-key1
\t@npx @fastify/secure-session | tr -d '\\n' > session-key2
\t@jq -n --rawfile k1 session-key1 --rawfile k2 session-key2 \\
\t\t'[$$k1, $$k2]' > secrets/session_keys.json
\t@rm session-key1 session-key2
\t@echo "Keys generated: secrets/session_keys.json"

Rotate to New Key

session-keys-rotate:
\t@echo "Rotating session keys..."
\t@npx @fastify/secure-session | tr -d '\\n' > session-key-new
\t@jq --rawfile new session-key-new '. = [$$new] + .' \\
\t\tsecrets/session_keys.json > secrets/session_keys_tmp.json
\t@mv secrets/session_keys_tmp.json secrets/session_keys.json
\t@rm session-key-new
\t@echo "New key added to front of array"

Remove Old Key

session-keys-cleanup:
\t@echo "Removing old session key..."
\t@jq '.[:-1]' secrets/session_keys.json > secrets/session_keys_tmp.json
\t@mv secrets/session_keys_tmp.json secrets/session_keys.json
\t@echo "Old key removed"

Check Status

session-keys-status:
\t@echo "Current session keys configuration:"
\t@jq 'length' secrets/session_keys.json | \\
\t\tawk '{print "Key count:", $$1}'
\t@docker-compose logs ui | grep -i "session keys" | tail -5

Now rotation is simple:

make session-keys-rotate
make restart-ui
# Wait 1 week
make session-keys-cleanup
make restart-ui

Security Considerations

No Fallback Logic: The code fails fast if keys are missing. This is intentional—better to fail loudly than run with insecure defaults.

Hex Encoding: Keys are stored as hex strings to avoid JSON encoding issues with binary data.

Docker Secrets: Keys are never in environment variables or config files—they’re mounted as Docker secrets at runtime.

gitignore: Session key files are explicitly excluded from version control.

Key Learnings

Working on session key rotation with Caroline reinforced several principles:

Zero Downtime is Possible: With thoughtful design, security operations don’t require service interruptions.

Automation Prevents Mistakes: Makefile targets ensure we follow the same process every time, reducing human error.

Structured Logging Matters: Logging key count and load status helps verify rotation worked correctly.

Grace Periods are Critical: Don’t rush to remove old keys. Give sessions time to migrate naturally.

Fastify Support: Using @fastify/secure-session with key arrays was the key (pun intended). The library handles all the complexity of trying multiple keys and re-signing.

The multi-key approach gives us enterprise-grade security without compromising user experience. Users stay logged in during key rotation, and we can rotate keys as frequently as security policy requires. When an audit asks “when did you last rotate session keys?”, we have a clear answer and a repeatable process.

Caroline and Claude helped us realize that good security doesn’t have to disrupt operations. By designing for rotation from the start, we avoided the trap of “we’ll add rotation later” that often never happens.