Distributed Caching Patterns for Kubernetes

Kubernetes brings unique challenges to caching. Pods come and go. IPs change. Storage is ephemeral by default. Here's how to build reliable distributed caching in K8s environments.

Pattern 1: External Managed Cache

The simplest approach—use cloud provider's managed cache service:

# AWS ElastiCache, GCP Memorystore, Azure Cache
apiVersion: v1
kind: Secret
metadata:
  name: redis-credentials
type: Opaque
data:
  host: cmVkaXMuYWJjMTIzLmNhY2hlLmFtYXpvbmF3cy5jb20=
  port: NjM3OQ==
  password: c2VjcmV0cGFzc3dvcmQ=

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
spec:
  template:
    spec:
      containers:
        - name: api
          env:
            - name: REDIS_HOST
              valueFrom:
                secretKeyRef:
                  name: redis-credentials
                  key: host

Pros: Zero maintenance, automatic failover, scaling. Cons: Cloud lock-in, network latency, cost.

Pattern 2: StatefulSet Redis Cluster

Run Redis inside Kubernetes with persistent storage:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
spec:
  serviceName: redis
  replicas: 3
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - name: redis
          image: redis:7-alpine
          ports:
            - containerPort: 6379
          volumeMounts:
            - name: data
              mountPath: /data
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi

Pattern 3: Sidecar Cache

Run a local cache alongside each application pod:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-with-cache
spec:
  template:
    spec:
      containers:
        - name: api
          image: my-api:latest
          env:
            - name: CACHE_HOST
              value: "localhost"  # Sidecar is on localhost
            - name: CACHE_PORT
              value: "6379"

        - name: cache-sidecar
          image: redis:7-alpine
          ports:
            - containerPort: 6379
          resources:
            requests:
              memory: "64Mi"
            limits:
              memory: "128Mi"

Pros: Ultra-low latency (no network hop), simple. Cons: Cache not shared between pods, more memory per pod.

Pattern 4: Two-Tier Caching

Combine local sidecar (L1) with shared Redis (L2):

// Application code
class TwoTierCache {
    constructor(localCache, sharedCache) {
        this.l1 = localCache;   // Sidecar Redis
        this.l2 = sharedCache;  // Cluster Redis
    }

    async get(key) {
        // Check L1 first (fastest)
        let value = await this.l1.get(key);
        if (value) return value;

        // Check L2
        value = await this.l2.get(key);
        if (value) {
            // Populate L1 for next time
            await this.l1.set(key, value, { ex: 60 });
            return value;
        }

        return null;
    }

    async set(key, value, ttl) {
        // Write to both
        await Promise.all([
            this.l1.set(key, value, { ex: Math.min(ttl, 60) }),
            this.l2.set(key, value, { ex: ttl })
        ]);
    }
}

Pattern 5: Redis Operator

Use Kubernetes operators for production-grade Redis:

# Using Redis Operator (e.g., Spotahome or Redis Enterprise)
apiVersion: databases.spotahome.com/v1
kind: RedisCluster
metadata:
  name: production-cache
spec:
  numberOfMasters: 3
  replicasPerMaster: 1
  image: redis:7-alpine
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      cpu: 500m
      memory: 1Gi
  storage:
    persistentVolumeClaim:
      metadata:
        name: redis-data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 20Gi

Operators handle: automatic failover, scaling, backups, and upgrades.

Service Discovery

Kubernetes services provide stable endpoints:

apiVersion: v1
kind: Service
metadata:
  name: redis
spec:
  clusterIP: None  # Headless for StatefulSet
  selector:
    app: redis
  ports:
    - port: 6379

---
# Connect using DNS
# redis-0.redis.default.svc.cluster.local
# redis-1.redis.default.svc.cluster.local

Resource Planning

Size your cache pods appropriately:

• Memory: Account for data + overhead (Redis uses ~1.2x data size)
• CPU: Redis is single-threaded; one core handles 100K+ ops/sec
• Storage: If persisting, plan for 2x data size for rewrites
• Network: Consider network policies for security