---

You’ve deployed your EKS cluster, your app is running beautifully… and then a pod restarts and all your data vanishes. Welcome to the world of ephemeral container storage. By default, containers are stateless—everything inside them is temporary. If you’re running databases, shared file systems, or applications that pull secrets from AWS Secrets Manager, you need a proper storage strategy.

In this guide, you’ll learn what the Kubernetes Container Storage Interface (CSI) is and why it matters, how to set up three essential CSI drivers (EBS, EFS, and Secrets Store) using Terraform, and when to use each one based on your workload requirements.

Conceptual Overview: Why Do We Need CSI Drivers?

Historically, Kubernetes included “in-tree” volume plugins, meaning the code to communicate with AWS EBS (Elastic Block Store) was built directly into the core Kubernetes codebase.

However, this architecture didn’t scale well. To fix this, the community moved to the Container Storage Interface (CSI)—a standard that allows storage vendors to develop plugins independently of the Kubernetes release cycle. Think of CSI like the USB standard, but for storage. It provides a universal interface so any storage vendor can build a “driver” that plugs into Kubernetes, without the Kubernetes team having to maintain the code for every vendor.

Crucially, starting in EKS 1.30+, the default gp2 storage class is no longer created automatically. If you deploy a cluster and try to create a PersistentVolumeClaim (PVC) without installing a CSI driver, it will stay in a Pending state forever. You are now responsible for installing the appropriate CSI driver and setting up the storage classes.

Let’s look at the three most common CSI options on AWS.

1. Amazon EBS CSI Driver (Block Storage)

Best for: Databases (MySQL, PostgreSQL, MongoDB) and any workload that needs high-performance, single-node block storage. Access Mode: ReadWriteOnce (RWO) — The volume can only be mounted to a single node at a time. Limitation: It is bound to a single Availability Zone (AZ). If a node fails, the pod must be scheduled on another node in the same AZ to attach the volume.

Infrastructure: Terraform

To install the EBS CSI driver, we need to create an IAM role with the AmazonEBSCSIDriverPolicy and associate it with the cluster using EKS Pod Identity (or IRSA). Then, we install the EKS Add-on.

data "aws_iam_policy_document" "ebs_csi_driver" {
    statement {
        effect = "Allow"
        principals {
            type        = "Service"
            identifiers = ["pods.eks.amazonaws.com"]
        }
        actions = ["sts:AssumeRole", "sts:TagSession"]
    }
}

resource "aws_iam_role" "ebs_csi_driver" {
    name               = "${aws_eks_cluster.eks.name}-ebs-csi-driver"
    assume_role_policy = data.aws_iam_policy_document.ebs_csi_driver.json
}

resource "aws_iam_role_policy_attachment" "ebs_csi_driver" {
    policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
    role       = aws_iam_role.ebs_csi_driver.name
}

resource "aws_eks_pod_identity_association" "ebs_csi_driver" {
    cluster_name    = aws_eks_cluster.eks.name
    namespace       = "kube-system"
    service_account = "ebs-csi-controller-sa"
    role_arn        = aws_iam_role.ebs_csi_driver.arn
}

resource "aws_eks_addon" "ebs_csi_driver" {
    cluster_name             = aws_eks_cluster.eks.name
    addon_name               = "aws-ebs-csi-driver"
    addon_version            = "v1.63.0-eksbuild.1"
    service_account_role_arn = aws_iam_role.ebs_csi_driver.arn
}

Application: Kubernetes StatefulSet

When deploying databases, you typically use a StatefulSet with a volumeClaimTemplates block. Here, we request 5Gi of storage using the gp2 storage class.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: myapp
spec:
  serviceName: nginx
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: aputra/myapp-195:v2
          volumeMounts:
            - name: data
              mountPath: /data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        storageClassName: gp2
        accessModes: [ReadWriteOnce]
        resources:
          requests:
            storage: 5Gi

2. Amazon EFS CSI Driver (Shared File System)

Best for: Machine learning datasets, WordPress media folders, CI/CD workspaces, or anything requiring shared files across multiple pods. Access Mode: ReadWriteMany (RWX) — The volume can be mounted by multiple nodes and pods simultaneously. Advantages: Spans across multiple AZs.

Infrastructure: Terraform

Unlike EBS (which provisions volumes automatically via the driver), EFS requires us to provision the actual file system and its mount targets in AWS first. Notice that our mount targets need a security group that allows NFS traffic (port 2049).

resource "aws_efs_file_system" "eks" {
    creation_token = "eks" 
    performance_mode = "generalPurpose"
    throughput_mode = "bursting"
    encrypted = true
}

# Mount targets in multiple subnets (AZs)
resource "aws_efs_mount_target" "zone_a" {
    file_system_id  = aws_efs_file_system.eks.id
    subnet_id       = aws_subnet.private_zone1.id
    security_groups = [aws_eks_cluster.eks.vpc_config[0].cluster_security_group_id]
}

resource "aws_efs_mount_target" "zone_b" {
    file_system_id  = aws_efs_file_system.eks.id
    subnet_id       = aws_subnet.private_zone2.id
    security_groups = [aws_eks_cluster.eks.vpc_config[0].cluster_security_group_id]
}

# (IAM Role setup for EFS CSI omitted for brevity, similar to EBS)

resource "kubernetes_storage_class_v1" "efs"{
    metadata {
        name = "efs"
    }
    storage_provisioner = "efs.csi.aws.com"
    parameters = {
        provisioningMode = "efs-ap"
        fileSystemId     = aws_efs_file_system.eks.id
        directoryPerms   = "700"
    }
    mount_options = ["iam"]
}

Application: Kubernetes PVC

Once the driver and StorageClass are created, workloads can request shared storage simply by specifying ReadWriteMany and the efs storage class:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myapp
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs
  resources:
    requests:
      storage: 5Gi # EFS is elastic; this size is ignored but required by the spec

3. Secrets Store CSI Driver

Best for: Securely passing credentials (like DB passwords or API keys) to your application without hardcoding them or relying purely on native Kubernetes Secrets. How it Works: It fetches secrets from AWS Secrets Manager (or Parameter Store) and mounts them directly into the pod as files. Optionally, it can also sync them into native Kubernetes Secret objects.

Infrastructure: Terraform

We install both the generic Secrets Store CSI driver and the AWS-specific provider via Helm.

resource "helm_release" "secrets_csi_driver" {
    name       = "secrets-store-csi-driver"
    repository = "https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts"
    chart      = "secrets-store-csi-driver"
    namespace  = "kube-system"
    version    = "1.4.3"
    set = [{
        name  = "syncSecret.enabled"
        value = "true"
    }]
}

resource "helm_release" "secrets_csi_driver_aws_provider" {
    name       = "secrets-store-csi-driver-provider-aws"
    repository = "https://aws.github.io/secrets-store-csi-driver-provider-aws"
    chart      = "secrets-store-csi-driver-provider-aws"
    namespace  = "kube-system"
    version    = "0.3.8"
}

We also create an IAM role for the application’s ServiceAccount (using IRSA) that permits secretsmanager:GetSecretValue.

Application: SecretProviderClass & Deployment

In Kubernetes, we define a SecretProviderClass telling the driver which secret to pull from AWS Secrets Manager:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: myapp-aws-secrets
spec:
  provider: aws
  parameters:
    region: us-east-2
    objects: |
      - objectName: staging/myapp-secret-v3
        objectType: secretsmanager
        jmesPath:
            - path: username
              objectAlias: myusername
            - path: password
              objectAlias: mypassword

Then, we configure our Deployment to mount this CSI volume:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      serviceAccountName: myapp
      containers:
        - name: myapp
          image: nginx:1.14.2
          volumeMounts:
            - name: secrets
              mountPath: /mnt/secrets
              readOnly: true
      volumes:
        - name: secrets
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: myapp-aws-secrets

Now, your pod securely reads the AWS Secret directly from the filesystem!

Gotchas and Practical Tips

1. EBS volumes are AZ-locked. An EBS volume exists in a single Availability Zone. If your StatefulSet pod gets rescheduled to a node in a different AZ, it will be stuck in Pending because the volume cannot be attached across AZs. Use pod topology spread constraints or node affinity rules to prevent this.

2. The storage field in EFS PVCs is misleading. The Kubernetes spec requires you to set a storage value in your PVC’s resources.requests (e.g., 5Gi). However, EFS is elastic—it grows and shrinks automatically—so this value is completely ignored by the driver. Don’t be confused into thinking you’re capping your EFS storage at 5Gi.

Conclusion

Persistent storage is one of the most critical pieces of a production EKS setup, and getting it right from the start saves you from painful data-loss incidents down the road.

Key Takeaways

1. You must bring your own storage drivers: As of EKS 1.30, CSI drivers and StorageClasses are your responsibility to install.
2. Use EBS for Databases (RWO): When high performance and data locality matter, block storage is the right choice.
3. Use EFS for Shared File Systems (RWX): When multiple pods across different Availability Zones need to read and write to the same files.
4. Use Secrets Store CSI for Secrets: It bridges the gap between AWS Secrets Manager and your pods, avoiding the need to manually pass secrets around in plain text.

By structuring your infrastructure as code with Terraform, you ensure that these dependencies are always correctly provisioned alongside your cluster, saving you from headaches during application deployment!