Kubernetes v1.35: A Better Way to Pass Service Account Tokens to CSI Drivers
Source: Kubernetes Blog
Understanding the Existing Approach
When CSI drivers use the TokenRequests feature, they can request service‑account tokens for workload identity by configuring the tokenRequests field in the CSIDriver spec. The tokens are currently passed to drivers as part of the volume attributes map, using the key:
csi.storage.k8s.io/serviceAccount.tokens
Why volume_context Is Problematic
-
Logging risk – The
protostanitizertool that many CSI drivers use does not treatvolume_contextas sensitive, so tokens can end up in logs when gRPC requests are recorded.- Example: CVE‑2023‑2878 in the Secrets Store CSI Driver.
- Example: CVE‑2024‑3744 in the Azure File CSI Driver.
-
Inconsistent sanitisation – Each driver that wants to avoid this issue must implement its own sanitisation logic, leading to inconsistency across drivers.
The CSI specification already defines a secrets field in NodePublishVolumeRequest that is meant for exactly this kind of sensitive information. However, moving the tokens there outright would break existing drivers that expect them in volume_context.
How the Opt‑in Mechanism Works
Kubernetes v1.35 introduces an opt‑in mechanism that lets CSI drivers choose how they receive service‑account tokens. Existing drivers continue to work unchanged, while drivers can move to the more appropriate secrets field when they are ready.
New field in the CSIDriver spec
# CAUTION: this is an example configuration.
# Do not use this for your own cluster!
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: example-csi-driver
spec:
# ... existing fields ...
tokenRequests:
- audience: "example.com"
expirationSeconds: 3600
# New field for opting into secrets delivery
serviceAccountTokenInSecrets: true # defaults to false
serviceAccountTokenInSecrets: false(default) – Tokens are placed involumeContextunder the keycsi.storage.k8s.io/serviceAccount.tokens(current behavior).serviceAccountTokenInSecrets: true– Tokens are placed only in thesecretsfield with the same key.
About the Beta Release
- The
CSIServiceAccountTokenSecretsfeature gate is enabled by default on both kubelet and kube‑apiserver. - Because
serviceAccountTokenInSecretsdefaults tofalse, enabling the feature gate does not change any existing behavior. - All drivers continue receiving tokens via
volume_contextunless they explicitly opt in. - This backward‑compatible approach allowed the feature to start at beta rather than alpha.
Guide for CSI Driver Authors
If you maintain a CSI driver that uses service‑account tokens, follow these steps to adopt the new mechanism safely.
1. Add Fallback Logic
Update your driver to check both locations for tokens. This makes the driver compatible with the old and new approaches.
const serviceAccountTokenKey = "csi.storage.k8s.io/serviceAccount.tokens"
func getServiceAccountTokens(req *csi.NodePublishVolumeRequest) (string, error) {
// New behavior – check the secrets field first
if tokens, ok := req.Secrets[serviceAccountTokenKey]; ok {
return tokens, nil
}
// Existing behavior – fall back to volume context
if tokens, ok := req.VolumeContext[serviceAccountTokenKey]; ok {
return tokens, nil
}
return "", fmt.Errorf("service account tokens not found")
}
This fallback logic is backward‑compatible and safe to ship in any driver version, even before clusters upgrade to v1.35.
2. Rollout Sequence
Driver preparation (can happen anytime)
- Add the fallback logic.
- Release the new driver version (or back‑port to maintenance branches).
Cluster upgrade & feature enablement
- Upgrade kube‑apiserver to v1.35 or later.
- Upgrade kubelet to v1.35 or later on all nodes.
- Ensure the CSI driver version with fallback logic is deployed (if not already done).
- Complete the CSI driver DaemonSet rollout across all nodes.
- Update the
CSIDrivermanifest to setserviceAccountTokenInSecrets: true.
3. Important Constraints
-
Timing is critical. If your CSI driver DaemonSet and
CSIDriverobject live in the same manifest or Helm chart, you need two separate updates:- Deploy the new driver version first and wait for the DaemonSet rollout to finish.
- Then update the
CSIDriverspec to enableserviceAccountTokenInSecrets.
-
Do not update the
CSIDriverbefore all driver pods have rolled out.
If you do, volume mounts will fail on nodes still running the old driver version, because those pods only look atvolume_context.
Why This Matters
Adopting this feature helps in several ways:
- Eliminates the risk of accidentally logging service‑account tokens in driver logs.
- Aligns token delivery with the CSI specification’s intended location for sensitive data.
- Reduces the need for driver‑specific sanitisation logic, promoting consistency across the ecosystem.
By opting in, you make your driver more secure and future‑proof while preserving compatibility with existing clusters.
Account Tokens as Part of Volume Context in gRPC Requests
- It uses the CSI specification’s designated field for sensitive data, which feels right.
- The protosanitizer tool automatically handles the
secretsfield correctly, so you don’t need driver‑specific workarounds. - It’s opt‑in, so you can migrate at your own pace without breaking existing deployments.
Call to Action
We (Kubernetes SIG Storage) encourage CSI driver authors to adopt this feature and provide feedback on the migration experience.
- If you have thoughts on the API design or run into any issues during adoption, please reach out to us on the #csi channel on Kubernetes Slack (for an invitation, visit ).
You can follow along on KEP‑5538 to track progress across the upcoming Kubernetes releases.