再也不要提交机密：从 AWS Secrets Manager 生成 .env 文件

Source: Dev.to

TL;DR

将密钥存放在 AWS Secrets Manager 中。使用 Python 脚本按需生成 .env 文件。再也不要把凭证提交到仓库。

The Problem

每个团队最终都会把密钥提交到代码库。GitHub 去年检测到超过 1200 万条泄露的凭证。

常见的做法都有缺陷：

• .gitignore 在开发者忘记添加或全新克隆后需要通过 Slack 索要文件时会失效。
• SOPS 加密 仍然把文件放进 git，增加密钥管理负担，并导致合并冲突噩梦。
• .env.example 模板会过时，需要手动复制。

我们需要更好的方案：密钥完全不在仓库中，且为开发者提供无摩擦的使用体验。

The Solution

密钥存放在 AWS Secrets Manager。开发者只需执行一个命令即可生成 .env 文件：

make env
# .env 在本地生成，随时可用

该文件已被 .gitignore，永不进入版本控制。当 AWS 中的密钥发生变化时，开发者重新生成即可获取最新值。

Implementation

1. Organize Secrets in AWS

按应用和环境组织密钥：

/myapp/dev/database   → {"DB_HOST": "...", "DB_PASSWORD": "..."}
/myapp/dev/api-keys   → {"STRIPE_KEY": "...", "SENDGRID_KEY": "..."}
/myapp/prod/database  → {"DB_HOST": "...", "DB_PASSWORD": "..."}
/myapp/prod/api-keys  → {"STRIPE_KEY": "...", "SENDGRID_KEY": "..."}

使用 AWS CLI 创建密钥：

aws secretsmanager create-secret \
  --name /myapp/dev/database \
  --secret-string '{"DB_HOST":"localhost","DB_PASSWORD":"devpass123"}'

2. The Python Script

generate_env.py 从 Secrets Manager 生成 .env 文件：

#!/usr/bin/env python3
"""
Generate .env file from AWS Secrets Manager.

Usage:
    python generate_env.py dev
    python generate_env.py prod --force
"""

import argparse
import json
import os
import sys
from pathlib import Path

import boto3
from botocore.exceptions import ClientError, NoCredentialsError

# Configuration
APP_NAME = "myapp"
AWS_REGION = os.environ.get("AWS_REGION", "us-east-1")
ENV_FILE = ".env"

SECRET_KEYS = ["database", "api-keys", "third-party"]

def get_secret(secret_name: str, region: str) -> dict:
    """Fetch a secret from AWS Secrets Manager."""
    client = boto3.client("secretsmanager", region_name=region)

    try:
        response = client.get_secret_value(SecretId=secret_name)
        return json.loads(response.get("SecretString", "{}"))
    except ClientError as e:
        if e.response["Error"]["Code"] == "ResourceNotFoundException":
            print(f"  Warning: Secret '{secret_name}' not found")
            return {}
        raise

def validate_aws_credentials() -> bool:
    """Check if AWS credentials are configured."""
    try:
        sts = boto3.client("sts")
        identity = sts.get_caller_identity()
        print(f"Authenticated as: {identity['Arn']}")
        return True
    except NoCredentialsError:
        print("Error: AWS credentials not found.")
        print("\nFix with one of:")
        print("  1. aws configure")
        print("  2. Set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY")
        print("  3. Use IAM role (if on AWS)")
        return False

def fetch_all_secrets(environment: str, region: str) -> dict:
    """Fetch all secrets for the environment."""
    all_secrets = {}
    for key in SECRET_KEYS:
        secret_path = f"/{APP_NAME}/{environment}/{key}"
        print(f"  Fetching: {secret_path}")
        all_secrets.update(get_secret(secret_path, region))
    return all_secrets

def generate_env_content(secrets: dict) -> str:
    """Generate .env content from secrets."""
    lines = [
        "# Auto-generated from AWS Secrets Manager",
        "# DO NOT COMMIT THIS FILE",
        "",
    ]
    for key, value in sorted(secrets.items()):
        if isinstance(value, str) and " " in value:
            value = f'"{value}"'
        lines.append(f"{key}={value}")
    return "\n".join(lines) + "\n"

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("environment", choices=["dev", "staging", "prod"])
    parser.add_argument("-f", "--force", action="store_true")
    parser.add_argument("-o", "--output", default=ENV_FILE)
    args = parser.parse_args()

    print(f"Generating .env for '{args.environment}'\n")

    if not validate_aws_credentials():
        sys.exit(1)

    secrets = fetch_all_secrets(args.environment, AWS_REGION)

    if not secrets:
        print(f"\nError: No secrets found at /{APP_NAME}/{args.environment}/*")
        sys.exit(1)

    print(f"\nFound {len(secrets)} secret values")

    content = generate_env_content(secrets)
    path = Path(args.output)

    if path.exists() and not args.force:
        if input(f"{args.output} exists. Overwrite? [y/N]: ").lower() != "y":
            sys.exit(0)

    path.write_text(content)
    print(f"Generated: {args.output}")

if __name__ == "__main__":
    main()

3. Shell Wrapper and Makefile

Shell wrapper (generate-env.sh)：

#!/bin/bash
# generate-env.sh
set -e
ENV=${1:-dev}

if ! python3 -c "import boto3" 2>/dev/null; then
    pip3 install boto3 --quiet
fi

python3 "$(dirname "$0")/generate_env.py" "$ENV" "${@:2}"

Makefile targets：

.PHONY: env env-dev env-prod env-dry

env:
	@./scripts/generate-env.sh dev

env-dev:
	@./scripts/generate-env.sh dev

env-prod:
	@./scripts/generate-env.sh prod

env-dry:
	@./scripts/generate-env.sh dev --dry-run

4. GitHub Actions with OIDC

无需存储凭证。使用 OIDC 来假设 AWS 角色：

name: Deploy

on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/github-actions
          aws-region: us-east-1

      - name: Generate .env
        run: |
          pip install boto3
          python scripts/generate_env.py prod --force

      - name: Deploy
        run: |
          # Your deployment commands
          echo "Deploying..."

      - name: Cleanup
        if: always()
        run: rm -f .env

5. GitLab CI

GitLab OIDC 同样的模式：

deploy:
  stage: deploy
  image: python:3.11-slim
  script:
    - pip install boto3
    - |
      export $(printf "AWS_ACCESS_KEY_ID=%s AWS_SECRET_ACCESS_KEY=%s AWS_SESSION_TOKEN=%s"
      $(aws sts assume-role-with-web-identity
      --role-arn ${AWS_ROLE_ARN}
      --role-session-name "gitlab-${CI_PIPELINE_ID}"
      --web-identity-token ${CI_JOB_JWT_V2}
      --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]'
      --output text))
    - python scripts/generate_env.py prod --force
    - echo "Deploying..."
  after_script:
    - rm -f .env

6. IAM Permissions

开发者需要一条允许读取相应密钥的策略，例如：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": [
        "arn:aws:secretsmanager:*:*:secret:/myapp/*"
      ]
    }
  ]
}