Solved: Do you separate template browsing from deployment in your internal IaC tooling?

Source: Dev.to

The Problem: Coupled Discovery & Deployment

Coupling IaC template browsing with deployment often results in:

1. Template sprawl & cognitive overload – Hundreds of Terraform modules, CloudFormation templates, Helm charts, etc., sit in a monolithic repo with little categorisation or search capability.
2. Error‑prone deployments – Poorly documented parameters force users to guess or manually extract values, leading to misconfigurations, failed runs, and security gaps.
3. Slow provisioning & low self‑service adoption – Teams rely on a central DevOps/platform group, creating bottlenecks and delaying projects.
4. Lack of standardisation & guardrails – Ad‑hoc copying/modifying of templates drifts away from organisational policies and compliance requirements.

Three Primary Solutions for Separation

Why GitOps Often Wins

• Single source of truth – All templates and version history reside in Git.
• Strong auditability – Every change is captured in a PR, with reviewers, comments, and approvals.
• Built‑in review process – Mis‑configurations are caught early during code review.

The trade‑off is a more complex initial setup (repositories, CI pipelines, policy enforcement), but the long‑term benefits in compliance and reliability are substantial.

How a Dedicated Template Catalog Works

1. Web‑based interface – presents a curated list of IaC templates.
2. Each entry includes:
   • Name & description – human‑readable.
   • Categorisation – e.g., Web App, Database, Networking.
   • Input parameters – descriptions, defaults, validation rules.
   • Estimated cost – optional, if integrated with cost‑management tools.
   • Links – to detailed documentation or source code.
3. User fills a form with the required variables.
4. The portal triggers deployment via an underlying CI/CD pipeline or directly through IaC tools, abstracting away the raw commands.

Examples of catalog platforms: Spotify’s Backstage, HashiCorp Terraform Cloud/Enterprise Workspaces, or custom internal portals.

Sample Template: AWS S3 Static Website (Terraform)

# main.tf for S3 static website module (in your template repo)

resource "aws_s3_bucket" "website_bucket" {
  bucket = var.bucket_name
  acl    = "public-read"

  website {
    index_document = "index.html"
    error_document = "error.html"
  }

  tags = {
    Environment = var.environment
    Project     = var.project
  }
}

resource "aws_s3_bucket_policy" "website_policy" {
  bucket = aws_s3_bucket.website_bucket.id

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect    = "Allow",
        Principal = "*",
        Action    = "s3:GetObject",
        Resource  = "${aws_s3_bucket.website_bucket.arn}/*"
      }
    ]
  })
}

variable "bucket_name" {
  description = "The unique name for the S3 bucket."
  type        = string
}

variable "environment" {
  description = "The deployment environment (e.g., dev, staging, prod)."
  type        = string
}

variable "project" {
  description = "The project name associated with this resource."
  type        = string
}

In a catalog UI, the variables bucket_name, environment, and project appear as form fields with the descriptions above, guiding the user. Submitting the form triggers a Terraform run in a pre‑configured workspace, provisioning the static website automatically.

Benefits of Decoupling Discovery & Deployment

• Enhanced Discoverability – A central, searchable catalog makes finding the right template trivial.
• Improved User Experience – Intuitive forms, clear descriptions, and pre‑validation reduce mistakes.
• Standardisation & Governance – Only vetted templates are exposed, and organisational policies are enforced automatically.
• Self‑Service Enablement – Teams can provision infrastructure without waiting on a central platform group, accelerating delivery cycles.

Bottom Line

Separating IaC‑template discovery from the deployment process—whether via a UI‑driven catalog, an enriched CLI, or a GitOps workflow—eliminates “template sprawl,” reduces cognitive load, and creates a safer, faster path to infrastructure provisioning. Choose the approach that aligns with your team’s maturity, tooling preferences, and governance requirements, and watch self‑service adoption and developer productivity soar.

Template Discovery Approaches

Approved templates are available, promoting best practices and compliance.

Benefits

• Reduced Cognitive Load – Users don’t need to know IaC syntax or CLI commands.

Considerations

• Development & Maintenance Overhead – Building and maintaining a custom portal can be resource‑intensive.
• Potential for Drift – If the portal’s template definitions aren’t kept in sync with the actual IaC code, it can lead to confusion.
• Limited Flexibility – May not support highly custom or experimental deployments easily.

CLI‑Based Approach

This approach emphasizes a robust command‑line interface (CLI) that provides specific commands for:

1. Template discovery – list available templates.
2. Detailed parameter explanation – show variables, descriptions, defaults, etc.
3. Pre‑deployment validation – verify inputs before the actual deployment command.

It’s ideal for teams comfortable with CLI tools and scripting.

How It Works

Instead of a GUI, users interact with a custom CLI tool (or a set of well‑defined scripts). Typical commands include:

• list-templates – show all discoverable templates.
• describe-template <template> – display rich metadata, variables, and documentation.
• validate-template <template> – run static checks on the template.
• deploy-template <template> [params…] – perform the actual deployment.

The discovery phase focuses on providing rich metadata directly at the command line, often pulling from structured documentation or code comments.

Directory Structure for IaC Templates

iac/
├── modules/
│   ├── s3-website/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── README.md
│   └── rds-instance/
│       ├── main.tf
│       ├── variables.tf
│       └── README.md
├── environments/
│   ├── dev/
│   ├── prod/
│   └── staging/
└── templates/
    ├── s3-static-site.yaml   # CloudFormation template
    ├── ec2-webserver.json    # CloudFormation template
    └── eks-cluster.tf        # Terraform root module for EKS

• Each module’s README.md provides detailed documentation.
• The templates/ folder contains ready‑to‑apply root modules.

Sample iac-helper Bash Script (Simplified)

#!/usr/bin/env bash

# Root directory for IaC assets
IAC_ROOT_DIR="iac"

# ----------------------------------------------------------------------
# List all available IaC templates (modules and root‑module files)
# ----------------------------------------------------------------------
list_templates() {
  echo "Available IaC Templates:"

  # List modules
  find "$IAC_ROOT_DIR/modules" -maxdepth 1 -mindepth 1 -type d -print0 |
    xargs -0 -n 1 basename |
    while read -r template; do
      echo "  - $template"
    done

  # List root‑module files (Terraform, CloudFormation, JSON)
  find "$IAC_ROOT_DIR/templates" -maxdepth 1 -type f \
    \( -name "*.tf" -o -name "*.yaml" -o -name "*.json" \) -print0 |
    xargs -0 -n 1 basename |
    sed -E 's/\.(tf|yaml|json)//' |
    while read -r template; do
      echo "  - $template (Root Module)"
    done
}

# ----------------------------------------------------------------------
# Show detailed information about a specific template
# ----------------------------------------------------------------------
describe_template() {
  local TEMPLATE_NAME=$1

  if [[ -d "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME" ]]; then
    echo "Description for module: $TEMPLATE_NAME"
    cat "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME/README.md" 2>/dev/null ||
      echo "No README.md found."
    echo
    echo "Variables:"
    terraform-docs md "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME" |
      sed -n '/^| Name | Description | Type | Default | Required |/,/^$/p'

  elif [[ -f "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.tf" ]]; then
    echo "Description for root module: $TEMPLATE_NAME.tf"
    terraform-docs md "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.tf" |
      sed -n '/^| Name | Description | Type | Default | Required |/,/^$/p'

  elif [[ -f "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.yaml" ]]; then
    echo "Description for CloudFormation template: $TEMPLATE_NAME.yaml"
    yq e '.Parameters | to_entries[] | .key + ": " + .value.Description' \
      "$IAC_ROOT_DIR/templates/$TEMPLATE_NAME.yaml"

  else
    echo "Template '$TEMPLATE_NAME' not found."
  fi
}

# ----------------------------------------------------------------------
# Deploy a template (placeholder – real implementation would be more complex)
# ----------------------------------------------------------------------
deploy_template() {
  local TEMPLATE_NAME=$1
  # ... logic to run `terraform apply` or `aws cloudformation create-stack`
  echo "Deploying $TEMPLATE_NAME..."
  # Example (simplified):
  # terraform apply -var="bucket_name=$2" -var="environment=$3" \
  #   "$IAC_ROOT_DIR/modules/$TEMPLATE_NAME"
}

# ----------------------------------------------------------------------
# Command‑line interface
# ----------------------------------------------------------------------
case "$1" in
  list)
    list_templates
    ;;
  describe)
    if [[ -z "$2" ]]; then
      echo "Usage: iac-helper describe <template-name>"
      exit 1
    fi
    describe_template "$2"
    ;;
  deploy)
    echo "Deploy command not fully implemented in this example."
    echo "Usage: iac-helper deploy <template-name> [params...]"
    ;;
  *)
    echo "Usage: iac-helper [list|describe|deploy] ..."
    ;;
esac

Example Usage

iac-helper list
iac-helper describe s3-website
iac-helper deploy s3-website bucket_name=my-unique-site-name environment=dev

Advantages & Disadvantages of the CLI Approach

GitOps‑Based Approach

GitOps separates the desired state of infrastructure (defined in Git) from the operational process that brings that state to life.

Repository Types

GitOps Operator

Tools such as Argo CD or Flux CD watch the configuration repositories, detect changes, and automatically apply the referenced templates with the supplied values.

Workflow

1. Browse the template repository to locate the needed module or chart.
2. Create a PR in the configuration repository that references the chosen template and supplies required parameters.
3. Review the PR—automated policy checks and human approvals ensure compliance.
4. Merge the PR; the GitOps operator detects the change and reconciles the infrastructure to match the declared state.

Closing Thoughts

Whether you opt for a UI‑driven catalog, a powerful CLI, or a GitOps‑centric workflow, the key is to decouple discovery from deployment. This separation reduces cognitive load, enforces governance, and empowers teams to provision infrastructure quickly and safely. Align the chosen approach with your organization’s maturity and tooling preferences to unlock faster, more reliable self‑service.