Building a Production-Ready CI/CD Pipeline: Automating Infrastructure with Terraform, GitHub Actions, and Ansible

Source: Dev.to

Project Goals and Overview

The primary objective of this project was to create a fully automated deployment pipeline for a multi‑service TODO application with complete infrastructure automation. The solution needed to address several key requirements:

Core Requirements

• Complete automation from code commit to production deployment
• Infrastructure provisioning using declarative configuration
• Automated configuration management for consistent server setup
• Zero‑downtime deployments with SSL/TLS termination
• Drift detection to maintain infrastructure consistency
• Distributed tracing for debugging microservices interactions
• Security‑first approach with encrypted secrets and minimal attack surface

Technology Stack Selected

• Infrastructure as Code: Terraform for AWS resource provisioning
• Configuration Management: Ansible for server configuration and application deployment
• CI/CD Orchestration: GitHub Actions for workflow automation
• Containerization: Docker and Docker Compose for service isolation
• Reverse Proxy: Traefik for routing, load balancing, and automatic SSL
• Observability: Zipkin for distributed request tracing
• Message Queue: Redis for asynchronous log processing

The end goal was a system where infrastructure changes and application updates could be deployed with a single git push, with built‑in safety mechanisms including drift detection, email notifications, and manual approval gates for production environments.

System Architecture and Design

The application follows a microservices pattern with seven distinct services, each written in a language best suited to its responsibilities.

Architecture Diagram

Figure 1: Complete microservices architecture showing all services, their technologies, and data flow.

Service Responsibilities

Network Architecture

Figure 2: Docker networking showing isolated app-network with Traefik as the only external gateway.

All services communicate via a dedicated Docker bridge network named app-network, providing:

• Isolation from the host system
• Service‑to‑service communication via container names (DNS)
• No exposed ports except through Traefik
• Encrypted traffic between external clients and Traefik

Infrastructure as Code with Terraform

Terraform was chosen for its declarative configuration, state management, and mature AWS provider.

AWS Resources Provisioned

1. EC2 Instance

# Data source ensures we always use the latest Ubuntu AMI
data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical's AWS account

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

# EC2 Instance resource definition
resource "aws_instance" "todo_app" {
  ami                    = data.aws_ami.ubuntu.id
  instance_type          = var.instance_type
  key_name               = aws_key_pair.deployer.key_name
  vpc_security_group_ids = [aws_security_group.todo_app.id]

  tags = {
    Name        = "todo-app-server-v2"
    Environment = "production"
    Project     = "hngi13-stage6"
  }
}

2. Security Group

resource "aws_security_group" "todo_app" {
  name        = "todo-app-sg"
  description = "Security group for TODO application"

  # HTTP access for initial Let's Encrypt challenges
  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # HTTPS access
  ingress {
    description = "HTTPS"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # SSH access (restricted)
  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["YOUR.TRUSTED.IP/32"]
  }

  # Allow all outbound traffic
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}