Another E2E Solution delivered. This time with CI/CD, AWS EventBridge and ECS Fargate

Source: Dev.to

Solution Overview

To wrap up the year, I built my latest end‑to‑end (E2E) project. It is a side project, but it will help us at work. We have a service that uploads documents from a third‑party system. This integration requires authentication, and the system enforces a monthly password rotation. When the password expires, uploads and downloads start failing, quickly turning into an operational issue.

To eliminate manual updates and the risk of someone forgetting the rotation, I created an automation that handles the entire process.

The solution is a Python worker that uses Selenium with headless Chromium. It runs on a schedule and is backed by a full CI/CD pipeline. On every push to the main branch, GitHub Actions assumes an AWS IAM role via OIDC (no access keys), builds the Docker image, and pushes it to Amazon ECR. The workflow then registers a new ECS task definition revision, updating only the container image.

Architecture Design

CI/CD

• GitHub Actions: Triggers on pushes to main.
• OIDC authentication: Assumes an AWS IAM role without static credentials.
• Docker build & push: Image is built and stored in Amazon ECR.
• Task definition update: Registers a new ECS task definition revision with the updated image.

Execution (EventBridge)

• Amazon EventBridge triggers the task every 29 days.
• The schedule ensures the password rotation occurs before the monthly expiration.

ECS Cluster

• The task runs on ECS Fargate in a public subnet with a public IP and outbound traffic allowed.
• When triggered, Fargate starts the container, runs automation.py, launches Selenium with Chromium and Chromedriver, logs into the system, performs the password rotation, and exits.
• On success, the task finishes automatically with exit code 0.
• If an exception occurs, logs are sent to CloudWatch, and the error is reported to a Slack alerts channel.

Architecture Decisions

Using ECS Fargate instead of AWS Lambda was a deliberate choice. Running Selenium with Chromium on Lambda typically requires custom layers and fine‑tuning, and it’s easy to hit limits around memory, package size, or execution time. With Fargate, the entire environment is packaged in the Docker image, providing predictable runtime behavior and flexible CPU/memory allocation, which makes this kind of workload much easier to operate.

In the end, this is a simple batch worker: it runs on a schedule, performs one job, and exits. For headless browser automation, this approach proved to be more straightforward and reliable.