Another E2E Solution delivered. This time with CI/CD, AWS EventBridge and ECS Fargate
Source: Dev.to
I recently completed a personal project focused on automating a password rotation process for a third‑party system.
The integration requires authentication, but the system enforces a monthly password rotation. When the password expires, uploads and downloads start failing, quickly turning into an operational issue.
To eliminate manual updates and the risk of forgetting, I built an end‑to‑end automation.
The solution is a Python worker that uses Selenium with headless Chromium, runs on a schedule, and is backed by a full CI/CD pipeline. On every push to the main branch, GitHub Actions assumes an AWS IAM role via OIDC, builds the Docker image, pushes it to Amazon ECR, and registers a new ECS task definition revision that updates only the container image.
Architecture design solution
CI/CD
Execution (EventBridge)
Execution is handled by Amazon EventBridge, which triggers the task every 29 days.
ECS Cluster
The task runs on ECS Fargate in a public subnet, with a public IP and outbound traffic allowed.
When triggered, Fargate starts the container, runs automation.py, launches Selenium with Chromium and Chromedriver, logs into the system, performs the password rotation, and exits. On success the task finishes with exit code 0. If an exception occurs, logs are sent to CloudWatch and the error is reported to a Slack alerts channel.
Architecture decisions
-
Public subnet – Chosen for simplicity and cost. The worker only needs outbound internet access and does not expose any inbound ports, so there is no additional risk as long as the security group has no inbound rules. This also avoids the cost and complexity of a NAT Gateway required for private subnets.
-
ECS Fargate vs. Lambda – Running Selenium with Chromium on Lambda typically requires custom layers and fine‑tuning, and it’s easy to hit limits around memory, package size, or execution time. With Fargate, the entire environment is packaged in the Docker image, providing predictable runtime behavior and flexible CPU/memory allocation, making this workload easier to operate.
In the end, this is a simple batch worker: it runs on a schedule, performs one job, and exits. For headless browser automation, this approach proved more straightforward and reliable.