DevOps / Infrastructure-as-Code / AWS
"Merge is the apply" as an explicit governance model
Ran the Terraform infrastructure repository through GitHub Actions, with no separate approval step between merge and production. To compensate, every flag-flip pull request stayed unmerged until a deliberate go decision, tracked per PR in a shared status ledger. (Medical Claim SaaS Platform, the infrastructure repo's merge policy)
Found and fixed a silent CI false-negative
Root-caused a bug in a Terraform pipeline running as a GitHub Actions workflow. An error-suppression flag combined with a shell-pipeline exit-code loss made failed plans report success and skip applies. Fixed it by rebuilding the workflow as a single GitHub Actions job that reads change counts directly from Terraform plan output. (Medical Claim SaaS Platform, the infrastructure CI pipeline's apply workflow)
Measured a live security change at the API rather than trusting the merge timestamp
Used AWS CloudTrail logs on the specific security group, AWS Application Load Balancer (ALB) request logs, and AWS WAF logs to prove exactly when the change was open and closed, and what traffic arrived, during a perimeter test. This replaced relying on pull-request history to report the duration. (Payor Rate Data Platform, the platform's network-perimeter verification)
Found and documented a silent Terraform footgun
A Terraform `replace()` function call, used to template per-payer schedules, returns its input unchanged when the search string doesn't match. Adding or reordering a value silently turns six isolated, payer-scoped jobs into six duplicate full-fleet jobs, with no signal in the Terraform plan output. This got tracked as a named, numbered finding, with a standing do-not-touch rule during freeze windows. (Payor Rate Data Platform, the data-capture scheduling infrastructure's landmines log)
Built a deploy-verification primitive independent of "green CI"
Added a single header on one unauthenticated health endpoint that carries the exact deployed source commit. Built this because "merged" and "deployed" had drifted apart multiple times across AWS ECS release waves. (Payor Rate Data Platform, the platform's health-check endpoint)
Sequenced a live capacity-provider migration around a discovered ordering bug
Migrated a GPU-backed compute service to CPU-based serverless containers (AWS ECS Fargate), split across two separate Terraform applies. This followed a discovery that removing an AWS ECS capacity provider and its cluster association in one apply times out, so the fix was to disassociate first. (Medical Claim SaaS Platform, the GPU-worker service's compute-provider cutover)
Closed a gap between apply and deploy that had orphaned revisions for weeks
Diagnosed that a Terraform lifecycle ignore-changes setting let new AWS ECS task-definition revisions get registered without the running service ever adopting them. Removed the setting fleet-wide, so a Terraform apply and a deploy became the same guaranteed event. (Medical Claim SaaS Platform, the infrastructure's task-definition ownership model)
CD hardened against two independent failure classes
Added a boot-smoke run of every new Docker image before swapping a live AWS ECS service, so a bad build gets caught before it serves traffic. Also added an AWS ECR immutable-tag probe before rebuilding, which fixes a rerun failure mode. (Medical Claim SaaS Platform, the platform's CD and migration safeguards)
Cost-engineering tradeoffs made and documented as explicit levers
Ran a single-NAT-gateway AWS VPC network topology and a scale-to-zero AWS ECS Fargate posture as deliberate pre-revenue cost cuts. Later, produced a rigorously verified scale-to-zero design for the most compute-heavy service. This meant soliciting multiple independent design drafts, fact-checking every load-bearing AWS-behavior claim against primary AWS documentation, and synthesizing a final approach. (Medical Claim SaaS Platform, the network topology and worker scale-to-zero design)
Plan-time preconditions to prevent crash-looping services
Added Terraform preconditions that fail a plan, instead of a live boot, when a feature flag would be enabled without its required configuration. Also chose non-empty default string values, specifically to avoid an empty-string enum crash at boot. (Medical Claim SaaS Platform, the infrastructure's feature-flag plan preconditions)
Removed a security-sensitive CI artifact
Stopped publishing the binary Terraform plan file as a GitHub Actions CI artifact, after recognizing it embeds refreshed state, including secrets, in plaintext. Replaced it with a masked text summary. (Medical Claim SaaS Platform, the infrastructure CI pipeline's plan-artifact handling)
"Merge is the apply" CI/CD discipline enforced consistently
Ran the same merge-is-the-apply convention across the platform's Terraform infrastructure, applied via GitHub Actions. The one documented exception is the marketing stack, which deliberately uses plan-on-pull-request plus manual apply. This way, operators never assume one convention applies everywhere. (Payor Rate Data Platform, the infrastructure repo's CI convention and its documented exception)
Designed for cost via architecture, not tuning
Used a scale-to-zero-by-default posture, with the AWS Application Load Balancer (ALB) as the one standing exception. Also used public-subnet AWS ECS Fargate tasks with no NAT gateway, a deliberate cost and security tradeoff that got documented rather than left implicit. (Payor Rate Data Platform, the platform's network and compute topology)
Declared and observed formal change freezes around first-ever scheduled batch windows
Coordinated freezes spanning multiple independent AWS ECS services (capture, extraction, verification). For each freeze, explicitly reasoned about which changes were in scope and which were out. (Payor Rate Data Platform, the platform's operational change-freeze log)
Ran a repeatable, four-times-executed credential-elevation runbook
Used a standing procedure on a PostgreSQL database with no administrative access path. The procedure was to temporarily re-point credentials, run the Alembic-managed migration, then revert. Each of the four executions was timed, verified, and reverted. Separately flagged the one component never brought into this pattern as an open critical finding, rather than assuming it was covered. (Payor Rate Data Platform, the database migration privilege-elevation procedure)