{"id":19,"date":"2026-01-10T02:24:50","date_gmt":"2026-01-10T02:24:50","guid":{"rendered":"https:\/\/platformsignals.dev\/?p=19"},"modified":"2026-01-10T02:27:37","modified_gmt":"2026-01-10T02:27:37","slug":"terraform-is-not-enough-why-infrastructure-as-code-drifts","status":"publish","type":"post","link":"https:\/\/platformsignals.dev\/?p=19","title":{"rendered":"Terraform is Not Enough: Why “Infrastructure as Code” Drifts"},"content":{"rendered":"\n

The lie we tell ourselves about IaC, and how to fix it with GitOps.<\/strong><\/p>\n\n\n\n

If I had a dollar for every time a team claimed to be “100% Infrastructure as Code” while simultaneously holding a root terminal open in another window, I could retire early.<\/p>\n\n\n\n

We tell ourselves a comforting lie in SRE: If it is in the Terraform file, it is in production.<\/em><\/p>\n\n\n\n

The reality is much messier. The reality is Configuration Drift<\/strong>. And if you aren’t actively detecting it, your Terraform state is nothing more than a historical document\u2014a snapshot of what you hoped<\/em> the infrastructure looked like three months ago.<\/p>\n\n\n\n

<\/p>\n\n\n\n

The “ClickOps” Reality<\/h3>\n\n\n\n

Here is the classic scenario:<\/p>\n\n\n\n

    \n
  1. You define an AWS Security Group in Terraform allowing port 443.<\/li>\n\n\n\n
  2. At 2:00 AM, the site goes down. The on-call engineer (maybe you) realizes the database needs to talk to a new microservice immediately.<\/li>\n\n\n\n
  3. Do they write a Pull Request, wait for CI, get approval, and apply? No. The site is down.<\/li>\n\n\n\n
  4. They log into the AWS Console, manually add the rule, fix the outage, and go back to sleep.<\/li>\n<\/ol>\n\n\n\n

    Result:<\/strong> Your infrastructure is now running a configuration that exists nowhere in your code. The next time someone runs terraform apply<\/code>, Terraform will either unknowingly revert that fix (causing another outage) or fail entirely because the state is desynchronized.<\/p>\n\n\n\n

    <\/p>\n\n\n\n

    Why Drift Happens<\/h3>\n\n\n\n

    It isn’t just panic-fixes. Drift happens because:<\/p>\n\n\n\n

      \n
    • External changes:<\/strong>\u00a0Cloud providers sometimes change default behaviors or resource IDs.<\/li>\n\n\n\n
    • Manual tinkering:<\/strong>\u00a0“I’ll just change this instance size to test something quick.” (It is never quick).<\/li>\n\n\n\n
    • Untracked resources:<\/strong>\u00a0S3 buckets created by scripts or other teams that Terraform doesn’t even know exist.<\/li>\n<\/ul>\n\n\n\n
      \"\"<\/figure>\n\n\n\n

      The Fix: From “IaC” to “GitOps”<\/h3>\n\n\n\n

      Writing HCL (HashiCorp Configuration Language) is the easy part. The hard part is the lifecycle of the apply<\/strong>. To be a true Platform Engineer, you need to move beyond running Terraform locally on your laptop.<\/p>\n\n\n\n

      Here is the maturity model for fixing drift:<\/p>\n\n\n\n

      Phase 1: The Ban (Policy)<\/h4>\n\n\n\n

      Remove write access to the cloud console for humans. This is extreme but effective. If you can’t click “Edit Security Group” in the AWS console because you literally don’t have the permission, you must<\/em> use Terraform.<\/p>\n\n\n\n

        \n
      • Pros:<\/em>\u00a0Stops drift cold.<\/li>\n\n\n\n
      • Cons:<\/em>\u00a0Makes emergency firefighting much harder. You need a “Break Glass” procedure for 2 AM incidents.<\/li>\n<\/ul>\n\n\n\n

        Phase 2: Automated Drift Detection (Observability)<\/h4>\n\n\n\n

        If you can’t lock the doors, at least install a security camera. Set up a scheduled job (Cron \/ Jenkins \/ GitHub Actions) that runs terraform plan<\/code> every hour.<\/p>\n\n\n\n

          \n
        • If the plan shows “No Changes,” send a \u2705 to Slack.<\/li>\n\n\n\n
        • If the plan shows “Changes Detected” (meaning reality doesn’t match code), send a \ud83d\udea8 alert to the SRE channel.<\/li>\n<\/ul>\n\n\n\n

          This turns Drift from a “deployment surprise” into a “monitoring alert.”<\/p>\n\n\n\n

          Phase 3: The GitOps Pipeline (Atlantis \/ Terraform Cloud)<\/h4>\n\n\n\n

          Stop running Terraform from your laptop. Use a tool like Atlantis<\/strong> or Terraform Cloud<\/strong> that ties the apply<\/code> to a GitHub Pull Request.<\/p>\n\n\n\n

            \n
          1. You open a PR.<\/li>\n\n\n\n
          2. Atlantis automatically runs\u00a0plan<\/code>\u00a0and comments the output on the PR.<\/li>\n\n\n\n
          3. Your colleague reviews the code and the plan.<\/li>\n\n\n\n
          4. You comment\u00a0atlantis apply<\/code>.<\/li>\n\n\n\n
          5. The bot applies the code and merges the PR.<\/li>\n<\/ol>\n\n\n\n

            This creates a perfect audit trail. We know who<\/em> changed the infra, what<\/em> the output was, and when<\/em> it happened.<\/p>\n\n\n\n

            <\/p>\n\n\n\n

            Conclusion<\/h3>\n\n\n\n

            Terraform code is static; infrastructure is dynamic. If you treat your .tf<\/code> files as “fire and forget,” you are building a house of cards. True reliability comes when you stop trusting the code blindly and start verifying the state continuously.<\/p>\n","protected":false},"excerpt":{"rendered":"

            The lie we tell ourselves about IaC, and how to fix it with GitOps. If I had a dollar for every time a team claimed to be “100% Infrastructure as Code” while simultaneously holding a root terminal open in another window, I could retire early. We tell ourselves a comforting lie in SRE: If it is […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[9,7,8,6,5,4],"class_list":["post-19","post","type-post","status-publish","format-standard","hentry","category-war-stories","tag-atlantis","tag-aws","tag-ci-cd","tag-gitops","tag-infrastructure-as-code","tag-terraform"],"_links":{"self":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts\/19","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=19"}],"version-history":[{"count":1,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts\/19\/revisions"}],"predecessor-version":[{"id":21,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts\/19\/revisions\/21"}],"wp:attachment":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=19"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=19"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=19"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}