P. Bai

8 min read

A team standard for Terraform state, with no DynamoDB lock table

This was not a side project. The team needed one agreed way to set up remote state for AWS work, so that the next project would not have to decide anything: copy the pattern, point it at an account, move on. What follows is the reasoning behind the three decisions that took the longest to settle, and one mistake that changed how we name files.

The result is published as terraform-aws-backend-bootstrap. The repository is a generic version of what we run. Account IDs, project names and regions in it are placeholders.

AWS ACCOUNT - DEV bootstrap/ local state on disk applied once, per account creates S3 bucket versioned, SSE-KMS, ACLs disabled public access blocked, prevent_destroy KMS key customer managed, rotating named per project, env and account id state + lock file root module your actual infrastructure -backend-config=dev.s3.tfbackend AWS ACCOUNT - PROD bootstrap/ local state on disk applied once, per account creates S3 bucket same hardening, its own bucket different name, different account id KMS key its own key, not shared no credential reaches across the line state + lock file root module the same code, applied elsewhere -backend-config=prod.s3.tfbackend The isolation boundary is the account, not a state key. That is why both sides can use the same fixed key = "terraform.tfstate", and why locking happens natively in S3 with no DynamoDB table.
Both columns are deliberately identical. Production is not a variant of development here, it is the same structure standing in its own account.

Dropping the DynamoDB lock table#

Our existing projects lock state with a DynamoDB table, because for years the S3 backend had no other option. State lives in S3, a separate table holds the lock, and the two have to be created together, permissioned together, and kept in sync forever.

Terraform 1.10 added native locking in S3. One setting replaces the table:

terraform {
  backend "s3" {
    key          = "terraform.tfstate"
    use_lockfile = true
    encrypt      = true
  }
}

The new standard does not create a lock table at all. Existing projects still have theirs, and we have not gone back to migrate them, so this is a fresh start rather than a migration story.

What convinced me it was safe was looking at how the lock actually fails. Terraform writes a terraform.tfstate.tflock object next to the state, and it writes it conditionally: if the object already exists, the write is rejected. Placing a lock file by hand and then running an apply produces this:

Error: Error acquiring the state lock

Error message: operation error S3: PutObject, https response error
StatusCode: 412, api error PreconditionFailed: At least one of the
pre-conditions you specified did not hold
Lock Info:
  ID:        3f2b1c44-0000-4000-8000-aaaaaaaaaaaa
  Operation: OperationTypeApply
  Who:       another-engineer
  Version:   1.15.8
  Created:   2026-08-03 13:00:00 +0000 UTC

The request ID and host ID lines are removed from that output; nothing else is edited. The lock holder shown comes from the lock file placed by hand, so the name is arbitrary, not evidence of a real colleague.

Status 412 is a conditional write refusing to overwrite an existing object, which is the same mutual exclusion a DynamoDB conditional put was providing, minus a second service. Terraform also reads the lock file back and tells you who holds it. After a normal apply finishes, the object is gone, so a crashed run leaves a stale lock you can see and delete rather than a row in a table you have to remember exists.

All of the above was checked against LocalStack 4.9.2 with Terraform 1.15.8, not against production. It exercises the same backend code path, but it is a simulator, and a LocalStack result is not proof of anything subtle about S3 consistency.

The trade is version pressure. use_lockfile needs Terraform 1.10 or newer everywhere, including CI. If some machine in your pipeline is pinned older, keep the table.

Separate accounts, not workspaces#

Company accounts were already split between development and production, so the environment boundary existed before Terraform did. Workspaces would have added a second boundary on top of the first one, and a weaker one: workspaces share a backend configuration and a set of credentials, so the thing standing between a dev apply and production is remembering to run terraform workspace select.

With separate accounts, the same protection comes from credentials that simply cannot reach the other environment. Nothing to remember.

That choice has a visible consequence in the repository, and it looks like a mistake until you know why:

key = "terraform.tfstate"

The same fixed key in both environments. That is only safe because each environment has its own bucket in its own account. If you isolate environments inside one account and one bucket, you need a distinct key per environment, and copying this file would put two environments on top of each other.

Each environment gets its own backend file instead:

bucket     = "example-app-dev-terraform-state-111111111111"
kms_key_id = "alias/example-app/dev/tf-state-key"
region     = "eu-central-1"

The bucket name carries project, environment and account ID. S3 bucket names are globally unique, so embedding the account ID removes a whole class of naming collision, and it makes the wrong bucket obvious when you read a config.

Because backend.tf holds only the settings that are the same everywhere, the rest arrives at init time:

terraform init --backend-config=dev.s3.tfbackend

That is a partial backend configuration. The environment is chosen by naming a file on the command line, and it is chosen once per working directory rather than per command, so there is no per-apply decision to get wrong.

The plan file we mixed up#

This is the part that came from getting it wrong.

The team writes plans to a file and applies the file, rather than applying from a fresh plan, so that the thing reviewed is the thing applied. The obvious name for that file is tfplan. With two environments, both plans want that same name, and we mixed them up.

Nothing was harmed. It was caught before anything landed. But the failure mode is unpleasant to think about, because a plan file is opaque: nothing in terraform apply tfplan tells you which environment produced it, and the command looks identical either way. Account isolation does not help here at all. The plan is a local file, and local files are one namespace no matter how many accounts you have.

So plans are named after their environment now:

terraform plan -var-file=dev.tfvars -out=dev.tfplan
terraform apply dev.tfplan

*.tfplan is in .gitignore. Plans contain resolved variable values and should not be committed.

The broader lesson I took from it: we had spent the whole design worrying about the boundary between accounts and none of it worrying about the boundary between two files on a laptop, and the laptop is where the mistake happened.

Why there is a bootstrap directory#

The S3 backend has to point at a bucket that already exists, and this project wants to keep its own state in that bucket. You cannot use a bucket as the backend for the run that creates it.

bootstrap/ is a separate root module with no remote backend, holding plain local state, whose only job is to create the state bucket and its KMS key. It runs once per account. After that, every other configuration points at what it made.

Local state is correct in exactly this one place, and the directory exists so that nobody has to think about it twice. The alternative, a one-off aws s3 mb and aws kms create-key in someone's shell history, produces a bucket whose configuration nobody can review and nobody can reproduce.

A customer managed KMS key rather than the default AES256 costs a little more and buys key rotation, a key policy you control, and CloudTrail records of key use. State files hold resource IDs and sometimes secrets in plain text, so that seemed worth paying for.

prevent_destroy on the bucket is not decoration. A plan generated with -refresh=false decided the bucket needed replacing, and the lifecycle rule stopped it:

Error: Instance cannot be destroyed

Resource aws_s3_bucket.terraform_state has lifecycle.prevent_destroy set,
but the plan calls for this resource to be destroyed.

That bucket holds every version of state for the environment. Making it hard to delete by accident is worth the small annoyance of having to disable the rule deliberately when you do mean it.

One thing that will bite you locally#

If you try to run this against LocalStack, the bootstrap module will create the bucket and then fail on the read that follows:

Error: listing tags for S3 Bucket: operation error S3 Control:
ListTagsForResource, StatusCode: 501
No moto route for service s3control found

The AWS provider version 6 reads S3 bucket tags through the S3 Control API. LocalStack community, including 4.9.2, does not implement that route, and it fails whether or not you have set any tags, because the read happens regardless. The bucket is created correctly; the resource just cannot be read back. Nothing is wrong with your configuration, and no provider setting turns that read off.

Backend operations are unaffected, since the backend is Terraform core rather than the AWS provider. State reads, writes and locking all work locally. Provisioning the bucket does not.

Where this does not apply#

If you run everything in one account, the fixed state key is wrong for you and you need a key per environment. If any machine that runs Terraform is older than 1.10, keep the DynamoDB table. If your team has no account separation to begin with, workspaces may genuinely be the smaller change, and the argument above does not transfer.

What I would keep in any of those cases is smaller than the backend design: name plan files after their environment. That one is free.