Structure Environments Across AWS Accounts

The typical recommendation for teams is to deploy each of their environments to a separate AWS account. We find that this ends up being excessive for most teams. In our experience, what seems to work for many teams is:

  • One account for the Production environment. You want to apply very strict IAM access permissions to this account.
  • One account for EACH Staging environment. If you have multiple staging environment ie. preprod, qa, uat, etc, use a separate AWS account for each. You don't want to have multiple Staging environments share the same account because each Staging environment needs to mirror Production as closely as possible.
  • One account for ALL Development environments. All feature and hotfix environments share the same AWS account. You will have many Development environments, many will be very short-lived. Creating a temporary AWS account for each environment and tearing it down after the change is merged into master is far too excessive. Especially when you need to push a quick hotfix. Also, as we mentioned in the [How to organize your services]({% link _chapters/organizing-serverless-projects.md %}) chapter, you have one repo for the infrastructure and one repo for the code. Each Development environment most likely does not need their own version of the infrastructure. Consider when the scenario where you push a hotfix. You can have multiple API environments all talk to the one Infrastructure environment. And having them all sit inside the same AWS account makes it easy for them to talk to each other without configuring cross-account IAM permissions. Of course, this is not a hard rule. If you have a major feature release, the release can have its own Infrastructure environment and the entire setup can be deployed to a separate AWS account.
  • One account for EACH Developer environment. Each developer on your team has their own playground account.

At first glance, this might just seem like a whole lot of extra work and you might wonder if the added complexity is worth while. However, for teams this really helps protect your production environment. And we’ll go over the various ways it does so.

Environment separation

Imagine that you (or somebody on your team) removes a DynamoDB table or Lambda function in your serverless.yml definition. Now, instead of deploying it to your dev environment, you accidentally deploy it to production. This happens more often than you think!

To avoid mishaps like these, not every developer on your team should have write access from their terminal to the production environment. However, if all your environments are in the same AWS account, you need to carefully craft a detailed IAM policy to restrict/grant access to specific resources. This can be hard to do and you are likely to make mistakes.

By keeping each environment in a separate account, you can manage user access on a per account basis. And for dev environments you could potentially grant your developers AdministratorAccess without worrying about the specific resources.

Resource limits

Another benefit of separating environments across AWS accounts is helping you work with AWS’ service limits. You might be familiar with the various hard and soft limits of the AWS services you use. Like the Lambda’s 75 GB code storage limit or the total number of S3 buckets per account limit. These limits are applicable on a per account basis. Therefore, an issue with having a single AWS account for all your environments is that, hitting these limits can affect your production environment. Hence, affecting your users!

For example, you are likely to deploy to your dev environment an order of magnitude more often than to your production environment. Meaning that you’ll hit the Lambda code storage limit on your dev environment quicker than on production. If you only have one account for all your environments, hitting this limit would critically affect your production builds to fail.

By using multiple AWS accounts, you can be sure that the service limits will not interfere across environments.

Consolidated billing

Finally, having separate accounts for your environments is recommended by AWS. And AWS has great support for it as well. In the AWS Organizations console, you can view and manage all the AWS accounts in your master account.

Accounts in AWS Organization console

You don’t have to setup the billing details for each account. Billing is consolidated to the master account. You can also view a breakdown of usage and cost for each service in each account.

In the next chapter, we are going to look at how to setup these AWS accounts using AWS Organizations.