Ever been in the situation where you have dozens (or even more) of AWS accounts and/or GCP projects, and are in need to run a security tool against your entire estate?

In this post, part of the “Continuous Visibility into Ephemeral Cloud Environments” series, I’ll try to summarize what cloud resources (like roles and users) are needed, and how to define them in a manner that safely allows to perform a security audit across a fleet of AWS accounts/GCP projects.

This post has been updated on to add a Terraform implementation of the AWS setup.
The code can be found at: https://github.com/marco-lancini/utils/blob/main/terraform/aws-security-reviewer.

AWS Setup

For AWS we are going to use the Hub and Spoke model, where users assume federated roles into a single AWS “identity account” and perform a second role assumption into sub-accounts. For a refresher into this model, you can have a look at the Identity federation with multiple AWS accounts post from Alex Smolen.

First of all, you should have a clear idea of how many AWS accounts you have created in your Organization. Make a list of all the accounts you own, and pick one to be the “Hub” (master) account, which will pull data from all the other accounts (“Spoke” accounts).

At a high level, if we want our security tool to be able to analyse all the AWS accounts within our organization, we will need the following:

  1. One role (let’s name it role-security-audit), with the built-in SecurityAudit policy attached to it, which is going to be created in every AWS account (Hub + all the Spoke ones).
  2. One role (role-security-assume), in the Hub account, able to assume the role-security-audit role on all the other Spoke accounts.
  3. One IAM user (user-security-audit), in the Hub account, able to assume the role-security-assume role.

Sounds convoluted? The following sections will go into details on how to set this up. At the end, though, our setup will look like the picture below.

Cross Account Auditing in AWS
Cross Account Auditing in AWS

1. Resource Definition

Once identified all the accounts we want to include, it’s time to focus on the definition of all the resources (like roles and users) we need. Someone might prefer Terraform, someone else CloudFormation, so here I’m going to try stay agnostic from every infrastructure-as-code tool.

An implementation in Terraform can be found on Github: AWS Security Reviewer.
A similar implementation in CloudFormation can be found on Gitlab: CloudFormation: AWS Security Reviewer.
Setup Role role-security-audit in every account

The first resource to define is the IAM role named role-security-audit: this is the end role holding the permissions needed to perform a security audit, hence why it needs to be created in every account we want to include in our analysis (including the Hub). Since the role requires the same set of policies in every account, you could create a Terraform/CloudFormation template and apply the resource definition in every account.

  • Attach the built-in AWS SecurityAudit IAM policy (arn:aws:iam::aws:policy/SecurityAudit) to the role.
  • Set up a trust relationship so that each account will allow the role-security-assume role (that we will create in the Hub account) to assume this role. In the snippet below remember to replace the placeholder HUB_ACCOUNT_ID with the ID of your Hub account.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::HUB_ACCOUNT_ID:role/role-security-assume"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
Setup Role role-security-assume in Hub account

Next, we need to create a role (named role-security-assume) in the Hub account: this is the role that is going to be able to assume the role-security-audit role we created in every other Spoke account.

This role requires a set of policies as well:

  • Attach a policy that will allow this role to assume the role-security-audit role in every other Spoke account (notice the * in the Resource field).
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Resource": "arn:aws:iam::*:role/role-security-audit",
      "Action": "sts:AssumeRole"
    }
  ]
}
  • [OPTIONAL] Attach another policy that will grant the permission to describe EC2 regions (required by many tools to identify available regions).
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ec2:DescribeRegions",
      "Resource": "*"
    }
  ]
}
  • [OPTIONAL] Attach another policy that will grant the permission to list the accounts within the AWS Organization.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "organizations:ListAccounts",
      "Resource": "*"
    }
  ]
}
  • Set up a trust relationship so that the user-security-audit user (that we will create next) will be able to assume this role (remember to replace HUB_ACCOUNT_ID with the ID of your Hub account).
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::HUB_ACCOUNT_ID:user/user-security-audit"
      },
      "Action": "sts:AssumeRole",
      "Condition": {}
    }
  ]
}
Setup User user-security-audit in Hub account

Finally, the last resource we need is an IAM user (named user-security-audit) to be defined in the Hub account. This user only requires one policy that will allow it to assume the role-security-assume role (again, remember to replace HUB_ACCOUNT_ID with the ID of your Hub account).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::HUB_ACCOUNT_ID:role/role-security-assume"
    }
  ]
}

2. Setup Tooling for Cross-Account Auditing

Now that we have setup properly our AWS accounts, we need to instruct our security tools on how to fetch the relevant credentials that will allow them, after a couple of AssumeRole jumps, to assume the role-security-audit in every account:

  1. The tool obtains short lived credentials for the user-security-audit IAM user.
  2. With these credentials, it first performs an AssumeRole call to assume the role-security-assume in the Hub account.
  3. Finally, the tool can go through all the accounts in scope, and for each of them perform another AssumeRole call to assume its role-security-audit role.
Setup ~/.aws/credentials

First, ne need to instruct our tool on ow to obtain short lived credentials for the user-security-audit user. This can be done by modifying the ~/.aws/credentials file used by the tool, and by persisting the temporary credentials of the user under a [default] profile.

[default]
aws_access_key_id=$AWS_ACCESS_KEY_ID
aws_secret_access_key=$AWS_SECRET_ACCESS_KEY
aws_session_token=$AWS_SESSION_TOKEN

Note that this could be automated by dynamically fetching these secrets, for example, from Vault.

Setup ~/.aws/config

Next, we can use the ~/.aws/config file to instruct our tool on how to correctly access the Spoke accounts. The file will first hold the config for the default profile (tied to the security-audit-user credentials):

[default]
region=eu-west-1
output=json

Then, you should add a profile for each AWS Spoke account, so that it looks like the example below:

[profile <SpokeAccount1>]
role_arn = arn:aws:iam::<SpokeAccount1_ID>:role/security_audit
output=json
source_profile=default

Here, the last line (source_profile=default) is what actually defines the profile that contains the credentials that should be used for the initial AssumeRole call.

GCP Setup

Setup for GCP is actually more straightforward compared to AWS: in order for our tools to be able to access GCP assets, they need one Service Account (security-audit) with the securityReviewer role attached to it.

It is important to note that, in order for the tools to be access all GCP Projects within an Organization, the Service Account assigned to them needs to be created at the Organization level. This is because IAM access control policies applied on the Organization resource apply throughout the hierarchy on all resources in the organization.

Cross Account Auditing in GCP
Cross Account Auditing in GCP

1. Resource Definition

As briefly mentioned above, we need to create a Service Account (which we can name security-audit) at the Organization level, and attach the following built-in roles to it:

  • roles/iam.securityReviewer
  • roles/resourcemanager.organizationViewer (needed to list GCP Organizations)
  • roles/resourcemanager.folderViewer (needed to list GCP Folders)

2. Setup Tooling for Cross-Account Auditing

Now that we have setup properly our GCP accounts, we need to instruct our security tools on how to fetch the relevant credentials.

This can easily be done by setting the GOOGLE_APPLICATION_CREDENTIALS environment variable to point to the file containing the credentials for the Service Account (see Getting Started with Authentication for more information).