S3 bucket access from the same and another AWS account

Creating an S3 bucket is easy enough, but to apply the principle of least privilege properly we need to understand how to create the right permissions for specific IAM identities. This might be straightforward if it weren’t for the multiple ways to configure permissions in S3, each having its own rules and edge cases.

This article helps you navigate this minefield, with details not only of how the S3 permissions work, but also how you can implement some common real-world scenarios such as S3 bucket access from another AWS account.

Understanding resource vs. identity policies

Before we talk about the different ways to define S3 bucket permissions, let’s make sure we’ve got a base understanding of resource and identity policies.

In the AWS world IAM is the Identity and Access Management service, and it handles making sure only the right identities have access to specific resources:

• a resource is the thing that you want to access, in our case an S3 bucket

• an identity is the thing that wants to access the resource. This could be an IAM user, a group, or a role.

A real world example of this might be that an IAM user bob needs to get objects from an S3 bucket. Without the correct permissions setup, that access won’t be possible because by default access to all AWS resources is denied.

So how can we fix this? Two options:

• create a resource-based policy attached to a specific resource. It describes which identities can do what actions on that resource.

• create an identity-based policy attached to a specific identity. It describes which actions can be done on what resources by that identity.

IAM lingo

Remember that in IAM an action is a specific thing that can be done on a resource, like s3:GetObject. A policy is a document which describes what actions can be done on what resources by what identities. When a policy is attached to a resource or identity this defines the permissions for that entity.

Not all AWS resources support resource-based policies. Other than S3, some popular services you can use them with are SQS and KMS.

S3 bucket policies (resource based policies)

The S3 implementation of the resource based policy concept is known as the S3 bucket policy. A bucket policy is attached to an S3 bucket, and describes who can do what on that bucket or the objects within it.

Here’s an example of a policy which allows our friend bob to get any object from the mountain-pics bucket:

{
    "Version": "2012-10-17",
    "Id": "Policy1606557924566",
    "Statement": [
        {
            "Sid": "Stmt1606557921184",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111111111111:user/bob"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mountain-pics/*"
        }
    ]
}

The policy consists of a series of statements (just one in this case), each specifying an Effect of Allow or Deny. Each statement contains:

• the Principal, which defines the user this statement applies to. The principal can also be an IAM role or an AWS account. In this case we’re specifying the user bob who exists in the same AWS account as the bucket (account id 111111111111).

• the Action defines what call can be made by the principal, in this case getting an S3 object. For a bucket policy the action must be S3 related.

• the Resource defines what resource the action applies to. For a bucket policy the resource must relate to the bucket the policy is attached to. This means it can be the ARN of the bucket itself, the ARN of objects within the bucket, or a wildcard describing a set of bucket objects as in this example.

Bucket actions vs. object actions

Some actions relate to the S3 bucket itself and some to the objects within the bucket. For example, s3:ListBucket relates to the bucket and must be applied to a bucket resource such as arn:aws:s3:::mountain-pics. On the other hand s3:GetObject relates to objects within the bucket, and must be applied to the object resources such as arn:aws:s3:::mountain-pics/mountains1.jpg.

When specifying resources in policies we can also use the wildcard character, which with S3 can be used to reference all objects within the bucket, like this arn:aws:s3:::mountain-pics/*.

Given the above policy is attached to the mountain-pics bucket (and no other policies are applied), Bob can access the contents of the bucket like this using the AWS CLI.

$ aws s3 cp s3://mountain-pics/mountains1.jpg . --profile bob
download: s3://mountain-pics/mountains1.jpg to .\mountains1.jpg

You can see why Bob wants to get his hands on this picture

Awesome! So an S3 bucket policy attached to the bucket has provided Bob permissions to get objects from that bucket.

IAM policies for S3 access (identity based policies)

The second main way to define S3 permissions is within a policy attached to an identity, which could be an IAM user, group, or role. This means from the perspective of that IAM identity, you can define what S3 resources can be accessed.

For example, we could attach the following policy to the user bob to allow him to get objects from the mountain-pics bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mountain-pics/*"
        }
    ]
}

Similar to the bucket policy, this policy consists of a series of statements with an Allow or Deny Effect.

• the Action defines what call can be made. Here we’re providing access to get objects from S3. Remember, that unlike within a bucket policy an identity policy allows you to specify an action for any AWS resource.

• the Resource defines the resource the action applies to. This could be any S3 bucket, bucket object, or set of objects specified with a wildcard as in this example.

Now let’s attach the above policy as an inline policy to the bob user, making sure we have no other policies relating to this bucket (including removing the bucket policy from earlier). Bob can now access the S3 bucket objects with the following AWS CLI command.

$ aws s3 cp s3://mountain-pics/mountains2.jpg . –profile bob download: s3://mountain-pics/mountains2.jpg to .\mountains2.jpg

Bob will have to remember his gloves for this one

So a user can access S3 buckets when we have only an IAM policy attached to that user giving S3 permissions. The same effect can be achieved by attaching a policy to an IAM group or role, allowing us to control S3 access from the perspective of the identity.

Now you should have a good idea about how to provide S3 access through bucket policies and IAM identity policies, which can be summarised like this.

How do we know which approach to use? One way to address this is depending on whether it’s more important for you to manage access from the point of view of the bucket or from the point of view of the IAM identities.

Given your specific use case though, there may be more to consider, especially when you’re taking into account more complex scenarios such as bucket access from another AWS account.

S3 bucket permissions for access cross-account

Everything we’ve talked about so far has been related to accessing buckets from IAM identities in the same account. What if you’ve got an AWS Organization setup with multiple accounts though?

Multi-account setups are popular for creating strong separation between different AWS resources. One common use case is to have development and production accounts, or accounts for different business areas.

Imagine a scenario where we have account A and account B. An S3 bucket exists in account A, and a user exists in account B who needs access to the S3 bucket in the other account.

In the following worked examples, our accounts have these ids:

• account A = 111111111111
• account B = 222222222222

Apply a cross-account bucket policy

As mentioned earlier, the bucket policy principal is flexible in that it allows you not only to specify users in the same account as the bucket, but also other entire accounts, and users within other accounts.

Following on from our earlier example, let’s say we have the mountain-pics bucket in Account A and a user alice in account B who needs to get objects from that bucket. We need to apply the following bucket policy to the bucket.

{
    "Version": "2012-10-17",
    "Id": "Policy1606557924566",
    "Statement": [
        {
            "Sid": "Stmt1606557921184",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::222222222222:root"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mountain-pics/*"
        }
    ]
}

Everything is the same as the bucket policy we used for bob in the earlier example, except now we’re specifying the Principle as account B with account id 222222222222. The account id has root appended to it which means all users. We could optionally be more specific and limit access to alice only using the alice user ARN of:

arn:aws:iam::222222222222:user/alice

So now Alice should have access to the S3 bucket, right?

$ aws s3 cp s3://mountain-pics/mountains1.jpg . --profile alice
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Sadly not. It turns out that to provide cross-account access, we have to apply an IAM identity policy to the alice user as well as a bucket policy. That identity policy needs to provide the relevant S3 permissions against the bucket in the other account.

This is an important point you need to bear in mind when setting up cross-account permissions for S3 access.

Combine bucket policies and identity policies for cross-account S3 bucket access

OK, so let’s apply to alice the same inline policy we used for bob earlier.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mountain-pics/*"
        }
    ]
}

This policy will allow alice who exists in account B to access the S3 bucket in account A. Let’s give it a go!

$ aws s3 cp s3://mountain-pics/mountains1.jpg . --profile alice
download: s3://mountain-pics/mountains1.jpg to .\mountains1.jpg

Cool! We’ve combined bucket policies and IAM identity policies attached to a user to give cross-account access to an S3 bucket. Remember, that for cross-account access like this a bucket policy or IAM identity policy alone isn’t enough to provide access.

Writing to S3 buckets cross-account

So far we’ve looked at the use case of getting objects from an S3 bucket. Nice and simple. What happens when we want to put an object into a bucket though?

Let’s say that Alice has been on an adventure and wants to upload some mountain photos she’s taken to the mountain-pics bucket for Bob to see. The action that allows her to do this is s3:PutObject and that needs to be allowed for the wildcarded resource arn:aws:s3:::mountain-pics/*.

Let’s first update the bucket policy.

{
    "Version": "2012-10-17",
    "Id": "Policy1606557924566",
    "Statement": [
        {
            "Sid": "Stmt1606557921184",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::222222222222:root",
                    "arn:aws:iam::111111111111:user/bob"
                ]
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::mountain-pics/*"
        }
    ]
}

The bucket policy now says that both bob in account A (account id 111111111111) and all users from account B (account id 222222222222) can get and put objects into the bucket. We also add s3:PutObjectAcl, which you don’t need to worry about now but will be explained later.

Remember how when accessing S3 cross-account we need to involve both bucket policies and IAM identity policies? We also need to update the inline policy attached to alice in Account B to include s3:PutObject.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::mountain-pics/*"
        }
    ]
}

Again, I’ve added the s3:PutObjectAcl action which will be explained later.

Now Alice can upload her amazing new picture from the AWS CLI.

$ aws s3 cp mountains3.jpg s3://mountain-pics/mountains3.jpg --profile alice
upload: .\mountains3.jpg to s3://mountain-pics/mountains3.jpg

Now all that’s left to do is for Bob to get the newly uploaded object. He can’t wait!

$ aws s3 cp s3://mountain-pics/mountains3.jpg . --profile bob
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Bob’s been denied! So what’s happening here?

Let’s take a look at the S3 bucket object through the AWS Console, logged in as an administrator user of account A. We’ll go into the details of the mountains3.jpg object (uploaded by alice from account B) and compare it to mountains1.jpg (uploaded by bob from account A).

Object uploaded to bucket from different account

Owned by the account that uploaded the bucket (account B 222222222222)

Object uploaded to bucket from same account

Owned by the bucket owner account (account A 111111111111)

Turns out that our mountains3.jpg S3 object is owned by account B, even though the bucket itself is owned by account A. This means that users from account A, such as bob, cannot view the object. How can we fix this?

Introducing S3 Access Control Lists (ACLs)

It turns out there’s a 3rd way to control access to buckets that I haven’t mentioned yet, Access Control Lists. The reason it hasn’t been mentioned so far is that it’s a legacy feature, which AWS themselves recommend avoiding.

As a general rule, AWS recommends using S3 bucket policies or IAM policies for access control. S3 ACLs is a legacy access control mechanism that predates IAM. 

From this AWS guide

At a very high level, S3 ACLs manage read and write access to buckets and objects. They can be attached to buckets and objects separately. In fact, when a bucket is created a bucket ACL is automatically generated for you giving the bucket owner (the AWS account) full control.

Now the problem we have is when Alice uploads the object to S3 the default ACL automatically created for that object doesn’t give full control to the owning account, where the bob user resides. This can be fixed by specifying how the ACL should be created when the AWS CLI command is called.

aws s3 cp mountains3.jpg s3://mountain-pics --acl bucket-owner-full-control --profile alice
upload: .\mountains3.jpg to s3://mountain-pics/mountains3.jpg

The additional --acl bucket-owner-full-control flag means that even though this object is created by another account, the account which owns the bucket will be given full control of the object.

Now Bob can try to download the object again.

$ aws s3 cp s3://mountain-pics/mountains3.jpg . --profile bob
download: s3://mountain-pics/mountains3.jpg to .\mountains3.jpg

This time we have a successful download because the mountains3.jpg object ACL gives full access to the account containing the bob user.

Bob thinks this image was worth the wait

Cross-account bucket access with STS assume role

Another way to access a bucket in another account is to use the Simple Token Service (STS) to generate temporary credentials for you to assume a role in the other account. If that role has access to the S3 bucket, whether that be through an S3 bucket policy or an IAM identity policy, then you will be able to access the bucket.

Using this approach might make the configuration simpler in some situations. Imagine if a user in account B needed access to other S3 buckets or even other AWS resources in account A, then it might be cleaner to define all those permissions within the role.

Assume we have a role called bucket-access-role. It has the same policy defined in IAM policies for S3 access, giving read access to all objects in the mountain-pics bucket. If Alice in account B wants to download mountains1.jpg, two things are required:

1. the bucket-access-role must have a trust relationship with account B, allowing users in account B to perform sts:AssumeRole

2. Alice in account B must have the permission for the sts:AssumeRole action, to assume the role in Account A

If this is setup, then Alice can run this assume role command:

aws sts assume-role --role-arn arn:aws:iam::111111111111:role/bucket-access-role --role-session-name bucket-access-session --profile alice

Which will return her temporary credentials she can use to access resources in account A, with all the permissions of the bucket-access-role.

{
    "Credentials": {
        "AccessKeyId": "ASIAULNPKVJ5UA4BLHG2",
        "SecretAccessKey": "irSzhnR2ne4U0jLF3YXk8355mjYFN2pp/Je1jYpE",
        "SessionToken": "FwoGZXIvYXdzEPr//////////wEaDAzb703aryOdXT5k/SK5AfeDsBFh8Zseys8Iwc4rILh4+fRUupg5jxpVHHGJqvmItitvZjRsxUaOv930NRQLGiBA6081ZXkyKNdyma96RYiuomw60UVNQjpytaXLhqcQS5mUug8h4sstVc5UyguSAmETvZh7FXgtmVc2mdqoZoiQJvcQr/BojoI6be7HGmj6OJz1hdAa2D2Tts/412Dwr9zk2asY/xOieLNTDYSrRXIw4zjKw3OkT5btEyv0NqBDTgoPhBj82cjiKO6i5IAGMi3jp5lij1rI24t+s9GiRQ5NLv37ixkcwmK0IqevalvTMRKpydh6d4N/t4r15PA=",
        "Expiration": "2021-02-02T09:46:38Z"
    },
    "AssumedRoleUser": {
        "AssumedRoleId": "AROAULNPKVJ57JKJ4TZ4K:bucket-access-session",
        "Arn": "arn:aws:sts::111111111111:assumed-role/bucket-access-role/bucket-access-session"
    }
}

If the returned temporary access key id, secret access key, and session token were configured in an AWS CLI profile alice-temp, she could then issue this command.

aws s3 cp s3://mountain-pics/mountains1.jpg . --profile alice-temp
download: s3://mountain-pics/mountains1.jpg to ./mountains1.jpg

Way to go Alice! You can see that Alice has successfully downloaded an object from an S3 bucket in another account, using the STS assume role functionality.

Side-by-side comparison of S3 access methods

We’ve delved into the 2 most important mechanisms to control S3 access, bucket policies and IAM identity policies. We’ve also touched briefly on the 3rd, ACLs.

Assume role functionality for cross-account bucket access can use both bucket policies and IAM identity policies, which is why it doesn’t feature in the comparison.

Final thoughts

Bucket policies and IAM identity policies provide different options for configuring bucket permissions within the same account, and also across different AWS accounts.

To learn more about S3 ACLs, the details of which are beyond the scope of this article, check out the AWS documentation Access Control List (ACL) Overview.

I hope you’ve found some information relevant to your use case in this article. If not, please post a comment or send me an email and I’ll try to help and perhaps cover your scenario in this article.