Last Updated on November 7, 2022
Creating an S3 bucket is easy enough, but to apply the principle of least privilege properly we need to understand how to create the right permissions for specific IAM identities. This might be straightforward if it weren’t for the multiple ways to configure permissions in S3, each having its own rules and edge cases.
This article helps you navigate this minefield, with details not only of how the S3 permissions work, but also how you can implement some common real-world scenarios such as S3 bucket access from another AWS account.
Understanding resource vs. identity policies
Before we talk about the different ways to define S3 bucket permissions, let’s make sure we’ve got a base understanding of resource and identity policies.
In the AWS world IAM is the Identity and Access Management service, and it handles making sure only the right identities have access to specific resources:
- a resource is the thing that you want to access, in our case an S3 bucket
- an identity is the thing that wants to access the resource. This could be an IAM user, a group, or a role.
A real world example of this might be that an IAM user bob needs to get objects from an S3 bucket. Without the correct permissions setup, that access won’t be possible because by default access to all AWS resources is denied.
So how can we fix this? Two options:
- create a resource-based policy attached to a specific resource. It describes which identities can do what actions on that resource.
- create an identity-based policy attached to a specific identity. It describes which actions can be done on what resources by that identity.
IAM lingo
Remember that in IAM an action is a specific thing that can be done on a resource, like s3:GetObject. A policy is a document which describes what actions can be done on what resources by what identities. When a policy is attached to a resource or identity this defines the permissions for that entity.
Not all AWS resources support resource-based policies. Other than S3, some popular services you can use them with are SQS and KMS.
S3 bucket policies (resource based policies)
The S3 implementation of the resource based policy concept is known as the S3 bucket policy. A bucket policy is attached to an S3 bucket, and describes who can do what on that bucket or the objects within it.
Here’s an example of a policy which allows our friend bob to get any object from the mountain-pics bucket:
{
"Version": "2012-10-17",
"Id": "Policy1606557924566",
"Statement": [
{
"Sid": "Stmt1606557921184",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111111111111:user/bob"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mountain-pics/*"
}
]
}
The policy consists of a series of statements (just one in this case), each specifying an Effect of Allow or Deny. Each statement contains:
- the Principal, which defines the user this statement applies to. The principal can also be an IAM role or an AWS account. In this case we’re specifying the user bob who exists in the same AWS account as the bucket (account id 111111111111).
- the Action defines what call can be made by the principal, in this case getting an S3 object. For a bucket policy the action must be S3 related.
- the Resource defines what resource the action applies to. For a bucket policy the resource must relate to the bucket the policy is attached to. This means it can be the ARN of the bucket itself, the ARN of objects within the bucket, or a wildcard describing a set of bucket objects as in this example.
Bucket actions vs. object actions
Some actions relate to the S3 bucket itself and some to the objects within the bucket. For example, s3:ListBucket relates to the bucket and must be applied to a bucket resource such as arn:aws:s3:::mountain-pics. On the other hand s3:GetObject relates to objects within the bucket, and must be applied to the object resources such as arn:aws:s3:::mountain-pics/mountains1.jpg.
When specifying resources in policies we can also use the wildcard character, which with S3 can be used to reference all objects within the bucket, like this arn:aws:s3:::mountain-pics/*.
Given the above policy is attached to the mountain-pics bucket (and no other policies are applied), Bob can access the contents of the bucket like this using the AWS CLI.
$ aws s3 cp s3://mountain-pics/mountains1.jpg . --profile bob download: s3://mountain-pics/mountains1.jpg to .\mountains1.jpg
Awesome! So an S3 bucket policy attached to the bucket has provided Bob permissions to get objects from that bucket.
IAM policies for S3 access (identity based policies)
The second main way to define S3 permissions is within a policy attached to an identity, which could be an IAM user, group, or role. This means from the perspective of that IAM identity, you can define what S3 resources can be accessed.
For example, we could attach the following policy to the user bob to allow him to get objects from the mountain-pics bucket.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mountain-pics/*"
}
]
}
Similar to the bucket policy, this policy consists of a series of statements with an Allow or Deny Effect.
- the Action defines what call can be made. Here we’re providing access to get objects from S3. Remember, that unlike within a bucket policy an identity policy allows you to specify an action for any AWS resource.
- the Resource defines the resource the action applies to. This could be any S3 bucket, bucket object, or set of objects specified with a wildcard as in this example.
Now let’s attach the above policy as an inline policy to the bob user, making sure we have no other policies relating to this bucket (including removing the bucket policy from earlier). Bob can now access the S3 bucket objects with the following AWS CLI command.
$ aws s3 cp s3://mountain-pics/mountains2.jpg . --profile bob download: s3://mountain-pics/mountains2.jpg to .\mountains2.jpg
So a user can access S3 buckets when we have only an IAM policy attached to that user giving S3 permissions. The same effect can be achieved by attaching a policy to an IAM group or role, allowing us to control S3 access from the perspective of the identity.
Now you should have a good idea about how to provide S3 access through bucket policies and IAM identity policies, which can be summarised like this.
How do we know which approach to use? One way to address this is depending on whether it’s more important for you to manage access from the point of view of the bucket or from the point of view of the IAM identities.
Given your specific use case though, there may be more to consider, especially when you’re taking into account more complex scenarios such as bucket access from another AWS account.
S3 bucket permissions for access cross-account
Everything we’ve talked about so far has been related to accessing buckets from IAM identities in the same account. What if you’ve got an AWS Organization setup with multiple accounts though?
Multi-account setups are popular for creating strong separation between different AWS resources. One common use case is to have development and production accounts, or accounts for different business areas.
Imagine a scenario where we have account A and account B. An S3 bucket exists in account A, and a user exists in account B who needs access to the S3 bucket in the other account.
In the following worked examples, our accounts have these ids:
- account A = 111111111111
- account B = 222222222222
Apply a cross-account bucket policy
As mentioned earlier, the bucket policy principal is flexible in that it allows you not only to specify users in the same account as the bucket, but also other entire accounts, and users within other accounts.
Following on from our earlier example, let’s say we have the mountain-pics bucket in Account A and a user alice in account B who needs to get objects from that bucket. We need to apply the following bucket policy to the bucket.
{
"Version": "2012-10-17",
"Id": "Policy1606557924566",
"Statement": [
{
"Sid": "Stmt1606557921184",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::222222222222:root"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mountain-pics/*"
}
]
}
Everything is the same as the bucket policy we used for bob in the earlier example, except now we’re specifying the Principle as account B with account id 222222222222. The account id has root appended to it which means all users. We could optionally be more specific and limit access to alice only using the alice user ARN of:
arn:aws:iam::222222222222:user/alice
So now Alice should have access to the S3 bucket, right?
$ aws s3 cp s3://mountain-pics/mountains1.jpg . --profile alice fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
Sadly not. It turns out that to provide cross-account access, we have to apply an IAM identity policy to the alice user as well as a bucket policy. That identity policy needs to provide the relevant S3 permissions against the bucket in the other account.
This is an important point you need to bear in mind when setting up cross-account permissions for S3 access.
OK, so let’s apply to alice the same inline policy we used for bob earlier.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mountain-pics/*"
}
]
}
This policy will allow alice who exists in account B to access the S3 bucket in account A. Let’s give it a go!
$ aws s3 cp s3://mountain-pics/mountains1.jpg . --profile alice download: s3://mountain-pics/mountains1.jpg to .\mountains1.jpg
Cool! We’ve combined bucket policies and IAM identity policies attached to a user to give cross-account access to an S3 bucket. Remember, that for cross-account access like this a bucket policy or IAM identity policy alone isn’t enough to provide access.
Writing to S3 buckets cross-account
So far we’ve looked at the use case of getting objects from an S3 bucket. Nice and simple. What happens when we want to put an object into a bucket though?
Let’s say that Alice has been on an adventure and wants to upload some mountain photos she’s taken to the mountain-pics bucket for Bob to see. The action that allows her to do this is s3:PutObject and that needs to be allowed for the wildcarded resource arn:aws:s3:::mountain-pics/*.
Let’s first update the bucket policy.
{
"Version": "2012-10-17",
"Id": "Policy1606557924566",
"Statement": [
{
"Sid": "Stmt1606557921184",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::222222222222:root",
"arn:aws:iam::111111111111:user/bob"
]
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::mountain-pics/*"
}
]
}
The bucket policy now says that both bob in account A (account id 111111111111) and all users from account B (account id 222222222222) can get and put objects into the bucket. We also add s3:PutObjectAcl, which you don’t need to worry about now but will be explained later.
Remember how when accessing S3 cross-account we need to involve both bucket policies and IAM identity policies? We also need to update the inline policy attached to alice in Account B to include s3:PutObject.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::mountain-pics/*"
}
]
}
Again, I’ve added the s3:PutObjectAcl action which will be explained later.
Now Alice can upload her amazing new picture from the AWS CLI.
$ aws s3 cp mountains3.jpg s3://mountain-pics/mountains3.jpg --profile alice upload: .\mountains3.jpg to s3://mountain-pics/mountains3.jpg
Now all that’s left to do is for Bob to get the newly uploaded object. He can’t wait!
$ aws s3 cp s3://mountain-pics/mountains3.jpg . --profile bob fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
Bob’s been denied! So what’s happening here?
Let’s take a look at the S3 bucket object through the AWS Console, logged in as an administrator user of account A. We’ll go into the details of the mountains3.jpg object (uploaded by alice from account B) and compare it to mountains1.jpg (uploaded by bob from account A).
Object uploaded to bucket from different account
Object uploaded to bucket from same account
Turns out that our mountains3.jpg S3 object is owned by account B, even though the bucket itself is owned by account A. This means that users from account A, such as bob, cannot view the object. How can we fix this?
Introducing S3 Access Control Lists (ACLs)
It turns out there’s a 3rd way to control access to buckets that I haven’t mentioned yet, Access Control Lists. The reason it hasn’t been mentioned so far is that it’s a legacy feature, which AWS themselves recommend avoiding.
As a general rule, AWS recommends using S3 bucket policies or IAM policies for access control. S3 ACLs is a legacy access control mechanism that predates IAM.
From this AWS guide
At a very high level, S3 ACLs manage read and write access to buckets and objects. They can be attached to buckets and objects separately. In fact, when a bucket is created a bucket ACL is automatically generated for you giving the bucket owner (the AWS account) full control.
Now the problem we have is when Alice uploads the object to S3 the default ACL automatically created for that object doesn’t give full control to the owning account, where the bob user resides. This can be fixed by specifying how the ACL should be created when the AWS CLI command is called.
aws s3 cp mountains3.jpg s3://mountain-pics --acl bucket-owner-full-control --profile alice upload: .\mountains3.jpg to s3://mountain-pics/mountains3.jpg
The additional --acl bucket-owner-full-control flag means that even though this object is created by another account, the account which owns the bucket will be given full control of the object.
Now Bob can try to download the object again.
$ aws s3 cp s3://mountain-pics/mountains3.jpg . --profile bob download: s3://mountain-pics/mountains3.jpg to .\mountains3.jpg
This time we have a successful download because the mountains3.jpg object ACL gives full access to the account containing the bob user.
Cross-account bucket access with STS assume role
Another way to access a bucket in another account is to use the Simple Token Service (STS) to generate temporary credentials for you to assume a role in the other account. If that role has access to the S3 bucket, whether that be through an S3 bucket policy or an IAM identity policy, then you will be able to access the bucket.
Using this approach might make the configuration simpler in some situations. Imagine if a user in account B needed access to other S3 buckets or even other AWS resources in account A, then it might be cleaner to define all those permissions within the role.
Assume we have a role called bucket-access-role. It has the same policy defined in IAM policies for S3 access, giving read access to all objects in the mountain-pics bucket. If Alice in account B wants to download mountains1.jpg, two things are required:
- the bucket-access-role must have a trust relationship with account B, allowing users in account B to perform
sts:AssumeRole - Alice in account B must have the permission for the
sts:AssumeRoleaction, to assume the role in Account A
If this is setup, then Alice can run this assume role command:
aws sts assume-role --role-arn arn:aws:iam::111111111111:role/bucket-access-role --role-session-name bucket-access-session --profile alice
Which will return her temporary credentials she can use to access resources in account A, with all the permissions of the bucket-access-role.
{
"Credentials": {
"AccessKeyId": "ASIAULNPKVJ5UA4BLHG2",
"SecretAccessKey": "irSzhnR2ne4U0jLF3YXk8355mjYFN2pp/Je1jYpE",
"SessionToken": "FwoGZXIvYXdzEPr//////////wEaDAzb703aryOdXT5k/SK5AfeDsBFh8Zseys8Iwc4rILh4+fRUupg5jxpVHHGJqvmItitvZjRsxUaOv930NRQLGiBA6081ZXkyKNdyma96RYiuomw60UVNQjpytaXLhqcQS5mUug8h4sstVc5UyguSAmETvZh7FXgtmVc2mdqoZoiQJvcQr/BojoI6be7HGmj6OJz1hdAa2D2Tts/412Dwr9zk2asY/xOieLNTDYSrRXIw4zjKw3OkT5btEyv0NqBDTgoPhBj82cjiKO6i5IAGMi3jp5lij1rI24t+s9GiRQ5NLv37ixkcwmK0IqevalvTMRKpydh6d4N/t4r15PA=",
"Expiration": "2021-02-02T09:46:38Z"
},
"AssumedRoleUser": {
"AssumedRoleId": "AROAULNPKVJ57JKJ4TZ4K:bucket-access-session",
"Arn": "arn:aws:sts::111111111111:assumed-role/bucket-access-role/bucket-access-session"
}
}
If the returned temporary access key id, secret access key, and session token were configured in an AWS CLI profile alice-temp, she could then issue this command.
aws s3 cp s3://mountain-pics/mountains1.jpg . --profile alice-temp download: s3://mountain-pics/mountains1.jpg to ./mountains1.jpg
Way to go Alice! You can see that Alice has successfully downloaded an object from an S3 bucket in another account, using the STS assume role functionality.
Side-by-side comparison of S3 access methods
We’ve delved into the 2 most important mechanisms to control S3 access, bucket policies and IAM identity policies. We’ve also touched briefly on the 3rd, ACLs.
| Bucket policy | IAM identity policy | ACL | |
|---|---|---|---|
| Attached to | A bucket | An IAM user, group, or role | A bucket or bucket object |
| Specifies | Which identities can do what actions on which bucket resources | Which actions can be done on which bucket resources by this identity | Read/write permissions |
| Same-account access | Can use alone. No other policies required. | Can use alone. No other policies required. | Legacy, not recommended |
| Same-account recommendation | Use if you want to configure access from the bucket’s perspective | Use if you want to configure access from the identity’s perspective | Legacy, not recommended |
| Cross-account access | Must be used with an IAM identity policy | Must be used with a bucket policy | When uploading objects from another account, specify --acl bucket-owner-full-control |
Assume role functionality for cross-account bucket access can use both bucket policies and IAM identity policies, which is why it doesn’t feature in the comparison.
Final thoughts
Bucket policies and IAM identity policies provide different options for configuring bucket permissions within the same account, and also across different AWS accounts.
To learn more about S3 ACLs, the details of which are beyond the scope of this article, check out the AWS documentation Access Control List (ACL) Overview.
I hope you’ve found some information relevant to your use case in this article. If not, please post a comment or send me an email and I’ll try to help and perhaps cover your scenario in this article.
Nice clear article describing the multiple S3 access approaches that AWS documentation sometimes doesn’t explain very well. To kick it up a notch, you might want to publish an update to this post describing how to provision similar cross-account access when the bucket is also encrypted with a KMS customer managed key.
Hi Greg. Your feedback and suggestion are much appreciated. Thanks!
Thank you for this write-up. The AWS docs are a mess with regards to this topic and obfuscate more than educate. This helped me setup cross-account access for an external team and was really helpful when it cam to debugging put failures.
Amazingly informative. I love the way you laid out the concepts. Kudos!
Thanks Greg!
Thank you for this clear article!
How is alice in Account B able to access mountains1.jpg, which is owned by bob in Account A? Wouldn’t the default ACL prevent alice from reading it?
Hi Yosef. The access is granted via the bucket policy which specifies a Principal of all users in the account in which Alice resides. The default ACL doesn’t prevent this access.
This is the first article that was easy to follow, so many get bogged down in details or just have confusing examples or examples that don’t go deep enough. You covered all bases here sir… Bravo!
I shall be sending my apprentices to this site as a good basis for some AWS IAM theory
Great to hear. Thanks Lee!
This is an amazing article.
Hi again,
I’m dig diving the topic.
Regarding the use of
--acl bucket-owner-full-control:It’s interesting to note that using this option doesn’t tranfer the objet ownership to the AWS Account that own the Bucket, it only add additional ACLs for the Account that own the Bucket.
You can configure S3 Bucket Ownership at the Bucket level (S3 > Your Bucket > Permissions > Edit Bucket Ownership > Define ‘Bucket owner preferred’.
By changing this option, the object Ownership is transferred to the AWS Account where reside the Bucket BUT users from others AWS Account are still able to put objects without specifying
--acl bucket-owner-full-control.You can also enforce them to do it by defining this additionnal statement in the resource based policy configured on your bucket:
"Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } }You can refer to the AWS documentation for more details. https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html
Thank you for sharing your knowledge.
Really clear explanation, now I know that when setting up a cross account access for a user, a bucket policy is not enough ;).
You may also add the access to a bucket using a S3 Access Point.
Delegating to the access point the bucket permission is also interesting.
You’re welcome Ghé! Did you come across such a requirement yet?
Hi,
Yes, it have been implemented by our Cloud provider to provide me a write access to a
bucket that reside in a prod account. The bucket was also accessed by a RDS SQL Server instance to perform native restore operations (not via an access point this time).
From my point of view, using a role (switch role) is the most simple solution to not care about the file ownership but there is no magic answer as always it depend on the use case 🙂
HI Tom,
Great article with simple and clear explanation! Would be great if you could add similar article on accessing S3 from EC2 on the same or different account. The topic is pretty stuffy :/
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html
Thanks!
Hi Vitalie. Cool, glad it helped. And thanks for the suggestion!
Nice precise explanation…It would be be great if you can create AWS tutorials or cover other topics as well. Thanks.
Thank you! Here are a few more AWS articles to check out. Did you have any other topics in mind, Yogendra?
Great! thanks for the simple explanation. It would be good to have STS part as well.
Thanks a lot Amin. The STS part is on the to-do list.
FYI using STS to assume a role to access an S3 bucket in another account has now been included in this article. Thanks again for the suggestion.