Common Issues in S3 Cross-Account Access

Let’s explore the potential issues that can arise when setting up cross-account access to S3. Cross-account access involves a situation where a user or role in one AWS account (account1) seeks to access an S3 bucket located in another AWS account (account2). It might sound straightforward, but it’s a common scenario that can present challenges.

Account-1 has IAM user “user” with AmazonS3FullAccess

Account-2 has s3 bucket “my-s3-bucket-971” with 3 objects — file1, file2 and file3 along with the following resource based policy (bucket policy).

{
    "Version": "2012–10–17",
    "Statement": [
        {
            "Sid": "s3CrossAccountAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::000000000000:user/user"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::my-s3-bucket-971"
            ]
        }
    ]
}

There are two distinct methods as listed below for granting cross-account access. Please refer to this blog post for insights on when to utilise each approach. In this discussion, we will focus on resource-based policies.

• IAM roles
• Resource based policy

Using the CLI credentials of the IAM user in Account-1, first try to list the objects in the bucket.

aws s3api list-objects --bucket my-s3-bucket-971

Contents:
- ETag: '"2205e48de5f93c784733ffcca841d2b5"'
  Key: file1
  LastModified: '2023-10-12T10:48:49+00:00'
  Size: 5
  StorageClass: STANDARD
- ETag: '"ae691aad6f0f547ddf6c9526691efcea"'
  Key: file2
  LastModified: '2023-10-12T10:52:19+00:00'
  Size: 5
  StorageClass: STANDARD
- ETag: '"2205e48de5f93c784733ffcca841d2b5"'
  Key: file3
  LastModified: '2023-10-12T11:09:27+00:00'
  Size: 5
  StorageClass: STANDARD
RequestCharged: null

Now Let’s try to get file1, file2 and file3 one by one to illustrate the common issues.

aws s3api get-object --bucket my-s3-bucket-971 --key file1 file1

An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Please note: we have given s3 full-access on both ends, i.e. in the bucket policy as well as to the IAM user, but what’s wrong? Why can’t I get the object despite full access?

For any object-level operation it’s necessary to mention bucket-name/* under resources in the bucket policy. Let’s fix this common mistake by updating the bucket policy.

"Resource": [
                "arn:aws:s3:::my-s3-bucket-971",
                "arn:aws:s3:::my-s3-bucket-971/*”
]

Now try to get the file again. It’s successful, awesome!

aws s3api get-object --bucket my-s3-bucket-971 --key file1 file1

AcceptRanges: bytes
ContentLength: 5
ContentType: binary/octet-stream
ETag: '"2205e48de5f93c784733ffcca841d2b5"'
LastModified: '2023-10-12T10:48:49+00:00'
Metadata: {}
ServerSideEncryption: AES256

Let’s try to get file2

aws s3api get-object --bucket my-s3-bucket-971 --key file2 file2

An error occurred (AccessDenied) when calling the GetObject operation:
User: arn:aws:iam::338973819204:user/user is not authorized to perform:
kms:Decrypt on the resource associated with this ciphertext because the resource does not exist in this Region
no resource-based policies allow access, or a resource-based policy explicitly denies access

Looking at the error, there seems to be something off with the encryption settings. Lets inspect the encryption settings of the two files — file1 and file2.

file1 uses SSE-S3 for encryption

Encryption settings for file1

file2 uses SSE-KMS for encryption

Encryption settings for file2

The encryption settings for file1 and file2 differ. If an object is encrypted with SSE-S3, no additional decrypt permission is required to get the object. While using SSE-S3, we can not edit the key policy as the keys are managed by AWS. However, if an object is encrypted using SSE-KMS, additional decrypt permission is needed in order to get the object and the permission can be given using the resource based policy (KMS key policy).

To address this, grant the IAM “user” in Account-1 decrypt permission, as they are currently unauthorised to perform “kms:Decrypt” on the resource.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "kms:Decrypt",
            "Resource": "*"
        }
    ]
}

This alone is not sufficient, we must also give decrypt permission using the key policy in Account-2.

{
            "Sid": "grant cross account access to decrypt",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::000000000000:user/user”
            },
            "Action": "kms:decrypt",
            "Resource": "*"
 }

Let’s try again after making the above changes, it’s successful now.

aws s3api get-object --bucket my-s3-bucket-971 --key file2 file2

AcceptRanges: bytes
BucketKeyEnabled: true
ContentLength: 5
ContentType: binary/octet-stream
ETag: '"ae691aad6f0f547ddf6c9526691efcea"'
LastModified: '2023-10-12T10:52:19+00:00'
Metadata: {}
SSEKMSKeyId: arn:aws:kms:us-east-1:111111111111:key/xx-8c76-6bd4e1ba48d8
ServerSideEncryption: aws:kms

Lets try to get the file3

aws s3api get-object --bucket my-s3-bucket-971 --key file3 file3
An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Let’s take a look at the owner of this object. As you can see below the owner of file3 is a different AWS account and hence the user gets an error when trying to get the object.

Object owners for file1, file2, file3

However, how did another AWS account end up as the object owner? When we examine the Access Control List (ACL) settings of the bucket, it’s evident that the individual who wrote the object is also designated as the object owner. Object ownership plays a crucial role in determining who has the authority to define access to objects.

This indicates that file3 was uploaded by a user from a separate AWS account, albeit with the necessary “putObject” permission granted through the bucket policy.

Bucket ACL settings

Lets fix this — it’s easy, just disable the bucket ACL, which is the recommended approach. It enforces the bucket owner as the object owner. Alternatively the object owner can also grant access permission for cross account access.

Settings to disable Bucket ACL

Now try to get the object

aws s3api get-object --bucket my-s3-bucket-971 --key file3 file3

AcceptRanges: bytes
ContentLength: 5
ContentType: binary/octet-stream
ETag: '"2205e48de5f93c784733ffcca841d2b5"'
LastModified: '2023-10-12T11:09:27+00:00'
Metadata: {}
ServerSideEncryption: AES256

Summary
In this post we talked about configuring s3 cross-account access and the following three factors which might go wrong along with the ways to fix them -

1. Bucket policy does not allow access to objects using bucket-name/*
2. Objects in the bucket are encrypted using SSE-KMS
3. Objects in the bucket are owned by a different AWS account

Thanks for reading! If there is something else which can cause issues in cross account s3 access, please put in comments.