[SOLVED] How can I enable an ec2 instance to have private access to an S3 bucket?

Issue

First of all i'm aware of these questions:

• Grant EC2 instance access to S3 Bucket
• Can't access s3 bucket using IAM-role from an ec2-instance
• Getting Access Denied when calling the PutObject operation with bucket-level permission

but the solutions are not working for me.

I created a role "sample_role", attached the AmazonS3FullAccess-policy to it and assigned the role to the ec2-instance.

My bucket-policy is as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::My-Account-ID:role/sample_role"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::my_bucket/*"
        }
    ]
}

On my ec2-instance, listing my buckets works fine, both from the command line (aws s3 ls) and from python script. But when I try to upload a file test.txt to my bucket, I get AccessDenied:

import boto3

s3_client = boto3.client('s3')
s3_resource = boto3.resource('s3')
bucket = s3_resource.Bucket('my_bucket')

with open('test.txt', "rb") as f:
    s3_client.upload_fileobj(f, bucket.name, 'text.txt')

Error message:

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

Same happens when i just try to list the objects in my bucket. Command line aws s3api list-objects --my_bucket or python script:

import boto3

s3_resource = boto3.resource('s3')
bucket = s3_resource.Bucket('my_bucket')

for my_bucket_object in bucket.objects.all():
    print(my_bucket_object)

Error message:

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

When I turn off "Block all public access" in my bucket settings and enable public access in my access control list, it obviously works. But I need to restrict access to the specified role.

What am I missing? Thanks for your help!

Solution

So I found the problem. The credentails of my ec2 instance were configured with the access key of a dev-user account to which the role was not assigned.

I found out by running aws sts get-caller-identity which returns the identity (e.g. IAM role) actually being used.

So it seems that the assigned role can be overwritten by the user identity, which makes sense. To solve the problem, I simply undid the configuration by deleting the configuration file ~/.aws/credentials. After that the identity changed to the assigned role.

Answered By - David Salb