[SOLVED] AWS S3: How to check if a file exists in a bucket using bash

Issue

I'd like to know if it's possible to check if there are certain files in a certain bucket.

This is what I've found:

href="https://stackoverflow.com/questions/17455782/checking-if-a-file-is-in-a-s3-bucket-using-the-s3cmd">Checking if a file is in a S3 bucket using the s3cmd

It should fix my problem, but for some reason it keeps returning that the file doesn't exist, while it does. This solution is also a little dated and doesn't use the doesObjectExist method.

Summary of all the methods that can be used in the Amazon S3 web service

This gives the syntax of how to use this method, but I can't seem to make it work.

Do they expect you to make a boolean variable to save the status of the method, or does the function directly give you an output / throw an error?

This is the code I'm currently using in my bash script:

existBool=doesObjectExist(${BucketName}, backup_${DomainName}_${CurrentDate}.zip)

if $existBool ; then
        echo 'No worries, the file exists.'
fi

I tested it using only the name of the file, instead of giving the full path. But since the error I'm getting is a syntax error, I'm probably just using it wrong.

Hopefully someone can help me out and tell me what I'm doing wrong.

!Edit

I ended up looking for another way to do this since using doesObjectExist isn't the fastest or easiest.

Solution

Last time I saw performance comparisons getObjectMetadata was the fastest way to check if an object exists. Using the AWS cli that would be the head-object method, example:

aws s3api head-object --bucket www.codeengine.com --key index.html

which returns:

{
    "AcceptRanges": "bytes",
    "ContentType": "text/html; charset=utf-8",
    "LastModified": "Sun, 08 Jan 2017 22:49:19 GMT",
    "ContentLength": 38106,
    "ContentEncoding": "gzip",
    "ETag": "\"bda80810592763dcaa8627d44c2bf8bb\"",
    "StorageClass": "REDUCED_REDUNDANCY",
    "CacheControl": "no-cache, no-store",
    "Metadata": {}
}

Answered By - Dave Maple

Answer Checked By - Marilyn (WPSolving Volunteer)