Wednesday, October 27, 2021

[SOLVED] Amazon s3 download in crontab not working?

Issue

I have a very simple script that I'm using to download a file from an Amazon s3 bucket and place it in a folder on my local machine:

aws s3 cp s3://sfbucket.bucket/sf_events.json /Users/Documents/data/sf_events.json

If I type this in command line, it works without issue. However, I want this script to run once a day automatically, so I'm trying to put it in crontab:

01 19 * * * aws s3 cp s3://sfbucket.bucket/sf_events.json /Users/Documents/data/sf_events.json

For some reason, this fails to run in crontab. Can anyone tell me why this wouldn't work? Thank you in advance!


Solution

first of all to figure out why it's not working, you can redirect the output of the command.

you can redirect the output:

01 19 * * * aws s3 aws s3 cp s3://sfbucket.bucket/sf_events.json /Users/Documents/data/sf_events.json >> /Users/Arun/Learning/help-project/cron-help/logs2.txt 2>&1

this solution will not solve your problem, but it will show you the problem.

Solution:

this worked for me

01 19 * * * /Library/Frameworks/Python.framework/Versions/3.7/bin/aws s3 cp s3://sfbucket.bucket/sf_events.json /Users/Documents/data/sf_events.json >> /Users/Arun/Learning/help-project/cron-help/logs2.txt 2>&1


Answered By - Arun K