Issue
I am trying to retrieve files from a remote server to my local PC using a cron job. However my script has to wait until the files are available on the remote server. From code pieces I gathered here and there, I came up with the code below
#!/bin/bash
year=$(date +%Y)
month=$(date +%m)
day=$(date +%d)
hour="00"
ssh-keygen # I suspect this line and the one below should be done once and not in the script.
ssh-copy-id _lms_2023
cd ${HOME}/ModelOutput_LMS/WRF_OUTPUT/tmp
cd ${HOME}/ModelOutput_LMS/WRF_OUTPUT/tmp
goto GOTO_1
if ssh [email protected] "test -e /${HOME}/DA/OUTPUT/"$year$month$day$hour"/noda/graphics/*.png"; then
scp [email protected]:${HOME}/DA/OUTPUT/"$year$month$day$hour"/noda/graphics/*.png .
if [ $? -eq 0 ]; then
exit
fi
else
sleep 30
GOTO_1
fi
I want the script to keep checking until the files are available and downloaded. The above script gives the errors below. Any assistance will be appreciated.
/usr/bin/ssh-copy-id: ERROR: ssh: Could not resolve hostname xxxxxxxxxx: Name or service not known
./cplocalfromRemote2.sh: line 14: goto: command not found
host's password:
Solution
A simpler way to do this is by using rsync instead of scp.
rsync checks/compares for new files automatically and copies only those files which are new/changed/modified or are not present in your destination location.
You can use rsync instead of scp in the script and set the script in cron to run every minute so that it gets the files which are newly added to the remote server.
- The script:
#!/bin/bash
year=$(date +%Y)
month=$(date +%m)
day=$(date +%d)
hour="00"
cd ${HOME}/ModelOutput_LMS/WRF_OUTPUT/tmp
sshpass -p'your_password' rsync -avh --progress [email protected]:${HOME}/DA/OUTPUT/"$year$month$day$hour"/noda/graphics/*.png .
- Set up a cronjob that runs the script every minute.
* * * * * /path/to/above/script.sh
EDIT: I have changed the script to add host password for rsync in non-interactive mode.
Hope this helps!
Answered By - mrrobot.viewsource Answer Checked By - Marilyn (WPSolving Volunteer)