[SOLVED] Crontab not finding python modules

Issue

Hello i have been following stackoverflow posts and im unable to get the crontab to run my python scripts in ubuntu.

After talking with the user below, i have modified my file but am not getting a "moduleNotFoundError: No module named aioredis"

when i deactivate my py3.9 venv environment, and call python feed_ingestor_to_postgres.py i get the same error. so clearly the pipenv isn't activating for me. any suggestions?

my launcher.sh file:

#!/bin/bash
  
WORKDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
cd $WORKDIR

activate(){
        ../../env/bin/activate
}


python3 feed_ingestor_to_postgres.py

my crontab file:

* * * * * echo "Great cron, awesome job!" > /var/log/nonsense
* * * * * /home/ubuntu/phobos/phobos-trading-engine/exchange_feeds/launcher.sh

I have no idea what else to do here. any help would be appreciated

Solution

To manage python scripts from the crontab more easily, I would advise to use a simple shell wrapper script, for example "launcher.sh" which will be called by the crontab. The launcher will cd to the script location, manage the virtualenv, or possibly set env vars if required:

#!/bin/bash

WORKDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
cd $WORKDIR

source ../../env/bin/activate
python feed_ingestor_to_postgres.py

Then in the crontab, call the launcher instead of your python script:

* * * * *       ~/path/to/launcher.sh

It will be much more flexible than editing the crontab directly

Answered By - piwai

Answer Checked By - Katrina (WPSolving Volunteer)