[SOLVED] Accessing python nltk with php fails

Issue

Im trying to call a python file containing a sentence/word tokenizer from my php file like this:

$output = shell_exec('python tokenizer.py $sentence')

I've tried single exec, full paths to python and tokenizer.py, wrapping $sentence in double quotes. But logically, It should not be the problem because calling print(1) at the beginning of python the python code before actually using any nltk packages makes $output equal to '1'. So I came to conclusion that the problem here is the nltk itself, like the path to the modules is not correct or something...

But, calling python from the shell using the same command as above gives me fully tokenized output! To conclude: looks like when calling python from php magically 'turns off' nltk, while it fully works when executed from the shell.

Here's the part of the python code I am using:

import sys
import nltk
from nltk.tokenize import sent_tokenize

sample_text2 = sys.argv[1]
gust = sent_tokenize(sample_text2)
#print(1) here doesn't work, but everywhere above (before calling sent_tokenize) it does.

The server's running on CentOS (Linux), I am accessing it via SSH.

Obvious question: What am I doing wrong here with PHP? Or generally? Any alternatives?

EDIT As visible in dvhh's answer and its comments, the situation happened because there were two versions installed on the server (2.6 and 2.7), while the www user had access to 2.6 and through console, the default version was 2.7. The solution was to change the default python to 2.7 for both cases and to put the nltk modules to one of the dependency folders. (Or append the dependency directory using sys.path.append)

Solution

Your php script is executed by the www user.

• You could check if the python script interpreter is correctly called, it is usually in one of the directory in the PATH environment variable (like /usr/bin/python), but the www user don't have a PATH environment variable set.
  Solution specify the whole path to your python interpreter in your shell_exec call ( also specify the full path to your script when you're at it )

• What about the path the nltk library is installed, you could check if the python interpreter would correctly look for it by looking at the sys.path while running python with the www user.
  Diagnostic : use the shell_exec call to run a python script to print the sys.path values Solution : append the library path to the sys.path in your python script before the import nltk

These would be the most obvious solutions considering the information provided in the question.

Update :
As there is 2 version version of python installed (on that haven't got the library installed ), it is recommended to specify the path to the desired interpreter. The first solution help correct the issue. In unix like system I would recommend using which python command to determine the path of your default python interpreter.

Answered By - dvhh