[SOLVED] Python C API - ModuleNotFoundError

Issue

I am using the Python C API, and I keep encountering the error ModuleNotFoundError: No module named '/home/user_1/project/ml/multiply. This is my c++ file, its path is /home/user_1/project/ml/testcapi.cpp, adapted from the Python C API documentation:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

int
main(int argc, char *argv[])
{
    PyObject *pName, *pModule, *pFunc;
    PyObject *pArgs, *pValue;
    int i;

    if (argc < 3) {
        fprintf(stderr,"Usage: call pythonfile funcname [args]\n");
        return 1;
    }

    Py_Initialize();
    pName = PyUnicode_FromString("/home/user_1/project/ml/multiply.py");
    /* Error checking of pName left out */

    pModule = PyImport_Import(pName);
    Py_DECREF(pName);

    if (pModule != NULL) {
        pFunc = PyObject_GetAttrString(pModule, argv[2]);
        /* pFunc is a new reference */

        if (pFunc && PyCallable_Check(pFunc)) {
            pArgs = PyTuple_New(argc - 3);
            for (i = 0; i < argc - 3; ++i) {
                pValue = PyLong_FromLong(atoi(argv[i + 3]));
                if (!pValue) {
                    Py_DECREF(pArgs);
                    Py_DECREF(pModule);
                    fprintf(stderr, "Cannot convert argument\n");
                    return 1;
                }
                /* pValue reference stolen here: */
                PyTuple_SetItem(pArgs, i, pValue);
            }
            pValue = PyObject_CallObject(pFunc, pArgs);
            Py_DECREF(pArgs);
            if (pValue != NULL) {
                printf("Result of call: %ld\n", PyLong_AsLong(pValue));
                Py_DECREF(pValue);
            }
            else {
                Py_DECREF(pFunc);
                Py_DECREF(pModule);
                PyErr_Print();
                fprintf(stderr,"Call failed\n");
                return 1;
            }
        }
        else {
            if (PyErr_Occurred())
                PyErr_Print();
            fprintf(stderr, "Cannot find function \"%s\"\n", argv[2]);
        }
        Py_XDECREF(pFunc);
        Py_DECREF(pModule);
    }
    else {
        PyErr_Print();
        fprintf(stderr, "Failed to load \"%s\"\n", argv[1]);
        return 1;
    }
    if (Py_FinalizeEx() < 0) {
        return 120;
    }
    return 0;
}

And here is my python script, taken straight from the C API documentation:

def multiply(a,b):
    print("Will compute", a, "times", b)
    c = 0
    for i in range(0, a):
        c = c + b
    return c

Now for some reason when I run the program, I get the no module found error. My compilation command is g++ testcapi.cpp -I /usr/include/python3.10 -L usr/lib/python3.10 -lpython -o a and I am running on a VM with the most recent stable version of Ubuntu. No compilation errors.

I am running the program with ./a multiply multiply 3 2 (taken straight from the Python C API documentation as well).

I am new to python (usually C/C++/Java) and I am not understanding why it cannot find the script. I first tried loading it using pName = PyUnicode_DecodeFSDefault(argv[1]); instead of pName = PyUnicode_FromString("/home/user_1/project/ml/multiply.py"); but that did not work either, same error.

Solution

I used

python -c "import sys; print(sys.path)"

which revealed a list of paths that python uses for searching for modules. I ended my moving my script into usr/lib/python3.10 and it was able to find the module and run the program as expected.

If anyone knows how to set the path from within the C++ code using the Python C API, that would be even more ideal of a solution. I tried using Py_SetProgramName and Py_SetPath to no avail.

Answered By - catfriendly

Answer Checked By - Clifford M. (WPSolving Volunteer)