[SOLVED] undefined symbols remain, but shared libraries compile and seem to function properly

Issue

My goal was to build shared libraries from SUNDIALS 2.7.0 - these are solvers of ordinary differential equations, written in C.

I've downloaded the source, and followed the installation guide:

1. run Cmake (with GUI), checked options "CVODE" (this is one of the available solvers - the only one I wanted), "build shared libraries" and "use generic (std-c) math libraries" and generated the Makefile.
2. run make && make install in the Linux console and my shared libraries got generated, with no error messages at all. Here is the output:

$ make
Scanning dependencies of target sundials_nvecserial_shared
[  3%] Building C object src/nvec_ser/CMakeFiles/sundials_nvecserial_shared.dir/nvector_serial.c.o
[  7%] Building C object src/nvec_ser/CMakeFiles/sundials_nvecserial_shared.dir/__/sundials/sundials_math.c.o
[ 11%] Linking C shared library libsundials_nvecserial.so
[ 11%] Built target sundials_nvecserial_shared
Scanning dependencies of target sundials_cvode_shared
[ 14%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode.c.o
[ 18%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_io.c.o
[ 22%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_direct.c.o
[ 25%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_band.c.o
[ 29%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_dense.c.o
[ 33%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_diag.c.o
[ 37%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_spils.c.o
[ 40%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_spbcgs.c.o
[ 44%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_spgmr.c.o
[ 48%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_sptfqmr.c.o
[ 51%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_sparse.c.o
[ 55%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_bandpre.c.o
[ 59%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/cvode_bbdpre.c.o
[ 62%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_nvector.c.o
[ 66%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_math.c.o
[ 70%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_direct.c.o
[ 74%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_band.c.o
[ 77%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_dense.c.o
[ 81%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_iterative.c.o
[ 85%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_sparse.c.o
[ 88%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_spbcgs.c.o
[ 92%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_spgmr.c.o
[ 96%] Building C object src/cvode/CMakeFiles/sundials_cvode_shared.dir/__/sundials/sundials_sptfqmr.c.o
[100%] Linking C shared library libsundials_cvode.so
[100%] Built target sundials_cvode_shared
$ sudo make install
[ 11%] Built target sundials_nvecserial_shared
[100%] Built target sundials_cvode_shared
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/include/sundials/sundials_config.h

Install shared components

-- Installing: /usr/local/include/sundials/sundials_band.h
-- Installing: /usr/local/include/sundials/sundials_dense.h
-- Installing: /usr/local/include/sundials/sundials_direct.h
-- Installing: /usr/local/include/sundials/sundials_iterative.h
-- Installing: /usr/local/include/sundials/sundials_math.h
-- Installing: /usr/local/include/sundials/sundials_nvector.h
-- Installing: /usr/local/include/sundials/sundials_fnvector.h
-- Installing: /usr/local/include/sundials/sundials_pcg.h
-- Installing: /usr/local/include/sundials/sundials_sparse.h
-- Installing: /usr/local/include/sundials/sundials_spbcgs.h
-- Installing: /usr/local/include/sundials/sundials_spfgmr.h
-- Installing: /usr/local/include/sundials/sundials_spgmr.h
-- Installing: /usr/local/include/sundials/sundials_sptfqmr.h
-- Installing: /usr/local/include/sundials/sundials_types.h

Install NVECTOR_SERIAL

-- Installing: /usr/local/lib/libsundials_nvecserial.so.2.7.0
-- Installing: /usr/local/lib/libsundials_nvecserial.so.2
-- Installing: /usr/local/lib/libsundials_nvecserial.so
-- Installing: /usr/local/include/nvector/nvector_serial.h

Install CVODE

-- Installing: /usr/local/lib/libsundials_cvode.so.2.9.0
-- Installing: /usr/local/lib/libsundials_cvode.so.2
-- Installing: /usr/local/lib/libsundials_cvode.so
-- Installing: /usr/local/include/cvode/cvode_band.h
-- Installing: /usr/local/include/cvode/cvode_bandpre.h
-- Installing: /usr/local/include/cvode/cvode_bbdpre.h
-- Installing: /usr/local/include/cvode/cvode_dense.h
-- Installing: /usr/local/include/cvode/cvode_diag.h
-- Installing: /usr/local/include/cvode/cvode_direct.h
-- Installing: /usr/local/include/cvode/cvode.h
-- Installing: /usr/local/include/cvode/cvode_sparse.h
-- Installing: /usr/local/include/cvode/cvode_spbcgs.h
-- Installing: /usr/local/include/cvode/cvode_spgmr.h
-- Installing: /usr/local/include/cvode/cvode_spils.h
-- Installing: /usr/local/include/cvode/cvode_sptfqmr.h
-- Installing: /usr/local/include/cvode/cvode_impl.h

These shared libraries seem to function properly in my Project, for which I needed them - ordinary differential equations are getting solved and the output seems to be identical to other solvers that are independent of these.

But there is one big but :): Checking for undefined symbols in the generated shared libraries show me the short list of standard functions - here are the outputs for the shared libraries:

$ nm -D --undefined-only libsundials_cvode.so.2.9.0
                 w __cxa_finalize
                 U exp
                 U fprintf
                 U fputc
                 U free
                 U fwrite
                 w __gmon_start__
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 U malloc
                 U memset
                 U pow
                 U printf
                 U putchar
                 U realloc
                 U sqrt
                 U __stack_chk_fail
                 U stderr
                 U vsprintf

and

$ nm -D --undefined-only libsundials_nvecserial.so.2.7.0
                 w __cxa_finalize
                 U exp
                 U free
                 w __gmon_start__
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 U malloc
                 U pow
                 U printf
                 U putchar
                 U sqrt

Printing shared library dependencies gives:

$ ldd -r libsundials_nvecserial.so
    linux-vdso.so.1 (0x00007ffd657a5000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fecd8155000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fecd874c000)
undefined symbol: exp   (./libsundials_nvecserial.so)
undefined symbol: pow   (./libsundials_nvecserial.so)
undefined symbol: sqrt  (./libsundials_nvecserial.so)

and

$ ldd -r libsundials_cvode.so        linux-vdso.so.1 (0x00007ffe6a1ff000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6cd55ac000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f6cd5bc8000)
undefined symbol: exp   (./libsundials_cvode.so)
undefined symbol: pow   (./libsundials_cvode.so)
undefined symbol: sqrt  (./libsundials_cvode.so)

There are, for instance, functions like SUNRsqrt that are defined over sqrt and are very important in actual numerical calculations - this fact as well as the presence of undefined symbols makes me confused and concerned about the reliability of the shared libraries.

Before generating the shared libraries, I've set all the appropriate paths to the environmental variable LD_LIBRARY_PATH (which, by the way, was empty before):

$ export LD_LIBRARY_PATH="/usr/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/gcc/x86_64-linux-gnu"
$ echo $LD_LIBRARY_PATH
/usr/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/gcc/x86_64-linux-gnu

For instance, the C standard library libc.so and the C math library libm.so are stored at /usr/lib/x86_64-linux-gnu, whereas the GNU Transactional Memory Library libitim.so is stored at /usr/lib/gcc/x86_64-linux-gnu (although the version 1.0.0 of it is also at /usr/lib/x86_64-linux-gnu).

Versions of OS, Cmake, GNU C compiler and GNU Make:

$ lsb_release -d
Description:    Ubuntu 18.04.5 LTS
$ cmake --version
cmake version 3.18.2
$ gcc --version
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
$ make --version
GNU Make 4.1

Any comment would be greatly appreciated.

P.S. This case is completely reproducible, since it is all about generating shared libraries from the SUNDIALS 2.7.0 source (direct download link) and checking them on undefined symbols.

Solution

You have undefined symbol with ldd because the so files were created without the -lm option. However it will not be a problem as long as the final executable is linked with -lm option. It is what is done for tests like test_nvector_serial which is linked with -lm -lrt.

You can see all this by running the make in a verbose mode with make VERBOSE=1. If you try to generate the executable without -lm you will have collect2: error: ld returned 1 exit status and the linker complaining about exp, pow and sqrt. if you add -lm when creating the so files ( I did it ). you will see :

ldd -r ./src/nvec_ser/libsundials_nvecserial.so
    linux-vdso.so.1 (0x00007ffe1a769000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe4a5c19000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe4a5828000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fe4a61bd000)

And in this case you can link your tests ( only to do some test for yourself if you want) without -lm.

To compile your so files with -lm. You can do it by running the command line with -lm option after entering the right folder (just for test purpose). The other way is by modifying the CMakeLists.txt which generate the so files. For libsundials_nvecserial.so for example, you modify sundials-2.7.0/src/nvec_ser/CMakeLists.txt by adding target_link_libraries(sundials_nvecserial_shared -lm) after the ADD_LIBRARY(sundials_nvecserial_shared SHARED ${nvecserial_SOURCES} ${shared_SOURCES}).

For the LD_LIBRARY_PATH, it does not have to contain something necessarily if you are looking in standard places. The linker use also /etc/ld.so.cache and search also in default path unless you use -z nodeflib. you can find more details in https://man7.org/linux/man-pages/man8/ld.so.8.html

For the nm -D. It is normal , try to compile a hello world gcc -o exec hello.c you will see that printf is undefined. Take a look at dynamic linking, and also to the -rdynamic option used to build the so files.

Answered By - yflelion