Issue
For the last 5 days, I am trying to make Keras/Tensorflow packages work in R. I am using RStudio for installation and have used conda, miniconda, virtualenv but it crashes each time in the end. Installing a library should not be a nightmare especially when we are talking about R (one of the best statistical languages) and TensorFlow (one of the best deep learning libraries). Can someone share a reliable way to install Keras/Tensorflow on CentOS 7?
Following are the steps I am using to install tensorflow in RStudio.
Since RStudio simply crashes each time I run tensorflow::tf_config() I have no way to check what is going wrong.
devtools::install_github("rstudio/reticulate")
devtools::install_github("rstudio/keras") # This package also installs tensorflow
library(reticulate)
reticulate::install_miniconda()
reticulate::use_miniconda("r-reticulate")
library(tensorflow)
tensorflow::tf_config() **# Crashes at this point**
sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)
Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
loaded via a namespace (and not attached):
[1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
[5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
[9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
[17] base64enc_0.1-3
Update 1 The only way RStudio does not crash while installing tensorflow is by executing following steps -
First, I created a new virtual environment using conda
conda create --name py38 python=3.8.0
conda activate py38
conda install tensorflow=2.4
Then from within RStudio, I installed reticulate and activated the virtual environment which I earlier created using conda
devtools::install_github("rstudio/reticulate")
library(reticulate)
reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
reticulate::py_available(initialize = TRUE)
ts <- reticulate::import("tensorflow")
As soon as I try to import tensorflow in RStudio, it loads the library /lib64/libstdc++.so.6 instead of /root/.conda/envs/py38/lib/libstdc++.so.6 and I get the following error -
Error in py_module_import(module, convert = convert) :
ImportError: Traceback (most recent call last):
File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
from tensorflow.python._pywrap_tensorflow_internal import *
File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
module = _import(
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Here is what inside /lib64/libstdc++.so.6
> strings /lib64/libstdc++.so.6 | grep GLIBC
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBC_2.3
GLIBC_2.2.5
GLIBC_2.14
GLIBC_2.4
GLIBC_2.3.2
GLIBCXX_DEBUG_MESSAGE_LENGTH
To resolve the library issue, I added the path of the correct libstdc++.so.6 library having GLIBCXX_3.4.20 in RStudio.
system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
and, also
Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
But still I get the same error ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20'. Somehow RStudio still loads /lib64/libstdc++.so.6 first instead of /root/.conda/envs/py38/lib/libstdc++.so.6
Instead of RStudio, if I execute the above steps in the R console, then also I get the exact same error.
Update 2: A solution is posted here
Solution
Took me more than 15 days and I finally solved this problem.
Boot up a clean CentOS 7 VM, install R and dependencies (taken from Jared's answer) -
yum install epel-release
yum install R
yum install libxml2-devel
yum install openssl-devel
yum install libcurl-devel
yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
Now, create a conda environment
yum install conda
conda clean -a # Clean cache and remove old packages, if you already have conda installed
# Install all the packages together and let conda handle versioning. It is important to give a Python version while setting up the environment. Since Tensorflow supports python 3.9.0, I have used this version
conda create -y -n "tf" python=3.9.0 ipython tensorflow keras r-essentials r-reticulate r-tensorflow
conda activate tf
Open a new port (7878 or choose any port number you want) on the server to access RStudio with new conda environment libraries
iptables -A INPUT -p tcp --dport 7878 -j ACCEPT
/sbin/service iptables save
then launch RStudio as follows -
/usr/lib/rstudio-server/bin/rserver \
--server-daemonize=0 \
--www-port 7878 \
--rsession-which-r=$(which R) \
--rsession-ld-library-path=$CONDA_PREFIX/lib
You will have your earlier environment intact on default port 8787 and a new environment with Tensorflow and Keras on 7878.
The following code now works fine in RStudio
install.packages("reticulate")
install.packages("tensorflow")
library(reticulate)
library(tensorflow)
ts <- reticulate::import("tensorflow")
Answered By - Saurabh Answer Checked By - Senaida (WPSolving Volunteer)