[SOLVED] ulimit different between ubuntu terminal and call from R

Issue

I am trying to create mapbox tiles using mapboxapi::tippecanoe() in R. Unfortunately, my work computer runs Windows 10, which greatly complicates what I am trying to do. Tippecanoe is a Unix executable, so I downloaded and installed Ubuntu and am running it on a Windows subsystem for Linux. To get tippecanoe to launch, I had to edit the source code of mapboxapi::tippecanoe() to pass arguments to WSL. I then ran into an issue where Tippecanoe would give me an error that it could not open database files. Some research on Github led me to believe that this was related the number of open files limit in Ubuntu. After a lot of digging, I was able to increase ulimit -n to 65535 for on my ubuntu terminal. As soon as I launch Ubuntu, if I type in ulimit -n, I get 65535. However, when I call `sytem2("wsl", "ulimit -n"), I get the default value of 1024. I thought this was due to the user that R was calling in Ubuntu, but running system2("wsl", "whoami") returned the username for who I increased both the hard and soft nofile limits for. I am really stumped. Apologies for not pasting a reproducible example, but I am not sure how to make one for this situation. Any help would be much appreciated. Thanks!

Solution

Well after a whole lot of tinkering, this ended up being mostly a straightforward R code issue. The ulimit issue may still have been a problem, but actually I need to fix the R code in mapboxapi::tippecanoe(). Because mapboxapi::tippecanoe()uses the system() command to call tippecanoe, not only did I need to change the call to invoke wsl through a login shell using system2("wsl", "- d Ubuntu -lc 'tippecanoe <arguments to tippecanoe>'"), but I also needed to change the paths that R sent to tippecanoe to be linux paths instead of Window paths. If anyone else is having trouble with this, here is the tweaked mapboxapi::tippecanoe() command code that actually worked for me:

tippecanoe2<-function (input, output, layer_name, min_zoom = NULL, 
                       max_zoom = NULL, drop_rate = NULL, overwrite = TRUE, other_options = NULL, 
                       keep_geojson = FALSE) 
{
  
  check_install <- system2("wsl", "tippecanoe -v") == 0
  linux_dir<-paste(getwd(), layer_name, sep="/")#make a directory in your linux directory for the .mbtiles
  parsed<-strsplit(linux_dir, split="/") #parse the windows directory path
  n<-length(parsed[[1]])
  dir_out<-paste("",parsed[[1]][n-1], parsed[[1]][n], sep="/") #construct the linux directory path
  dir.create(linux_dir)
  op<-options(useFancyQuotes = FALSE)
  if (!check_install) {
    rlang::abort(c("tippecanoe is not installed or cannot be found by the application you are using to run mapboxapi.", 
                   "If you haven't installed tippecanoe, please visit https://github.com/mapbox/tippecanoe for installation instructions.", 
                   "If you have installed tippecanoe, run `Sys.getenv('PATH')` and make sure your application can find tippecanoe. If it cannot, adjust your PATH accordingly."))
  }
  opts <- c()
  if (!is.null(min_zoom)) {
    opts <- c(opts, sprintf("-Z%s", min_zoom))
  }
  if (!is.null(max_zoom)) {
    opts <- c(opts, sprintf("-z%s", max_zoom))
  }
  if (is.null(min_zoom) && is.null(max_zoom)) {
    opts <- c(opts, "-zg")
  }
  if (!is.null(drop_rate)) {
    opts <- c(opts, sprintf("-r%s", drop_rate))
  }
  else {
    opts <- c(opts, "-as")
  }
  if (overwrite) {
    opts <- c(opts, "-f")
  }
  collapsed_opts <- paste0(opts, collapse = " ")
  if (!is.null(other_options)) {
    extra_opts <- paste0(other_options, collapse = " ")
    collapsed_opts <- paste(collapsed_opts, extra_opts)
  }
  dir <- linux_dir
  if (any(grepl("^sf", class(input)))) {
    input <- sf::st_transform(input, 4326)
    if (is.null(layer_name)) {
      layer_name <- stringi::stri_rand_strings(1, 6)
    }
    if (keep_geojson) {
      outfile <- paste0(layer_name, ".geojson")
      path <- file.path(dir_out, outfile)
      sf::st_write(input, path, quiet = TRUE, delete_dsn = TRUE, 
                   delete_layer = TRUE)
    }
    else {
      tmp <- tempdir("//wsl$/Ubuntu/tmp")#Here you would need to tweak to the file path for your linux distribution's temporary directory
      tempfile <- paste0(layer_name, ".geojson")
      path <- file.path(tmp, tempfile)
      sf::st_write(input, path, quiet = TRUE, delete_dsn = TRUE, 
                   delete_layer = TRUE)
    }
    call <- sprintf("tippecanoe -o %s/%s %s %s", dir_out, output, 
                    collapsed_opts, path)
    call2<-paste("-d Ubuntu /bin/bash -lc", sQuote(call, op), sep=" ")
      system2("wsl", call2)
  }
  else if (inherits(input, "character")) {
    if (!is.null(layer_name)) {
      collapsed_opts <- paste0(collapsed_opts, " -l ", 
                               layer_name)
    }
    call <- sprintf("tippecanoe -o %s/%s %s %s", dir_out, output, 
                    collapsed_opts, input)
    call2<-paste("-d Ubuntu /bin/bash -lc", sQuote(call, op), sep=" ")
    system2("wsl", call2)
  }
}

Answered By - Sean McKenzie

Answer Checked By - Senaida (WPSolving Volunteer)