[SOLVED] update to httr 1.4.7 and curl 5.0.2 package is causing authentication issue

Issue

I have a script that uses httr package for GET and POST. The script used to work without any issues for years (since 2019). but recently after updating httr package (with curl 5.0.2) to 1.4.7, the GET is resulting in code 401. POST is working fine.

I believe this is due to something in R specifically httr 1.4.7, I tried the same url, headers, and credentials with python requests package and it works there. I also tried command line curl and it worked, then I tried system(curl) command in R, which is same as command line curl and it still works.

It just does not work with the code below. I will have to post this code to Posit Connect server where its being used by several applications to fetch data.

Can someone please confirm the issue is with updates to curl/httr? And how this can be fixed?

r = POST(
    url = paste(apiUrl, "auth/login", sep = ""),
    use_proxy("http://proxy-xyz.pqr.com", 9090, "pxy-uid", "pxy-pwd"),
    add_headers(Accept = "application/vnd.seeq.v1+json",
                `Content-Type` = "application/vnd.seeq.v1+json"),
    body = paste('{"authProviderClass":"Auth",
                   "authProviderId":"Vendor",
                   "username":"uid",
                   "password":"pwd"}',sep = "")
                 )
                              
token <<- r[["headers"]][["x-sq-auth"]]

r = GET(url = "https://myurl",
        use_proxy("https://proxy-xyz.pqur.com", 9090, "pxy-uid", "pxy-pwd"),
        query = list(start = start, end = end, period = period),
        add_headers(Accept = "application/vnd.seeq.v1+json",
                    `x-sq-auth` = token
                    #`x-sq-csrf` = token
                    )
        )
fromJSON(content(r, as="text", encoding="UTF-8"))[["samples"]]

Solution

I do not know a whole lot about curl other than I can use it for GET and POST from API's. I found out that GET worked for me after I do httr::reset_config and restarted the R session.

I realized that What actually was happening, that when I ran GET without running the POST (manually adding the token, obtained by running POST previously), it worked but it failed when I ran POST first and then GET.

I found that when I run POST, it sets some cookies! and the cookies have two different authentication tokens x-sq-auth and x-sq-csrf.

So If I use only x-sq-auth token after running POST, I am getting an 401 authentication error.

After running POST I have to use x-sq-csrf token in my GET request. This solved my issue but I am not sure why it works this way.

I am still not sure if this is due to the updates in httr/curl package or something changed with the api.

Answered By - ok1more

Answer Checked By - Marilyn (WPSolving Volunteer)