Thursday, October 6, 2022

[SOLVED] JQ subtracts 13 from my integer when it adds it to a second location, but not the first

Issue

I have a script that uses Docker env vars to configure a json file on start. The function uses the same ENV var twice in the same JQ command, and in the first location the integer remains as set, and in the second line JQ subtracts 13 (WTF?) from the number. Can anyone explain this? And more importantly how do I prevent this from happening.

Note: In order for the config to be valid, the second instance must be an unquoted integer. Testing shows that quoting the integer results in the expected output, but invalid config.

Env Var

DISCORD_CHANNEL=448887356515418113

Bash Function:

function updateDiscordConfig {
  echo "Setting Discord configuration..."
  jq ".discord |= . + {\"token\":\"${DISCORD_TOKEN}\"} | .discord.channels.channels |= . + {\"${DISCORD_CHANNEL}\": {}} | .minecraft.dimensions.generic |= . + {\"discordChannel\": [${DISCORD_CHANNEL}]}" ${DISCORD_SRCCONFIG} | sponge ${DISCORD_DESTCONFIG}
}

Output:

// First output
"channels": {
  "448887356515418113": {}
}

// Second output
"discordChannel": [
    448887356515418100
],

Update: I've replaced the second instance update with a line using sed:

function updateDiscordConfig {
  echo "Setting Discord configuration..."
  jq ".discord |= . + {\"token\":\"${DISCORD_TOKEN}\"} | .discord.channels.channels |= . + {\"${DISCORD_CHANNEL}\": {}}" ${DISCORD_SRCCONFIG} | sponge ${DISCORD_DESTCONFIG}
  sed -i "/discordChannel/c\   \"discordChannel\" : [${DISCORD_CHANNEL}]," ${DISCORD_DESTCONFIG}
}

Solution

Unfortunately you've run into one of the major limitations of the C-based implementation of jq - numerical precision. This implementation maps JSON numbers to IEEE 754 64-bit numbers. This is discussed in the jq FAQ under "Caveats".

Workarounds

(1) Use gojq, the Go implementation of jq.

(2) If you don't want to work with the C implementation of jq, you could use the "BigInt" module for jq - https://gist.github.com/pkoppstein/d06a123f30c033195841

However, this works with string representations, so it may be more trouble than its worth for your use case.

If you just need addition, you could use this jq def:

# The args should be strings representing non-negative integers
# without a leading "+":
def add(num1;num2):
  if (num1|length) < (num2|length) then add(num2;num1)
  else  (num1 | explode | map(.-48) | reverse) as $a1
      | (num2 | explode | map(.-48) | reverse) as $a2
      | reduce range(0; num1|length) as $ix
          ($a2;  # result
           ( $a1[$ix] + .[$ix] ) as $r
           | if $r > 9 # carrying
             then
               .[$ix + 1] = ($r / 10 | floor) + 
                            (if $ix + 1 >= length then 0 else .[$ix + 1] end )
               | .[$ix] = $r - ( $r / 10 | floor ) * 10
             else
               .[$ix] = $r
             end )
      | reverse | map(.+48) | implode
  end ;

Usage

One way to use this would be to copy it to a file (say add.jq) in your jq library (e.g. ~/.jq/) and use include to include it, e.g. as illustrated by the following:

jq -n  'include "add"; 
        add("448887356515418113";"448887356515418113")'
"897774713030836226"


Answered By - peak
Answer Checked By - Katrina (WPSolving Volunteer)
at