[SOLVED] How can I write the data received after the loop to a json file?

Issue

I want to make a script that will generate a list of certificates that will expire in a month and write it in json format to a file.

Making a loop and getting the list as regular strings is not a problem. I can’t yet understand how to format the resulting list in json form. So that it would be like this:

{
  "data": [
    {
      "{expiration_date}": "test1_expiration_date_value",
      "{serial_number}": "test1_expiration_date_value"
    },
    {
      "{expiration_date}": "test2_expiration_date_value",
      "{serial_number}": "test2_expiration_date_value"
    },
    {
      "{expiration_date}": "testN_expiration_date_value",
      "{serial_number}": "testN_expiration_date_value"
    }
  ]
}

I make a curl request to get a list of certificates curl --header "..." --request LIST https://some_uri | jq -r '.data.keys' | grep '\w' | tr -d '""',',' In response I receive:

[
  "test1",
  "test2",
  "testN"
]

Then you can request additional fields for each of the lists curl -s --header "..." --request GET https://some_uri/test1 | jq -r '.data.data' In response I receive:

{
  "expiration_date": "2023-10-11 11:27:10",
  "serial_number": "18:df:b4:a8:34:25:b1:43:c9:a3:6d:83:59:34:53:ab:2c:5e:c2:7a"
}

I tried different designs, but have not won yet. The last option, without any conditions for now:

#!/bin/bash

date=$(date -d "+1 month")

cert_list=$(curl -s --header "..." --request LIST https://some_uri | jq -r '.data.keys' | grep '\w' | tr -d '""',',')

list=()
for i in $cert_list
do
  cert_data=$(curl -s --header "..." --request GET https://some_uri/$i | jq -r '.data.data | "\(.expiration_date) \(.serial_number)"')
  list+="$cert_data"$'\n'
done

list_json="$(printf '%s\n' "${list}")"
json="$(jq -Rn '{data: [inputs]}' <<< "$list")"

printf '%s\n' "$json"

Solution

Assuming curl … https://some_uri | jq '.data.keys' outputs a JSON array of strings like ["test1", "test2", "testN"], and curl … https://some_uri/test1 | jq '.data.data' outputs a JSON object like {"expiration_date": "…", "serial_number": "…"}, then the following might solve your task.

It fetches the results from the first URI, and outputs a NUL-delimited list of strings that were contained in the array under .data.keys. Shell's while then iterates over that list using read -d '' to segment along the NUL characters, and in each step it fetches the results from the other URI (with the current list item appended), and extracts the object under .data.data, but leaves it JSON-encoded. A final call to jq then collects all of them in an array using the -s flag, and wraps it into an object with field name data.

#!/bin/bash

curl … "https://some_uri" | jq --raw-output0 '.data.keys[]' |
while read -r -d '' key; do
  curl … "https://some_uri/${key}" | jq '.data.data'
done | jq -s '{data: .}'

Answered By - pmf

Answer Checked By - Senaida (WPSolving Volunteer)