[SOLVED] Yml file parsing with shell scripting ( awk, sed, grep , ... )

Issue

The out put of the script after initial parsing data is like this at this point

  - hostname: lfpm9001
        - id: 700
          addr: 100.241.50.118/28
        - id: 800
          addr: 10.241.50.161/28
  - hostname: lfpm9002
        - id: 355
          addr: 100.243.52.129/25
        - id: 228
          addr: 100.241.51.161/25
        - id: 190
          addr: 100.245.25.1/24       
  - hostname: lfpm9003
        - id: 400
          addr: 100.250.55.121/24
        - id: 600
          addr: 100.242.56.168/28
        - id: 185
          addr: 100.240.26.10/24

trying to convert this file to have like this in output :

lfpm9001     700      100.241.50.118   28

lfpm9001     800      10.241.50.161    28

lfpm9002     355      100.243.52.129   25

lfpm9002     288      100.241.51.161   25

lfpm9002     190      100.245.25.1     24   

lfpm9003     400      100.250.55.121   24

lfpm9003     600      100.242.56.168   28

lfpm9003     185      100.240.26.10    24

Tried this, and partially solved the issue but can't capture hostname as desired.

sed -E '/-/{N;s~[^0-9]*([0-9]+)\n[^0-9]*([0-9.]+)/([0-9]+)~\1,\2,\3~}'

Solution

awk approach

The input file has hostname-keyed records, each followed by any number of pairs of records containing associated data.

The following awk procedure has three action blocks, each with pattern conditions targeting particular lines.

The first block targets lines containing the string "hostname" in their 3rd field ($3). It captures the value associated with the hostname key in a variable named host. It also sets a line number variable named ln to the current record (line) number of the input file:

/hostname/{host=$3; ln=NR}

The second block targets the next line by referencing the line number ln, stored in the first block. It constructs a string stored in a variable named line with the stored host name, a tab delimeter, and field 3, containing the id value.

NR==ln+1 {line=host"\t"line"\t"$3}

The third block targets the next line containing the IP data. It splits the IP value held in field $2 at he slash, storing the parts in a two-element array named IP, using the parts to continue the output string before printing the line (and adding a new line to achieve the blank line in your required output).

Lastly, the third block resets the line variable to an empty string and advances the stored value of the line containing the hostname by two. This last step applies the second and third blocks to the next two lines providing they are not hostname-containing lines (in which case the line number will be reset within block 1 to restart the cycle).

NR==ln+2 {split($2,IP,"/");line=line"\t"IP[1]"\t"IP[2]; print line"\n"; line=""; ln=ln+2}

The entire awk procedure

awk '/hostname/{host=$3; ln=NR} NR==ln+1 {line=host"\t"line"\t"$3} NR==ln+2 {split($2,IP,"/");line=line"\t"IP[1]"\t"IP[2]; print line"\n"; line=""; ln=ln+2}' inputFile

Test

tested on gnu awk 5.1.0 API: 3 on Raspberry Pi 400.

output:

lfpm9001        700 100.241.50.118  28

lfpm9001        800 10.241.50.161   28

lfpm9002        355 100.243.52.129  25

lfpm9002        228 100.241.51.161  25

lfpm9002        190 100.245.25.1    24

lfpm9003        400 100.250.55.121  24

lfpm9003        600 100.242.56.168  28

lfpm9003        185 100.240.26.10   24

Answered By - Dave Pritlove

Answer Checked By - Katrina (WPSolving Volunteer)