[SOLVED] How to map values from one file to another file and write it to another file using shell script

Issue

I have 2 text files.

below text file is projectnameid.txt. first column is project name. second column in project id

basket-items                      9189
cloudfabric-notifications         10789
cloud-ports                       10965
common                            9081
customer-port-management          8550
deploy-quote                      8348
geographical-location-management  8549
internet-connections              9293
ipaddress                         8536
ip-addresses                      9294
order-notifications               11725
order-status                      8353
port-inventory                    8486
port-locations                    8490
pricing-quotes                    8493
product-availability              8488
product-catalogue                 8489
product-countrylist               8492
stub-service                      8510
customer-port-management-sf       10488
internet-connections-order-sf     11166
ip-addresses-order-sf             11165

below text file is endfilter3-all-b.txt

337718  10965  "refs/merge-requests/13/head"  "2023-07-19T11:39:41.739Z"
318933  8536   "develop"                      "2023-07-05T11:41:28.482Z"
366210  8549   "develop"                      "2023-08-11T13:49:18.905Z"
338835  8510   "main"                         "2023-07-20T06:45:59.823Z"
135208  8348   "main"                         "2023-02-17T11:25:07.723Z"
115402  8493   "main"                         "2023-02-07T06:52:05.486Z"
361979  9293   "refs/merge-requests/83/head"  "2023-08-09T07:38:32.831Z"
345703  11725  "main"                         "2023-07-26T08:31:11.004Z"
101775  8353   "main"                         "2023-02-02T09:22:47.402Z"
115414  8486   "main"                         "2023-02-07T07:41:35.478Z"
150861  9081   "main"                         "2023-03-13T05:37:31.370Z"
135733  8489   "main"                         "2023-02-17T16:14:51.280Z"

first column is pipeline id, second column is project id, third column is branch name, fourth column is timestamp

I have to read endfilter3-all-b.txt first then select each project id and match the project id from projectnameid.txt after pick the matching project id and project name write it to finalized-project-names.txt

expected output: (finalized-project-names.txt)

cloud-ports                 10965
ipaddress               8536
geographical-location-management    8549
stub-service                8510 
deploy-quote                8348   
pricing-quotes              8493   
internet-connections            9293   
order-notifications         11725  
order-status                8353   
port-inventory              8486   
common                  9081   
product-catalogue           8489    

this is what I tried:

#!/bin/bash

> finalized-project-names.txt

project_name_id_file="projectnameid.txt"
while read -r project_name project_id; do
   project_name_id_map[$project_id]=$project_name
done < "$project_name_id_file"

endfilter3_all_b_file="endfilter3-all-b.txt"
while read pipeline_id project_id branch_name timestamp; do
  project_name=${project_name_id_map[$project_id]}
  if [ -n "$project_name" ]; then
    echo "$project_name $project_id" >> finalized-project-names.txt
  fi
done < "$endfilter3_all_b_file"

output I got:

cloud-ports 10965
ipaddress 8536
geographical-location-management 8549
stub-service 8510
deploy-quote 8348
pricing-quotes 8493
internet-connections 9293
order-notifications 11725
order-status 8353
port-inventory 8486
common 9081
product-catalogue 8489
cloud-ports 10965
ipaddress 8536
geographical-location-management 8549
stub-service 8510
deploy-quote 8348
pricing-quotes 8493
internet-connections 9293
order-notifications 11725
order-status 8353
port-inventory 8486
common 9081
product-catalogue 8489

Can someone help me to figure this out pls?

Solution

Should be pretty-straightforward with Awk. Store the project names in a map data-structure in the endfilter3-all-b.txt file and look out for keys matching in the other file projectnameid.txt and write the output,

awk 'FNR==NR{key[$2]; next}$2 in key' endfilter3-all-b.txt projectnameid.txt

If the above results look good, write the output to the destination file by using the re-direct operator. Append > finalized-project-names.txt at the end of the above command.

Answered By - Inian

Answer Checked By - Mildred Charles (WPSolving Admin)