[SOLVED] how to grep the multiple strings from a file and print them in group wise

Issue

I'm trying to grep a list of errors form host log file, being a huge file it prints a lot of data and hard to see what errors repeated and logged

    0x45bae19d6bc0 IO type 16648 (READ) isOrdered:NO isSplit:NO isEncr:NO since 7990 msec status I/O error
        Throttled: 82 IO failed on disk e3d17cdb-3190-9e21-ea45-4cff39420501, Wake up 0x45ba3a34f9c0 with status I/O error
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10432 microseconds to 5392073 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10444 microseconds to 10822733 microseconds.
        naa.5000c500bb7a661f performance has improved. I/O latency reduced from 10822733 microseconds to 2163435 microseconds.
        naa.5000c500bb7a661f performance has improved. I/O latency reduced from 2163435 microseconds to 426054 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10465 microseconds to 925119 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10469 microseconds to 1904014 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10472 microseconds to 3936215 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10479 microseconds to 8517984 microseconds.
        cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
        cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
        cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
        cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10490 microseconds to 17358740 microseconds.
        0x45bae0fefe40 IO type 16648 (READ) isOrdered:NO isSplit:NO isEncr:NO since 48543 msec status I/O error
        Throttled: 82 IO failed on disk e3d17cdb-3190-ea45-4cff39420501, Wake up 0x45da36318840 with status I/O error
        naa.5000c500ba661f performance has improved. I/O latency reduced from 17358740 microseconds to 3372968 microseconds.
        naa.5000c500bb7a661f performance has improved. I/O latency reduced from 3372968 microseconds to 674458 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10677 microseconds to 1353205 microseconds.
        naa.5000c500bb7a661f performance has improved. I/O latency reduced from 1353205 microseconds to 268942 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10682 microseconds to 419051 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10682 microseconds to 872847 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10684 microseconds to 1770518 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10687 microseconds to 3640051 microseconds.
        0x45dae4fe25c0 IO type 16648 (READ) isOrdered:NO isSplit:NO isEncr:NO since 15991 msec status I/O error
        Throttled: 82 IO failed on disk e3d17cdb-3190--ea45-4cff39420501, Wake up 0x45da362677c0 with status I/O error
        0x45dae4fe2340 IO type 16648 (READ) isOrdered:NO isSplit:NO isEncr:NO since 24806 msec status I/O error
        cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
        cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
        cpu10:36926358)MemSchedAdmit: 471: Admission failure in path: vm.36926352/vmmanon.36926352
        cpu23:36926381)MemSchedAdmit: 471: Admission failure in path: vm.36926375/vmmanon.36926375
        Throttled: 82 IO failed on disk e3d17cdb-3190-9e21-ea45-4cff39420501, Wake up 0x45ba3abe8880 with status I/O error
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10696 microseconds to 7557465 microseconds.
        Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10711 microseconds to 15202991 microseconds.
        naa.5000c500bb7a661f performance has improved. I/O latency reduced from 15202991 microseconds to 2944264 microseconds.
        naa.5000c500bb7a661f performance has improved. I/O latency reduced from 2944264 microseconds to 577176 microseconds.
        naa.5000c500bb7a661f performance has improved. I/O latency reduced from 577176 microseconds to 112712 microseconds.

I'm expecting the following output, I've searched alot of places and didn't find a suitable solution, hoping it may possible with awk and sed

   egrep -i "latency|I/O error|Failure" error.log 
    
    Failure

   cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
   cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
   cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure
   cpu3:2099278)Migrate: 448: Error reading from pending connection: Failure

    IO Errors

  cpu5:2098752)WARNING: LSOM: RCIOCompletionLoop:93: Throttled: 82 IO failed on disk e3d17cdb-3190-9e21-ea45-4cff39420501, Wake up 0x45da362677c0 with status I/O error
   cpu6:2097866)LSOMCommon: IORETRYCompleteIO:470: Throttled: 0x45dae4fe2340 IO type 16648 (READ) isOrdered:NO isSplit:NO isEncr:NO since 24806 msec status I/O error
cpu2:2098752)WARNING: LSOM: RCIOCompletionLoop:93: Throttled: 82 IO failed on disk e3d17cdb-3190-9e21-ea45-4cff39420501, Wake up 0x45ba3abe8880 with status I/O error
 cpu9:2099365 opID=add9908b)WARNING: ScsiDeviceIO: 12028: READ CAPACITY on device “naa.5000c500bb7a661f” from Plugin “HPP” failed. I/O error

    LAtency

cpu5:2097866)WARNING: ScsiDeviceIO: 1596: Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10682 microseconds to 419051 microseconds.
cpu19:2097867)WARNING: ScsiDeviceIO: 1596: Device naa.5000c500bb7a661f performance has deteriorated. I/O latency increased from average value of 10682 microseconds to 872847 microseconds

Solution

Assumptions:

• if multiple patterns match a single line we'll display the line in each of the output groups
• group headings are exact reprints of the search patterns (ie, won't be reformatting the group headers as is done in the question where search pattern I/O error becomes group heading IO Errors)
• there is no requirement to match on only whole words (eg, failure will match on failure, failures, nonfailures, stufffailuresXYZ)
• within an output group we wish to maintain the input ordering of the rows

The question's current input/output doesn't match so until fixed we'll use a small(er) set of input data for demonstration purposes:

$ cat test.log
you can ignore this line
you should match this line on abcLaTeNcYxyz
yeah, match this line on Failures and throttled
you can ignore this line
more matches for i/o error and latency
single match on I/O error
couple more matches on failures
couple more matches on failure
ignore this line, too

Adding a non-matching string (no-match) to the mix:

$ patterns='latency|I/O error|Failure|throttled|no-match'

One GNU awk idea (for array of arrays and PROCINFO["sorted_in"]):

awk -v plist="${patterns}" '
BEGIN   { IGNORECASE=1
          delete groups

          n=split(plist,arr,"|")                     # break plist up into components
          for (i=1;i<=n;i++) {
              ptns[arr[i]]                           # assign as indices of ptns[] array for easier processing
              groups[arr[i]][0]                      # place holder to allow us to print an empty group
          }
        }

        { for (ptn in ptns)                          # loop through list of patterns and ...
              if ($0 ~ ptn)                          # if found then ...
                 groups[ptn][c++]=$0                 # save in groups[] array
        }

END     { PROCINFO["sorted_in"]="@ind_str_asc"
          for (ptn in ptns) {
              printf "\n######### %s\n\n", ptn
              PROCINFO["sorted_in"]="@ind_num_asc"   # sort the c++ values in ascending order => maintain input ordering
              for (i in groups[ptn])
                  if (groups[ptn][i] != "")
                     print groups[ptn][i]
          }
        }
' test.log

This generates:

######### Failure

yeah, match this line on Failures and throttled
couple more matches on failures
couple more matches on failure

######### I/O error

more matches for i/o error and latency
single match on I/O error

######### latency

you should match this line on abcLaTeNcYxyz
more matches for i/o error and latency

######### no-match


######### throttled

yeah, match this line on Failures and throttled

Answered By - markp-fuso

Answer Checked By - Cary Denson (WPSolving Admin)