Tuesday, January 4, 2022

[SOLVED] How can I use grep command to capture correlation ID along with java stacktrace and multiline message from log file

Issue

In the log file, each request sent in will be automatically assigned with one unique correlation ID, for example X-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa and X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as in the log sample below that created by log4j. 

2019-06-03 11:27:22,697|X-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa|INFO |com.example.ExampleService|Start execute
2019-06-03 11:27:22,697|X-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa|INFO |com.example.ExampleService|ENTRY| performIntegration()
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|INFO |com.example.ExampleService|EXIT| executeService()
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|INFO |com.example.ExampleService|EXIT| MyObject = This,
is
a
multiline log
message
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|ERROR|com.example.ExampleAdapter|error
com.example.ABCRuntimeException: Network error
    at ...
    at ...
Caused by: ...
    at ...
    at ...
2019-06-03 11:27:22,698|X-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa|INFO |com.example.ExampleService|EXIT| performIntegration()
2019-06-03 11:27:22,699|X-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa|INFO |com.example.ExampleService|EXIT| executeService()
2019-06-03 11:27:22,699|X-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa|ERROR|com.example.ExampleAdapter| another error
com.example.ABCRuntimeException: Network error
    at ...
    at ...
Caused by: ...
    at ...
    at ...

How can I grep the correlation id with multiline message and java stacktrace together? For example I want to grep X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb, then the expected output should be as per below

2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|INFO |com.example.ExampleService|EXIT| executeService()
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|INFO |com.example.ExampleService|EXIT| MyObject = This,
is
a
multiline log
message
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|ERROR|com.example.ExampleAdapter|error
com.example.ABCRuntimeException: Network error
    at ...
    at ...
Caused by: ...
    at ...
    at ...

Is linux grep command able to achieve this? Or is there any other recommended tools? However, I am not allowed to install new package on the production servers. The operation system is Red Hat 7.


Solution

Try:

$ awk -F\| -v id='X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' '/^[0-9]{4}-[0-9]{2}-[0-9]{2} /{f=0} $2==id{f=1} f' file
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|INFO |com.example.ExampleService|EXIT| executeService()
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|INFO |com.example.ExampleService|EXIT| MyObject = This,
is
a
multiline log
message
2019-06-03 11:27:20,759|X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb|ERROR|com.example.ExampleAdapter|error
com.example.ABCRuntimeException: Network error
    at ...
    at ...
Caused by: ...
    at ...
    at ...

How it works

  • -F\|

    Use | as the field separator.

  • -v id='X-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'

    Create an awk variable called id and assign to it the id of interest.

  • /^[0-9]{4}-[0-9]{2}-[0-9]{2} /{f=0}

    If the current line begins with a date, set variable f to false (zero).

  • $2==id{f=1}

    If the second field matches the desired id, set variable f to true (one).

  • f

    If f is true, print the line.



Answered By - John1024
at