[SOLVED] Is there any reason why my AWK functions work only on a shortened version of my file

Issue

I have a simple AWK function:

awk '
    BEGIN { FS=" "; RS="\n\n" ; OFS="\n"; ORS="\n" }
    /ms Response/ { print $0 }
    ' $FILE

The FILE is a large log that holds sections like this:

2021-10-13 12:15:12 CDT 526ms Request 
POST / HTTP/1.1 
Content-Type: application/x-www-form-urlencoded 
Host: xxxxxxxxxxxxxxxxxxx 
Content-Length: 279 

<query xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><product><name>drill</name><price>99</price><stock>5</stock></product>/query> 
2021-10-13 12:15:12 CDT 880ms Received

2021-10-13 12:15:12 CDT 896ms Response 
HTTP/1.1 200 OK 
Content-Type: application/xml 
Content-Length: 472

 <?xml version="1.0"?> 
<query type="c" xmlns="xxxxxxxxxxxxxx">  
<product>
<name>screwdriver</name>
<price>5</price>
<stock>51</stock>
</product>
</query>

2021-10-13 12:15:12 CDT 947ms Request 
POST / HTTP/1.1 
Content-Type: application/x-www-form-urlencoded 
Host: xxxxxxxxxxxxxxx
Content-Length: 515 
Expect: 100-continue

The above is just a snippet, the file continues for over 14000 lines, repeating the same pattern.

Now when I run my AWK function on the whole file, it just returns the whole file back. But when I run it on a file that was created with (cat $FILE | head -200), It works as expected by returning:

2021-10-13 12:15:12 CDT 896ms Response
HTTP/1.1 200 OK
Content-Type: application/xml
Content-Length: 472

2021-10-13 12:15:13 CDT 075ms Response
HTTP/1.1 200 OK
Content-Type: application/xml
Content-Length: 3207

2021-10-13 12:15:13 CDT 208ms Response
HTTP/1.1 200 OK
Content-Type: application/xml
Content-Length: 4220

Why can I run this on a shortened file but when I run it on a longer version, it does not work? Even though its the same data in the file?

I am working on Ubuntu 18.04 LTS in Bash.

Thank you!

Solution

@markp-fuso's comment helped me. My input file had Windows line endings and I just needed to run the below command prior to executing the AWK:

tr -d '\15\32' < OGfile.txt > unixFile.txt

Then it ran as expected.

I received additional syntax help from the following question: Convert line endings

Answered By - Shelby Anne

Answer Checked By - David Marino (WPSolving Volunteer)