Issue
I have a tab separated file and need to filter the rows with zero value.
$ head my_file.txt
id SRR1385501
chr1:11672:+::chr1:12009:+ 0
chr1:14830:-::chr1:14969:- 8
chr1:14830:-::chr1:15795:- 0
chr1:15039:-::chr1:15795:- 2
chr1:15948:-::chr1:16606:- 10
I am trying with the command below (awk version: GNU 4.0.2, but it is still removing header)
$ awk '$2 !=0' my_file.txt
chr1:14830:-::chr1:14969:- 8
chr1:15039:-::chr1:15795:- 2
chr1:15948:-::chr1:16606:- 10
My desired output is:
id SRR1385501
chr1:14830:-::chr1:14969:- 8
chr1:15039:-::chr1:15795:- 2
chr1:15948:-::chr1:16606:- 10
Solution
Using any awk, this will do what you want:
$ awk '$2 != "0"' file
id SRR1385501
chr1:14830:-::chr1:14969:- 8
chr1:15039:-::chr1:15795:- 2
chr1:15948:-::chr1:16606:- 10
The above assumes your 2nd column always has exactly the string 0 for "numerical zero", not 0.0 or -0 or +0 or anything else.
The result you're getting is inconsistent with the way awk comparisons are supposed to work:
When two operands are compared, either string comparison or numeric comparison may be used. This depends upon the attributes of the operands, according to the following symmetric matrix:
+---------------------------------------- | STRING NUMERIC STRNUM +---------+------------------------------ | STRING | string string string | NUMERIC | string numeric numeric | STRNUM | string numeric numeric +---------+------------------------------
$2 should be of type STRING for the first line of your input because SRR1385501 doesn't look like a number to awk, and type STRNUM for the rest because all of those other values such as 0 and 8 do look like numbers but awk can't initially tell if you want to use them as numbers or strings.
According to the above table, then, the type of comparison between SRR1385501 (STRING) and 0 (NUMERIC) should be string and those 2 values are not the same string so the header should be printed. For the rest it's STRNUM vs NUMERIC which is simply a numeric comparison.
I suggest getting a new version of gawk (we're now on 5.3) as yours is out of date and maybe it contains a bug, but I highly doubt it as this would be major breakage. More likely there's something about your input file or your script that are different from what you've shown and told us about so far.
Answered By - Ed Morton Answer Checked By - Dawn Plyler (WPSolving Volunteer)