[SOLVED] How do I pipe comm outputs to a file?

Issue

I've used the comm command to compare two files, but I'm unable to pipe it to a third file:

comm file1 file2 > file3 

comm: file 1 is not in sorted order
comm: file 2 is not in sorted order

How do I do this? The files are sorted already.

(comm file1 file2 works and prints it out)

sample input:
file1:

21
24
31
36
40
87
105
134
...

file2:

10
21
31
36
40
40
87
103
...

comm file1 file2: works

comm file1 file2 > file3 

comm: file 1 is not in sorted order
comm: file 2 is not in sorted order

Solution

You've sorted numerically; comm works on lexically sorted files.

For instance, in file2, the line 103 is dramatically out of order with the lines 21..87. Your files must be 'plain sort sorted'.

If you've got bash (4.x), you can use process substitution:

comm <(sort file1) <(sort file2)

This runs the two commands and ensures that the comm process gets to read their standard output as if they were files.

Failing that:

(
sort -o file1 file1 &
sort -o file2 file2 &
wait
comm file1 file2
)

This uses parallelism to get the file sorted at the same time. The sub-shell (in ( ... )) ensures that you don't end up waiting for other background processes to finish.

Answered By - Jonathan Leffler