[SOLVED] Piping output of `echo` to netcat fails while piping output of `printf` passes

Issue

I am sending a command to a power supply device over TCP Ethernet using netcat (nc) on Linux Ubuntu 18.04 and 20.04. If I use echo the device fails to properly receive the command, but if I use printf it works fine. Why?

# Example command to Ethernet-connected digital power supply over TCP

# fails
echo 'measure:voltage? ch1' | timeout 0.2 nc 192.168.0.1 9999

# works
printf 'measure:voltage? ch1' | timeout 0.2 nc 192.168.0.1 9999

References:

1. timeout cmd: https://unix.stackexchange.com/questions/492766/usage-of-nc-with-timeouts-in-ms/492796#492796
   1. Update/Note (written after I wrote the answer, so the printf form below is correct): without the timeout command, we would have to use the -w option with netcat, but it accepts only whole integer wait period timeouts in seconds, like this (notice the -w 1 to set the 'w'ait period, or timeout, to 1 whole second):

      printf '%s' "my command to send" | nc -w1 192.168.0.1 9999 
      

Solution

Ugh! Found it.

Ensure you're not accidentally sending a newline char (\n) at the end of the command

It looks like echo adds a trailing newline to the string, whereas printf does NOT, and this trailing newline character is interfering with the device's ability to parse the command. If I forcefully add it (a newline char, \n) to the end of the printf cmd, then it fails too--meaning the device will not respond to the command as expected:

# fails:
printf 'measure:voltage? ch1\n' | timeout 0.2 nc 192.168.0.1 9999

...and it looks like you can suppress the trailing newline from echo by using -n:

# works!
echo -n 'measure:voltage? ch1' | timeout 0.2 nc 192.168.0.1 9999

# also works, of course, as stated in the question
printf 'measure:voltage? ch1' | timeout 0.2 nc 192.168.0.1 9999

From man echo:

-n     do not output the trailing newline

Key takeaway: use printf or echo -n to NOT have a trailing newline character at the end of your prints.

BUT, that's not the end of it! Here are two more points:

1. Don't use echo at all in any shell scripts if you want portable and expected behavior, and the ability to send any command to the device!

Using echo at all is actually a bad idea when trying to use it to send any possible string to the device! Why? Well, @Jeff Schaller pointed out the following resource to me in the comments, and it has tons of really valuable information: Unix & Linux: Why is printf better than echo?.

A few of the key reasons why NOT to use echo include:

1. Its implementation, arguments, and behavior are all a bit hacky.

2. It is not portable: it has various implementations which differ widely across systems. Some support -e and -n, some don't. Some have the -e behaviors by default without -e. Etc.

3. It cannot print -n as a command, at all on some shells. See my comment here.

   On bash on Ubuntu 18.04 and 20.04, echo "-n" should output -n. Instead, it outputs nothing since it accepts that as the same thing as the -n flag. echo -- "-n" should solve that and output -n, but it doesn't. It outputs -- -n instead. I see your point very well now. printf is better.

4. As @Charles Duffy points out in the comments: even the POSIX specification for echo recommends not using echo due to many of these inconsistencies and reasons:

   It is not possible to use echo portably across all POSIX systems...

   New applications are encouraged to use printf instead of echo.

FYI, here are a couple key quotes from Unix & Linux: Why is printf better than echo?:

Regarding echo:

...make sure that $var doesn't contain backslash characters and doesn't start with -

Regarding printf:

But remember the first argument is the format, so shouldn't contain variable/uncontrolled data.

2. Use printf properly as printf '%s' "my command string", NOT printf "my command string"

See more of the discussion in the comments below, between myself and @Charles Duffy. You should use printf like this instead:

# CORRECT USAGE: >>> FINAL ANSWER; DO THIS! <<<
printf '%s' 'measure:voltage? ch1' | timeout 0.2 nc 192.168.0.1 9999  

# NOT this:
printf 'measure:voltage? ch1' | timeout 0.2 nc 192.168.0.1 9999

This is because the first argument passed to printf is parsed as the format string. If the command string you're trying to send to the device is %s, for instance, this doesn't send anything!:

printf "%s"

The output is empty since printf interprets the first argument as a format string, and %s means something special in format strings. But this does send the %s as a literal:

printf "%s" "%s"

The output is

%s

The first "%s" in the arguments to printf is the format string, and the second "s" is the string literal to substitute into the format string where the first %s lies, according to the printf specification.

This means that the correct usage of printf will be able to send ANY command to the device, whereas the incorrect usage will strip out any formatting chars such as %s, %02X, %u, etc etc--any of the printf-style format string special chars or sequences.

See also:

1. Unix & Linux: Why is printf better than echo?
2. the POSIX specification for echo

Answered By - Gabriel Staples

Answer Checked By - Cary Denson (WPSolving Admin)