[SOLVED] Gnu parallel pass each line via stdin

Issue

I want to encode each line of a file using base64. However, base64 (and other command line utilities) do not allow to pass values via command line arguments but require them to be passed via stdin.

In my case, I tried to run cat alphanum.txt | parallel --pipe base64. This however passes the whole file to base64. As far as I could figure out the reason for this is that with the --pipe switch active, Gnu parallel splits up inputs into "blocks", where each block is 1M in size by default. Is there a way to make the blocks one line long each? I tried using -L 1, -l 1 and --line-buffered, none of which seem to work.

Solution

The simplest way to achieve this is like so:

cat alphanum.txt | parallel "echo {} | base64"

Gnu parallel replaces {} with the line. As such, each line value is echoed via stdin to base64.

See https://www.gnu.org/software/parallel/man.html#options

Edit: The switch -N1 seems to be what I was looking for.

cat alphanum.txt | parallel --pipe -N1 base64

Answered By - J Heschl

Answer Checked By - Marilyn (WPSolving Volunteer)