[SOLVED] read, IFS, and Bash loops strange behaviour

Issue

I have the following toy script:

names=Tom,Jack,Bill

echo 'For loop output:'
IFS=',' read -ra names_array <<< "$names"
for name in "${names_array[@]}"; do
        echo "$name"
done

echo 'While loop output (correct):'
while read -rd, name; do
        echo "$name"
done <<< "$names,"

echo 'While loop output (missing last element):'
while read -rd, name; do
        echo "$name"
done <<< "$names"

echo 'While loop output (not using IFS at all):'
while IFS=',' read -r name; do
        echo "$name"
done <<< "$names,"

This prints the following:

For loop output:
Tom
Jack
Bill
While loop output (correct):
Tom
Jack
Bill
While loop output (missing last element):
Tom
Jack
While loop output (not using IFS at all):
Tom,Jack,Bill,

---

Here are my questions:

1. Why does read not need an extra , at the end of "$names" when reading into an array (in the for loop example), but does need one when using the delimiter option (in the first and second while loop examples)?
2. In the last while loop, why does read seem to completely ignore the IFS value? I would have imagined that setting IFS is the same as setting the delimiter via -d.

It would be interesting to properly understand the reasons why those 4 examples behave the way they do.

Solution

To the first question: When you provide a delimiter using -d and read gets to the end of input without finding that delimiter, it exits non-zero which causes the body of the loop not to execute. In your case it reads Bill but does not find a terminating comma so it "fails". Appending a terminating character is one workaround. Another is to check if your variable was actually populated:

# if read exits non-zero, check if name has anything in it before exiting the loop
while read -rd, name || [[ "$name" ]]; do
        echo "$name"
done <<< "$names"

To the second question: The -d option tells read where to terminate an input line, and then IFS is used to separate that line into fields. In other words, read -d, will only read up to a comma, while IFS=, read will read up to a newline and then split whatever was read using commas as the separator.

Note that, if there are more fields than you provide variable names, all the remaining fields are placed into the last variable. Essentially, you are splitting the word but then stuffing it all back into a single variable because that's all you provided to store it in.

Answered By - tjm3772

Answer Checked By - Terry (WPSolving Volunteer)