[SOLVED] Alternative to using sed inside loop

Issue

I want to replace multiple (non-consecutive) lines of a big file, using files that contain one single line. What I found to work is

for i in ${list[@]}; do
   line=$(cat file_$i.txt);
   sed -i "$i c $line" bigfile.txt;
done

Here, list contains the lines I want to replace. Here is an example:

$ list=(1 3 4 7)
$ cat file_1.txt
this is the new line 1
$ cat file_3.txt
this is the new line 3
$ cat file_4.txt
this is the new line 4
$ cat file_7.txt
this is the new line 7

$ cat bigfile.txt
line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8

The output of the above script is

$ cat bigfile.txt 
newline 1
line 2
newline 3
newline 4
line 5
line 6
newline 7
line 8

It works, but in every step of the loop sed opens and reads the whole file, as I understand it, so this method is very slow. What are the other faster ways to do this, preferably using sed?

Solution

A possible solution:

sed "$(for i in $list; do echo "$i c $(cat file_$i.txt)"; done)" bigfile.txt

($list could have to be ${list[@]} or ${list[*]} or anything else, depending how it is constructed.)

Your original loop is used to build a Sed script each line of which is like 1 c content_of_file_1_dot_txt; this script is then run only once on bigfile.txt.

Answered By - Enlico

Answer Checked By - Senaida (WPSolving Volunteer)