Issue
A file contains ASCII characters to represent hex value. (each line ends in ", ")
cat temp.txt
0x6A,
0xF2,
0x44,
.....
0xF8,
0x1A,
I try to combines each 16 words/lines into one line like this
cat hex_result.txt
0x6A, 0xF2, 0x44, 0xF8, 0x45, 0x41, 0x88, 0xD1,0x4E, 0x8B, 0xA3, 0xB1, 0x8C, 0xE0, 0x37, 0x2D,
....
0xE2, 0x1C, 0x06, 0x8A, 0x75, 0x2B, 0xBC, 0x3C, 0xC5, 0x08, 0xB7, 0x4E, 0xB0, 0xE4, 0xF8, 0x1A,
Is any bash commands to accomplish it ?
Solution
Benchmarking six different merging methods,
for merging specific number of lines.
Basicaly, there are many commands:
pr - convert text files for printing
pr -at16 <file
Try:
pr -a -t -16 < <(seq 1 42)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42
xargs - build and execute command lines from standard input
... and executes the command (default is /bin/echo) ...
xargs -n 16 <file
Try:
xargs -n 16 < <(seq 1 42)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42
paste - merge lines of files
printf -v pasteargs %*s 16
paste -d\ ${pasteargs// /- } < <(seq 1 42)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42
sed - stream editor for filtering and transforming text
printf -v sedstr 'N;s/\\n/ /;%.0s' {2..16};
sed -e "$sedstr" < <(seq 1 42)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42
awk - pattern scanning and processing language
awk 'NR%16{printf "%s ",$0;next;}1' < <(seq 1 42)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42
But, you could use pure bash:
group=()
while read -r line;do
group+=("$line")
(( ${#group[@]} > 15 ))&&{
echo "${group[*]}"
group=()
}
done < <(seq 1 42) ; echo "${group[*]}"
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42
or as a function:
lgrp () {
local group=() line
while read -r line; do
group+=("$line")
((${#group[@]}>=$1)) && {
echo "${group[*]}"
group=()
}
done
[ "$group" ] && echo "${group[*]}"
}
or
lgrp () { local g=() l;while read -r l;do g+=("$l");((${#g[@]}>=$1))&&{
echo "${g[*]}";g=();};done;[ "$g" ] && echo "${g[*]}";}
then
lgrp 16 < <(seq 1 42)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40 41 42
( Note: All this tests was arbitrarily done over 42 values, don't ask my why! ;-)
Other languages
Of course, by using any language, you could do same:
perl -ne 'chomp;$r.=$_." ";( 15 < ++$cnt) && do {
printf "%s\n", $1 if $r =~ /^(.*) $/;$r="";$cnt=0;
};END{print $r."\n"}' < <(seq 1 42)
Like python, ruby, lisp, C, ...
Comparison of execution time.
Ok, there are more than 3 simple ways, let do a little bench. How I do it:
lgrp () { local g=() l;while read -r l;do g+=("$l");((${#g[@]}>=
$1))&&{ echo "${g[*]}";g=();};done;[ "$g" ] && echo "${g[*]}";}
export -f lgrp
printf -v sedcmd '%*s' 15
sedcmd=${sedcmd// /N;s/\\n/ /;}
export sedcmd
{
printf "%-12s\n" Method
printf %7s\\n count real user system count real user system
for cmd in 'paste -d " " -{,,,}{,,,}' 'pr -at16' \
'sed -e "$sedcmd"' \
$'awk \47NR%16{printf "%s ",$0;next;}1;END{print ""}\47'\
$'perl -ne \47chomp;$r.=$_." ";( 15 < ++$cnt) && do {
printf "%s\n", $1 if $r =~ /^(.*) $/;$r="";$cnt=0;
};END{print $r."\n"}\47' 'lgrp 16' 'xargs -n 16'
do
printf %-12s\\n ${cmd%% *}
for length in 5042 50042; do
printf %7s\\n $(bash -c "TIMEFORMAT=$'%R %U %S';
time $cmd < <(seq 1 $length) | wc -l" 2>&1)
done
done
} | paste -d $'\t' -{,,,,,,,,}
(This could be cut'n paste in a bash terminal). Produce, on my computer:
Method count real user system count real user system
paste 316 0.002 0.002 0.000 3128 0.003 0.003 0.000
pr 316 0.003 0.000 0.003 3128 0.008 0.005 0.002
sed 316 0.005 0.001 0.003 3128 0.018 0.019 0.000
awk 316 0.003 0.001 0.003 3128 0.017 0.017 0.002
perl 316 0.008 0.002 0.004 3128 0.017 0.014 0.004
lgrp 316 0.058 0.042 0.021 3128 0.733 0.568 0.307
xargs 316 0.232 0.178 0.058 3128 2.249 1.730 0.555
There is same bench on my raspberry pi:
Method count real user system count real user system
paste 316 0.149 0.032 0.012 3128 0.204 0.014 0.054
pr 316 0.163 0.017 0.038 3128 0.418 0.069 0.096
sed 316 0.275 0.088 0.031 3128 1.586 0.697 0.045
awk 316 0.440 0.146 0.049 3128 2.809 1.305 0.050
perl 316 0.421 0.122 0.040 3128 2.042 0.902 0.067
lgrp 316 7.261 3.159 0.446 3128 71.733 31.223 3.558
xargs 316 9.464 3.038 1.066 3128 93.607 32.035 9.177
Hopefully all line count are same, then paste are clearly the quicker, followed by pr. Pure bash function is not slower than xargs (I'm surprised about poor performance of xargs!).
Answered By - F. Hauri - Give Up GitHub Answer Checked By - Marilyn (WPSolving Volunteer)