[SOLVED] Write info in variables without create a file

Issue

I have some script on bash

#!/usr/bin/env bash

time_event="`date +%Y-%m-%d' '%H:%M:%S`"
current_day="`date +%Y%m%d`"
event="start_job"

src='hdfs://table1/'
dst='hdfs://table2/'

hadoop distcp -bandwidth 256 -m 50 -delete -update -strategy dynamic $src $dst &> log.txt

status=($?)

link=$(grep -oP "The url to track the job: \K[^']+" log.txt)
job=$(grep -oP "Submitting tokens for job: \K[^']+" log.txt)

hive -e "insert into table <table> partition (day) select '$src' as source_path, '$dst' as target_path, '$time_event' as time_event, '$event' as event, '$status' as status, '$link' as url_job, '$job' as url_pipeline, $current_day as day;"

How can I do the same without write info in file and read it?

Solution

Simply store the result in a variable via command substitution, like you are already doing for link and job.

Also make sure to quote your parameters when expanding, otherwise you might run into issues later on (word splitting, wildcard expansion, etc.)

log="$(hadoop distcp -bandwidth 256 -m 50 -delete -update -strategy dynamic "$src" "$dst" 2>&1)"

link=$(printf '%s\n' "$log" | grep -oP "The url to track the job: \K[^']+")
job=$(printf '%s\n' "$log" | grep -oP "Submitting tokens for job: \K[^']+")

Answered By - knittl

Answer Checked By - David Marino (WPSolving Volunteer)