[SOLVED] Shell to run StreamSets Pipeline

Issue

I want to connect to the Control Hub and want to run the particular StreamSets pipeline using Shell Script.

id='dv3'>

Solution

In the StreamSets Control Hub web UI, click on RESTful API and then Job Runner. You will see instructions for authenticating from the command line using curl. Scroll down and you will see the API for starting a job given the job id.

Putting them together, you'll need to do something like:

# login to Control Hub security app
curl -X POST -d '{"userName":"SCHUserID", "password": "SCHUserPassword"}' \
https://cloud.streamsets.com/security/public-rest/v1/authentication/login \
--header "Content-Type:application/json" --header "X-Requested-By:SCH" \
-c cookie.txt

# generate auth token from security app
sessionToken=$(cat cookie.txt | grep SSO | rev | grep -o '^\S*' | rev)
echo "Generated session token : $sessionToken"

# Start a job using the auth token
curl -X GET https://cloud.streamsets.com/jobrunner/rest/v1/job/{jobId}/start \
--header "Content-Type:application/json" --header "X-Requested-By:SCH" \
--header "X-SS-REST-CALL:true" --header "X-SS-User-Auth-Token:$sessionToken" \
-i

Answered By - metadaddy

Answer Checked By - Gilberto Lyons (WPSolving Admin)