Issue
I'm using this script to try to upload files to an S3 bucket using cURL.
#S3 parameters
$S3KEY="AKIAYZL7ANTULMU355OD"
$S3SECRET="AKIAYZL7ANTULMU355OD"
$S3BUCKET="quixel-ml-team-data"
$S3STORAGETYPE="STANDARD" #REDUCED_REDUNDANCY or STANDARD etc.
$AWSREGION="us-west-2"
file_path="C:\Users\DELL\AppData\Local\Temp\input_settings.txt"
aws_path="seam-removal\"
bucket="${S3BUCKET}"
date=$(date -R)
acl="x-amz-acl:private"
content_type="application/x-compressed-tar"
storage_type="x-amz-storage-class:${S3STORAGETYPE}"
string="PUT\n\n$content_type\n$date\n$acl\n$storage_type\n/$bucket$aws_path${file_path##/*/}"
signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64)
curl -s --retry 3 --retry-delay 10 -X PUT -T "$file_path" \
-H "Host: $bucket.${AWSREGION}.amazonaws.com" \
-H "Date: $date" \
-H "Content-Type: $content_type" \
-H "$storage_type" \
-H "$acl" \
-H "Authorization: AWS ${S3KEY}:$signature" \
"https://$bucket.${AWSREGION}.amazonaws.com$aws_path${file_path##/*/}"
However, I'm getting these errors. Any idea about them? Sorry, I'm a novice at this sort of programming.
file_path=C:\Users\DELL\AppData\Local\Temp\input_settings.txt : The term
'file_path=C:\Users\DELL\AppData\Local\Temp\input_settings.txt' is not recognized as the name of a cmdlet, function,
script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is
correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:11 char:1
+ file_path="C:\Users\DELL\AppData\Local\Temp\input_settings.txt"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (file_path=C:\Us...ut_settings.txt:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
aws_path=seam-removal\ : The term 'aws_path=seam-removal\' is not recognized as the name of a cmdlet, function, script
file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct
and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:12 char:1
+ aws_path="seam-removal\"
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (aws_path=seam-removal\:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
bucket=${S3BUCKET} : The term 'bucket=${S3BUCKET}' is not recognized as the name of a cmdlet, function, script file,
or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and
try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:13 char:1
+ bucket="${S3BUCKET}"
+ ~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (bucket=${S3BUCKET}:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
date=$(date -R) : The term 'date=$(date -R)' is not recognized as the name of a cmdlet, function, script file, or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:14 char:1
+ date=$(date -R)
+ ~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (date=$(date -R):String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
acl=x-amz-acl:private : The term 'acl=x-amz-acl:private' is not recognized as the name of a cmdlet, function, script
file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct
and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:15 char:1
+ acl="x-amz-acl:private"
+ ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (acl=x-amz-acl:private:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
content_type=application/x-compressed-tar : The term 'content_type=application/x-compressed-tar' is not recognized as
the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:16 char:1
+ content_type="application/x-compressed-tar"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (content_type=ap...-compressed-tar:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
storage_type=x-amz-storage-class:${S3STORAGETYPE} : The term 'storage_type=x-amz-storage-class:${S3STORAGETYPE}' is
not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or
if a path was included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:17 char:1
+ storage_type="x-amz-storage-class:${S3STORAGETYPE}"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (storage_type=x-...{S3STORAGETYPE}:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
string=PUT\n\n$content_type\n$date\n$acl\n$storage_type\n/$bucket$aws_path${file_path##/*/} : The module 'string=PUT'
could not be loaded. For more information, run 'Import-Module string=PUT'.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:18 char:1
+ string="PUT\n\n$content_type\n$date\n$acl\n$storage_type\n/$bucket$aw ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (string=PUT\n\n$...file_path##/*/}:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoLoadModule
signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64) : The term 'signature=$(echo -en
"${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64)' is not recognized as the name of a cmdlet, function,
script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is
correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:19 char:1
+ signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (signature=$(ech...inary | base64):String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
Invoke-WebRequest : Parameter cannot be processed because the parameter name 'T' is ambiguous. Possible matches
include: -TimeoutSec -TransferEncoding.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:20 char:43
+ curl -s --retry 3 --retry-delay 10 -X PUT -T "$file_path" \
+ ~~
+ CategoryInfo : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException
+ FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
-H : The term '-H' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:21 char:5
+ -H "Host: $bucket.${AWSREGION}.amazonaws.com" \
+ ~~
+ CategoryInfo : ObjectNotFound: (-H:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
-H : The term '-H' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:22 char:5
+ -H "Date: $date" \
+ ~~
+ CategoryInfo : ObjectNotFound: (-H:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
-H : The term '-H' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:23 char:5
+ -H "Content-Type: $content_type" \
+ ~~
+ CategoryInfo : ObjectNotFound: (-H:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
-H : The term '-H' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:24 char:5
+ -H "$storage_type" \
+ ~~
+ CategoryInfo : ObjectNotFound: (-H:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
-H : The term '-H' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:25 char:5
+ -H "$acl" \
+ ~~
+ CategoryInfo : ObjectNotFound: (-H:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
-H : The term '-H' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\DELL\Desktop\Abdullah Chand\quixel-ml-khalid-chand\script_upload_to_aws_using_curl.ps1:26 char:5
+ -H "Authorization: AWS ${S3KEY}:$signature" \
-
~~- CategoryInfo : ObjectNotFound: (-H:String) [], CommandNotFoundException
- FullyQualifiedErrorId : CommandNotFoundException
Solution
You're mixing a PowerShell with a traditional Unix script.
Further, there's no need to call out the external utilities like OpenSSL and Curl since PowerShell has similar capabilities built in. To sign the URL and perform the Upload without using the AWS SDK, the script would look something like this:
$method = 'PUT'
$service = 's3'
$bucket = "SAMPLES3BUCKETNAME"
$key = 'example-s3-key.txt'
$local_file = "test.txt"
$region = 'us-west-2'
$host1 = $bucket + '.s3-' + $region + '.amazonaws.com'
$access_key = 'SAMPLEACCESSKEY'
$secret_key = 'SAMPLESECRETKEY'
function HmacSHA256($message, $secret)
{
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = $secret
$signature = $hmacsha.ComputeHash([Text.Encoding]::ASCII.GetBytes($message))
return $signature
}
function getSignatureKey($key, $dateStamp, $regionName, $serviceName)
{
$kSecret = [Text.Encoding]::UTF8.GetBytes(("AWS4" + $key).toCharArray())
$kDate = HmacSHA256 $dateStamp $kSecret;
$kRegion = HmacSHA256 $regionName $kDate;
$kService = HmacSHA256 $serviceName $kRegion;
$kSigning = HmacSHA256 "aws4_request" $kService;
return $kSigning
}
function hash($request)
{
$hasher = [System.Security.Cryptography.SHA256]::Create()
$content = [Text.Encoding]::UTF8.GetBytes($request)
$bytes = $hasher.ComputeHash($content)
return ($bytes|ForEach-Object ToString x2) -join ''
}
$now = [DateTime]::UtcNow
$amz_date = $now.ToString('yyyyMMddTHHmmssZ')
$datestamp = $now.ToString('yyyyMMdd')
$signed_headers = 'host'
$credential_scope = $datestamp + '/' + $region + '/' + $service + '/' + 'aws4_request'
$canonical_querystring = 'X-Amz-Algorithm=AWS4-HMAC-SHA256'
$canonical_querystring += '&X-Amz-Credential=' + [uri]::EscapeDataString(($access_key + '/' + $credential_scope))
$canonical_querystring += '&X-Amz-Date=' + $amz_date
$canonical_querystring += '&X-Amz-Expires=300'
$canonical_querystring += '&X-Amz-SignedHeaders=' + $signed_headers
$canonical_headers = 'host:' + $host1 + "`n"
$canonical_request = $method + "`n"
$canonical_request += "/" + $key + "`n"
$canonical_request += $canonical_querystring + "`n"
$canonical_request += $canonical_headers + "`n"
$canonical_request += $signed_headers + "`n"
$canonical_request += "UNSIGNED-PAYLOAD"
$algorithm = 'AWS4-HMAC-SHA256'
$canonical_request_hash = hash -request $canonical_request
$string_to_sign = $algorithm + "`n"
$string_to_sign += $amz_date + "`n"
$string_to_sign += $credential_scope + "`n"
$string_to_sign += $canonical_request_hash
$signing_key = getSignatureKey $secret_key $datestamp $region $service
$signature = HmacSHA256 -secret $signing_key -message $string_to_sign
$signature = ($signature|ForEach-Object ToString x2) -join ''
$canonical_querystring += '&X-Amz-Signature=' + $signature
$request_url = "http://" + $host1 + "/" + $key + "?" + $canonical_querystring
# Write-Host $request_url
# Using Curl:
# curl.exe '-s', '--retry', '3', '--retry-delay', '10', '-X', 'PUT', '-T', $local_file, $request_url
# Or just put to the URL directly:
Invoke-RestMethod -Uri $request_url -Method Put -InFile $local_file
Answered By - Anon Coward Answer Checked By - Cary Denson (WPSolving Admin)