Issue
I try to make a tool to get list of proxies and You have downloaded Index for one of the free proxy sites I use this :
wget http://free-proxy.cz/en/proxylist/country/all/https/ping/all
and outputs something like that :
<script type="text/javascript">document.write(Base64.decode("MTg1LjExNC4xMzcuMTQ="))</script></td><td style=""><span class="fport" style=''>12195</span></td><td><small>HTTPS</small></td><td class="left"><div style="padding-left:2px"><img src="/flags/blank.gif" class="flag flag-ua" alt="Ukraine" /> <a href="/en/proxylist/country/UA/all/ping/all">Ukraine</a></div></td><td class="small"><small></small></td><td class="small"><small></small></td><td class="small"><small>High anonymity</small></td><td> <i class="icon-black icon-question-sign"></i></td><td><small>2.4%</small><div class="progress"><div class="fill" style="width:4%;background-color:red;"></div></div></td><td><div style="padding-left:5px"><small>649 ms</small> <div class="progress"><div class="fill" style="width:94%;background-color:#A5DA74;;"></div></div></div></td><td><small>8 hours ago</small></td></tr><tr><td style="text-align:center" class="left"><script type="text/javascript">document.write(Base64.decode("MTYxLjk3LjEzOC4yMzg="))</script></td><td style=""><span class="fport" style=''>3128</span></td><td>
As you can see the IP is encrypted in base64 and port is normal
I try to grep base64 codes first and this is work ↓
echo (outputs) | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2
and I try this code to get ports ↓
echo (output) | grep -Eo "(class=\"fport\" style=''>[0-9]{1,9})" | cut -d '>' -f2
how can I mix it and make it like that
(base64 code):(port)
and after that I wanna decrypt the base64 code and make it look like :
IP:PORT
Solution
1st step
base64
is not an encryption, but an encoding. If you are working on
Linux or other Unix-variants, the command base64
, base64-encoder/decoder,
will be pre-installed. If not, it will be easily installed with your
OS-dependent package manager.
Then please try to execute:
base64 -d <<< "MTg1LjExNC4xMzcuMTQ="
It will output:
185.114.137.14
2nd step
Then we can combine the base64 decoder with your command pipeline.
The problem is base64-coding ignores newlines and we need to process
the result of the pipeline line by line. Assuming the variable $output
holds the output of the wget
command, please try:
while IFS= read -r line; do
base64 -d <<< "$line"
echo
done < <(echo "$output" | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2)
It will print something like:
185.114.137.14
161.97.138.238
The <(command)
notation is a process substitution and the output of
echo .. grep .. cut
pipeline is fed to the while
loop via stdin
and the while
loop processes the base64-encoded strings line by line.
3rd step
Now we want to merge the IPs and PORTs in a format of IP:PORT
.
We can make use of paste
command. The final script will be:
paste -d ":" <(
while IFS= read -r line; do
base64 -d <<< "$line"
echo
done < <(echo "$output" | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2)
) \
<(echo "$output" | grep -Eo "(class=\"fport\" style=''>[0-9]{1,9})" | cut -d '>' -f2)
Output:
185.114.137.14:12195
161.97.138.238:3128
The paste
command takes filenames as arguments. Here we make use
of process substitution again in a manner as: paste <(command1) <(command2)
which saves to create temporary files.
Answered By - tshiono