Issue
I have a cgi script which imports functions from a different python file. I'm using these files to setup a Hadoop cluster. So it's mostly got commands.getstatusoutput codes. The cgi file just calls these functions while passing the parameters.
But it doesn't execute the functions. This works fine when i run it through terminal. I would appreciate any help on this.
cgi file:
#!/usr/bin/python
import cgi
import cgitb
import os
import commands
import base2
cgitb.enable()
print "content-type:text/html"
print ""
dn_iplist=[]
base2.ipscanner();
base2.freemem();
base2.procno();
base2.freehd();
dn_iplist = sorted(base2.spacedict, key=lambda x: float(base2.spacedict[x][:-1]), reverse=True)[:2]
masterdict = base2.memdict.copy()
for key in dn_iplist:
if key in masterdict:
del masterdict[key]
#print masterdict
#####################Resource Manager IP
rmip = max(masterdict, key=lambda key: int(masterdict[key]))
del masterdict [rmip]
#####################Namenode IP
nnip = max(masterdict, key=lambda key: int(masterdict[key]))
del masterdict[nnip]
#####################Client IP
cip = max(masterdict, key=lambda key: int(masterdict[key]))
print """
<html>
<head>
<title>Hadoop Cluster Services - Create and Manage your Hadoop clusters</title>
<link rel="stylesheet" type="text/css" href="/css/style1.css">
</head>
<body>
<br><b>
<div class="drop">
<ul class="drop_menu">
<li><a href="home.html">HOME</a></li>
<li><a href="create.cgi">CLUSTER</a>
<ul>
<li><a href="/loader.html">Create Cluster</a></li>
<li><a href="#">Manage Cluster</a></li>
</ul>
</li>
<li><a href="#">SERVICES</a>
<ul>
<li><a href="#">Shell</a></li>
<li><a href="#">Create Cluster</a></li>
<li><a href="#">System Admin</a></li>
</ul>
</li>
</ul>
</div>
<br><br><br><br><br>
<div id="autoips">
Resource Manager IP: """
print rmip
print "<br><br>Namenode IP: "
nport = "9001"
print nnip + ":" + nport
print "<br><br>Client IP: "
print cip
print "<br><br>Datanodes/NodeManagers: "
for k in dn_iplist:
print k + ","
print """<br><br><input type="submit" value="Accept"> """
print "<br><br><br> BUILDING CLUSTER"
print"""
</div>
</body>
</html>
"""
base2.make_rm(rmip,nnip,nport)
print "<br>"
base2.make_nn(nnip,nport)
print "<br>"
base2.make_client(nnip,nport,rmip,cip)
print "<br>"
for dnip in dn_iplist:
print dnip
base2.make_dnnm(nnip,nport,rmip,dnip)
print "<br>"
a part of the py file:
#resourcemanager creation
def make_rm(rmip,nip,nport):
fp = open("temp1.txt",mode="w")
fp.write('<?xml version="1.0"?>\n<!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.-->\n\n<configuration>\n\n<!-- Site specific YARN configuration properties -->\n\n<property>\n<name>yarn.resourcemanager.resource-tracker.address</name>\n<value>'+rmip+':8025</value>\n</property>\n\n<property>\n<name>yarn.resourcemanager.scheduler.address</name>\n<value>'+rmip+':8030</value>\n</property>\n\n</configuration>')
fp.close()
commands.getstatusoutput("sshpass -p redhat scp temp1.txt "+rmip+":/hadoop2/etc/hadoop/yarn-site.xml")
fp = open("temp2.txt",mode="w")
fp.write('<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?>\n<!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.-->\n\n<!-- Put site-specific property overrides in this file. -->\n\n <configuration>\n\n<property>\n <name>fs.default.name</name>\n<value>'+nip+':' +nport+ '</value>\n</property>\n</configuration>\n')
fp.close()
commands.getstatusoutput("sshpass -p redhat scp temp2.txt "+rmip+":/hadoop2/etc/hadoop/core-site.xml")
# print("Resourcemanager setup.\n")
commands.getstatusoutput("sshpass -p redhat ssh -o StrictHostKeyChecking=no root@"+rmip+" yarn-daemon.sh start resourcemanager")
a = commands.getstatusoutput("sshpass -p redhat ssh -o StrictHostKeyChecking=no root@"+rmip+" /usr/java/jdk1.7.0_79/bin/jps | grep 'ResourceManager' | awk '{print $2}'")
if a[1] == "ResourceManager":
print("ResourceManager setup complete!")
This is the result i get. It shows Setup complete. But nothing's been done. But it executes perfectly via terminal.
There should be a similar Datanode setup complete! beside the 192.168.100.107 IP at the bottom. This code isn't copying the temp files to the other VM.
EDIT: Could it be something related to the commands.getstatusoutput command?? Why is it only affecting scp
Solution
Finally fixed it. I had to add a root@ipaddress to the scp commands
from
commands.getstatusoutput("sshpass -p redhat scp temp1.txt "+rmip+":/hadoop2/etc/hadoop/core-site.xml")
to
commands.getstatusoutput("sshpass -p redhat scp temp1.txt root@"+rmip+":/hadoop2/etc/hadoop/core-site.xml")
This change has to be made to all the ssh/scp commands. Without this, httpd tries to ssh using the "apache" user which doesn't have root permissions. This is the reason for the code to fail in the browser.
Answered By - Aditya