[SOLVED] Socket in Python with Ubuntu

Issue

I am using Python Socket to enable interaction between two local processes in a Ubuntu Server. There are some strange things happened. When I run the two processes locally in my Mac (i7), they runs quickly within 1s. However, when I run them in the Ubuntu Server, it costs over 30s.

I think the TCP is blocked between Send() and Recv(). When I just let the server send and the client receive messages. Then the processes can also be run within 1s. So I want to know what happened and is it possible to fix it. Thanks very much!

Here is the code

Server:

if __name__=='__main__':
    server1 = ServerSocket("127.0.0.1", 4321)
    server1._conn_client() 
    print("xxx");

    start_time = time.time()
    for i in range(10000):
        server1.Recv()
        server1.Send("hello") ## delete this row to disable the server to send messages
        print(i)
    end_time = time.time()
    print("Server time:", end_time - start_time)

    server1.Send("hello")
    msg = server1.Recv()
    print("hi:", msg)
    server1.close()

Client:

if __name__ == '__main__':
    client = ClientSocket("127.0.0.1", 4321)
    client.Connect()
    print("xxx");

    start = time.time()
    for i in range(10000):
        client.Send("hello from client1")
        client.Recv() ## delete this to disable the cleint to receive messages
    end = time.time()
    print("Client time:", end - start)

    msg =client.Recv()
    print(msg)
    client.Send("hello from client1")
    client.close()

Socket functions defined from others:

class ServerSocket(object):
    def __init__(self, ip="127.0.0.1", port=7778):
        self.ip = ip
        self.port = port
        self._buffer_size = 80000
        self._init_socket()
        self._socket.settimeout(3000)

    def _init_socket(self):
        self._socket = socket.socket()
        ip_port = (self.ip, self.port)
        self._socket.bind(ip_port)

    def _conn_client(self):
        self._socket.listen(100)
        self._client_socket, addr = self._socket.accept()
        self._client_socket.settimeout(3000)

    def Recv(self):
        msg_len_pack = self._client_socket.recv(4)
        # print(struct.unpack('i', msg_len_pack))
        msg_len = struct.unpack('i', msg_len_pack)[0]
        # print("msg_len:", msg_len, "buffer:", self._buffer_size)
        if msg_len < self._buffer_size:
            msg_pack = self._client_socket.recv(msg_len)
            msg = json.loads(msg_pack.decode('utf-8'))
        return msg


    def Send(self,data):
        msg_pack = json.dumps(data).encode('utf-8')
        msg_len_pack = struct.pack('i', len(msg_pack))
        self._client_socket.send(msg_len_pack)
        self._client_socket.send(msg_pack) 

    def close(self):
        self._client_socket.close()

class ClientSocket(object):
    def __init__(self, ip="127.0.0.1", port=7778):
        self.ip = ip  
        self.port = port  
        self._buffer_size = 80000 

    def Connect(self):
        self._socket = socket.socket()  
        self._socket.connect((self.ip,self.port)) 

    def Send(self, data):
        msg_pack = json.dumps(data).encode('utf-8')
        msg_len_pack = struct.pack('i', len(msg_pack))
        self._socket.send(msg_len_pack)
        self._socket.send(msg_pack) 

    def Recv(self):
        msg_len_pack = self._socket.recv(4) 
        msg_len = struct.unpack('i', msg_len_pack)[0]
        if msg_len < self._buffer_size:
            msg_pack = self._socket.recv(msg_len)
            msg = json.loads(msg_pack.decode('utf-8'))
        return msg


    def close(self):
        self._socket.close()

Solution

Huge delay comes from your optimization :D Ahah, joke apart, I understand your will of using struct to send "just" the size of the msg over 4 bytes, and then later on sending the full msg, while the receiver waits for the exact msg length, etc... But that's a "False" optimization in your case. Just stick to a default buffer size like 1024 or 4096 or whatever power of 2 not too big not too small.

Then deal with it : You can still send JSON data to give info about the future, about next "big message" which will be 4 GB or whatever, and it will be sent over multiple socket packets, and that's all

Also, let's avoid doubling code when it's identical, in your case a single class can be done :

class Socket(object):
    def __init__(self, ip="127.0.0.1", port=7778, buffer=1024, is_server=False):
        self.buffer = buffer
        if is_server:
            self._socket = socket.socket()
            self._socket.bind((ip, port))
            self._socket.listen()
            self._connected_socket, addr = self._socket.accept()
        else:
            self._connected_socket = socket.socket()
            self._connected_socket.connect((ip, port))

    def Recv(self):
        msg_pack = self._connected_socket.recv(self.buffer)
        msg = json.loads(msg_pack.decode('utf-8'))
        return msg

    def Send(self, data):
        msg_pack = json.dumps(data).encode('utf-8')
        self._connected_socket.send(msg_pack)

    def close(self):
        self._connected_socket.close()

Then you just instantiate a different way for client :

if __name__ == '__main__':
    client = Socket(port=4321)
    print("xxx");

    start = time.time()
    for i in range(10000):
        client.Send("hello from client1")
        client.Recv()
    end = time.time()
    print("Client time:", end - start)

    msg = client.Recv()
    print(msg)
    client.Send("hello from client1")
    client.close()

And for server :

if __name__=='__main__':
    server1 = Socket(port=4321, is_server=True)
    print("xxx");

    start_time = time.time()
    for i in range(10000):
        server1.Recv()
        server1.Send("hello")
        print(i)
    end_time = time.time()
    print("Server time:", end_time - start_time)

    server1.Send("hello")
    msg = server1.Recv()
    print("hi:", msg)
    server1.close()

Here on Ubuntu the loop over 10 000 is under 0.3 seconds in total, including the prints which take most of the time I guess...

Answered By - Tricotou

Answer Checked By - Katrina (WPSolving Volunteer)