Help with multi client socket threading

[Sebastian San Martin]

Hi everyone,
I'm having problems implementing a multi-client socket using threads. The problem is that I can't find a way for the "receive_data()" routine to wait until the threads are done, it ends before the threads return. I'm also not sure if the way I'm using the threads is correct, right now the "receive_thread()" function tries to append the received data to the list "data_f" but it doesn't seem to be working, any ideas?

My code is something like this:

import _threads

def receive_thread(connection,client_address,data_f, lock):
    print('connection from', client_address)    
    data = connection.recv(128)
    print('received {!r}'.format(data))
    if len(data)==0:
        connection.close()
    else:
        with lock:
            data_f.append(data)        
        connection.close()

    return

def receive_data():
    """Receive data from socket connection"""

    import socket
    import sys
    import _thread

    # Create a TCP/IP socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # Bind the socket to the port
    server_address = ('0.0.0.0', 8080)
    print('starting up on {} port {}'.format(*server_address))
    sock.bind(server_address)    

    # Listen for incoming connections
    sock.listen(5)

    # Create data_f list and lock
    data_f = []    
    lock = _thread.allocate_lock()

    max_connections = 3 # for now I suppose a max number of connections 
               
    for i in range(max_connections):
        connection, client_address = sock.accept()
        _thread.start_new_thread(receive_thread,(connection,client_address,data_f,lock))
   

    ### In this part I should handle all of the received data to return it to the main script ###

    sock.close()

The idea of this code is to return the received data from all of the socket connections to a main script. For now I'm assuming that a fixed number of clients will try to connect, so it won't be necessary to have the socket listening all the time.

Thank you all :)

[Sebastian San Martin]

@jcaron Thank you very much for your advice, after much thinking and trying out different approaches I decided to not use threads at all, the select/poll library is all what I needed. I'm using this for a wireless sensor network and since there's no Wake on Lan on WiPy 3.0 I figured out a solution in which some devices need to go into deepsleep mode after receiving and forwarding all of the expected data. That's why I needed to wait for all of the connections to have been received so I can continue with the main program.

Thank u again :)

[jcaron]

@Sebastian-San-Martin I would be curious to understand your use case and why you need to wait for all connections to have been received to process the data, but let’s use that assumption.

Each of your threads is started in parallel. As soon as you start a new thread, execution on the current thread continues. Nothing waits for any thread to be done.

What you could do if you want to use threads is to have one lock for each thread. You lock it at the beginning of the thread, unlock it as soon as the thread ends, and in your main thread you try to get a lock on it and wait for it to be released. There may be a bit if a timing issue as you want to make sure the main thread does not acquire the lock before the connection thread.

You could also simply use variables to indicate the state of each thread (or an array, really) or just to count the number of active threads, but then you have to continuously loop over a check on this state (whereas the lock will just wait).

Another alternative is to maintain the state or number of threads, and at the end of the connection thread, check if the conditions for the final code are met (number of connections and all threads ended), and at this point run the final code. This is probably the simplest option.

Yet another option is not to use threads at all but to manage the multiple connections with uselect, and use a logic similar to the above to decide when you are “done” and run the final code.