Wlan communication problems

lmo

Hello all!

I was contacting support for this, but got no answer from them for more than a week, so I'll ask here.

I'm trying to connect two pycom devices and transfer some data via wlan, but no success so far.

I tried with two Wipys, Wipy and android phone, two Fipys. None of the examples that I found are working. I also tried to change the firmware and I'm getting different kind of errors, for different versions of firmware. Some can't even scan the wlan networks, other crash when I'm trying to use the socket, in best case I got the client (Wipy) to successfully connect to the server socket (running on the android). When I try to send some data, the client reports the number of bytes sent, but the server receives nothing. It is working if I use two android phones for both client and server.

As far as Fipys go, I did manage to establish two way communication via Lora, but no success with wlan.

Is there some specific firmware version that should work and I can try, or is there some proper example that does work, combination of both?

jcaron

@lmo It would probably help a lot if you gave us the specifics of one setup you tried and did not work, with the relevant source, the resulting logs and errors, the firmware versions, etc.

ShanevanJ

@lmo Post some code where you are trying to connect to the wifi and lets try and help you diagnose the issues?

lmo

Here's the current code:

from network import WLAN
import time
import machine
import socket
import os

print(str(os.uname()))

wlan = WLAN(mode=WLAN.STA)

print('start scan')
nets = wlan.scan()
for net in nets:
    print('network found')
    if net.ssid == 'xxxxxx':
        print('connecting to network')
        wlan.connect(net.ssid, auth=(net.sec, 'xxxxxx'), timeout=5000)
        while not wlan.isconnected():
            machine.idle()
        print('wlan connected')
        break
print('end scan')

if wlan.isconnected():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    time.sleep(1)
    print("socket created")
    server_address = socket.getaddrinfo('192.168.1.123', 1234)[0][-1]
    sock.connect(server_address)
    print("socket connected")

    time.sleep(1)
    message = 'test message'
    bytes_sent = sock.sendall(message)
    print("bytes sent = "+str(bytes_sent))

    bytes_received = 0
    bytes_expected = 3
    while bytes_received < bytes_expected:
        data = sock.recv(16)
        bytes_received += len(data)
        print("bytes received = "+str(bytes_received))

    sock.close()
    print("socket closed")

print("end")

and results from different firmwares:

(sysname='WiPy', nodename='WiPy', release='1.20.2.rc3', version='v1.11-2037465 on 2020-02-05', machine='WiPy with ESP32', pybytes='1.3.1')
start scan
end scan
end


(sysname='WiPy', nodename='WiPy', release='1.20.1', version='v1.11-12f4ce0 on 2019-10-06', machine='WiPy with ESP32', pybytes='1.1.2')
start scan
end scan
end


(sysname='WiPy', nodename='WiPy', release='1.20.0.rc13', version='v1.9.4-94bb382 on 2019-08-22', machine='WiPy with ESP32')
start scan
Traceback (most recent call last):
  File "main.py", line 12, in <module>
OSError: Scan operation Failed!


(sysname='WiPy', nodename='WiPy', release='1.19.0.b4', version='v1.9.4-50f84dd on 2018-07-13', machine='WiPy with ESP32')
start scan
network found
network found
connecting to network
wlan connected
end scan
socket created
socket connected
bytes sent = 12

With the latest firmwares, it just doesn't find any wifi networks. With some previous ones it seems to send the message and I can see that socket was established on the phone, but no data is received, so no response is given back on the phone.

ShanevanJ

@lmo This what I am using to establish my network connection and so far has been super reliable - it is similar to what you have - could you try it and see how it works for you ? (change said and password to suit your wifi)

known_nets = {
    'ssid': {'pwd': 'password'}
    # 'ssid2': {'pwd': 'password2', 'wlan_config':  ('10.0.0.114', '255.255.0.0', '10.0.0.1', '10.0.0.1')}, # (ip, subnet_mask, gateway, DNS_server)
}


# Start of code
print("Booting " + str(config.deviceID))
print(str(os.uname()))

if machine.reset_cause() != machine.SOFT_RESET:
    print("Initialise WiFi")
    from network import WLAN
    wl = WLAN()
    wl.mode(WLAN.STA)
    original_ssid = wl.ssid()
    original_auth = wl.auth()

    print("Scanning for known wifi nets")
    available_nets = wl.scan()
    nets = frozenset([e.ssid for e in available_nets])

    known_nets_names = frozenset([key for key in known_nets])
    net_to_use = list(nets & known_nets_names)
    try:
        net_to_use = net_to_use[0]
        net_properties = known_nets[net_to_use]
        pwd = net_properties['pwd']
        sec = [e.sec for e in available_nets if e.ssid == net_to_use][0]
        if 'wlan_config' in net_properties:
            wl.ifconfig(config=net_properties['wlan_config'])
        wl.connect(net_to_use, (sec, pwd), timeout=10000)
        while not wl.isconnected():
            machine.idle()  # save power while waiting
        config.wifiConnected = True
        print("Connected to "+net_to_use +
              " with IP address:" + wl.ifconfig()[0])

    except Exception as e:
        print("Failed to connect to any known network")

If this works reliably - then try the sockets section ?

lmo

Thanks for the example, it works on the 1.19.x firmware, but not on the 1.20.x, wl.scan() finds 0 networks.

ShanevanJ

@lmo wow - mine is running

(sysname='WiPy', nodename='WiPy', release='1.20.2.rc3', version='v1.11-2037465 on 2020-02-05', machine='WiPy with ESP32', pybytes='1.3.1')

So there is something else happening here ?

jcaron

@lmo Haven't played with that in a while, but I wouldn't be surprised if wlan.scan actually only returned the networks already (recently) received rather than actually waiting for them to arrive. So if you have wifi_on_boot set to false and do a scan right after starting Wi-Fi, it may not return anything.

Things you could try:

wait a bit between init and scan
loop around wl.scan and the associated logic until you find something
connect directly without doing the scan first (this is actually not needed if you already know the SSID you want to connect to)

Not sure if anything changed in that respect between those firmware versions.

When you do have the network connected, can you try to connect to a server or other computer rather than your phone? It would probably help with debugging, you could use tcpdump or Wireshark to see what is actually exchanged on the wire.

On the server you can just use nc -l (netcat) to listen on a port and dump the received data on your terminal session.

lmo

A few second delay between init and scan fixed the issue with wifi discovery, so now I can use the latest firmware. Still no luck with the socket issue.

ShanevanJ

@lmo I know why mine is working - I added the device to pybytes and used its network configuration manager to setup my wifi connection - so it already is connected before it runs the wifi code I sent - this only checks for new networks outside of what was configured in the pybytes section

jcaron

@lmo As I wrote above, I would make sure the endpoint is something where you can do easy low-level debugging (with tcpdump or Wireshark) to see on what end the problem lies.

What are you using the read from the socket on the server side? Hopefully not some library which reads "lines" and thus waits for a newline which is never sent? Or maybe is blocking waiting for a larger number of bytes to be received?

I would run:

tcpdump -ni <interface name> -s 0 -vvvXXX -c 100 tcp port 1234 &
nc -l -p 1234 | hexdump -C

On a Unix box and see what shows up when pointing the script on the WiPy to that box.

lmo

Thanks @jcaron, it was the newline problem. Now that the client is working ok, I'll try to fix the server on wipy. I'll report how it goes.

lmo

For the server I'm trying with this:

from network import WLAN
import time
import machine
import socket

wlan = WLAN(mode=WLAN.STA_AP, ssid='test', auth=(WLAN.WPA2,'testpass'))
time.sleep(6)

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("socket created")
server_address = socket.getaddrinfo('localhost', 1234)[0][-1]
sock.bind(server_address)
print("socket bound")

sock.listen(1)
print("listening")

connection, client_address = sock.accept()
print("connection from "+str(client_address))

while True:
    data = connection.recv(16)
    if data:
        print("received "+str(data))
        connection.sendall("ok!")
        print("sent response")
        break
            
connection.close()
socket.close()

print("end")

I guess the ip of the server should be 192.168.4.1? Access point is working and client connects to it ok. No error is reported on the server, on client it crashes on sock.connect(server_address) with OSError: [Errno 104] ECONNRESET. I tried also to use my existing wifi network, so that both client and server connect to it (server not used as an access point), but I'm still receiving the same error. Since the client should be working now, I guess I'm missing something on the server side. Last thing it outputs is "listening", so it's waiting on sock.accept().

jcaron

@lmo never tried having a *Py act as a TCP server, but I don’t think localhost is the appropriate choice. That should restrict connections to only local ones.

You probably want to use either the IP address or 0.0.0.0. Not sure if there is a more appropriate way of writing this.

The error should have been ECONNREFUSED rather than ECONNRESET, though, but that may be a minor deviation.

lmo

Thanks again @jcaron, 0.0.0.0 or ip (192.168.4.1) instead of localhost did resolve the problem, and server responds properly now.
I have one more issue to resolve. About a minute after server is set up, no matter if it's still waiting for a client to connect or if it's already finished with execution (and sockets were closed) it crashes with:

Unhandled exception in callback handler
Traceback (most recent call last):
  File "_pybytes.py", line 495, in __smart_config_callback
TimeoutError: Connection to AP Timeout!

How should this be handled?

jcaron

@lmo I'd say you should disable pybytes. Depending on the firmware version, this is done by either using a different image or modifying settings (see https://forum.pycom.io/topic/5649/new-firmware-release-1-20-2-rc3 for details on the latter).

andreas

Dear @lmo,

@jcaron said in Wlan communication problems:

disable pybytes using a different image

We are regularly building firmware images not including Pybytes, so the most recent WiPy-1.20.2.rc3-0.8.0-vanilla-squirrel.tar.gz might make you happy.

With kind regards,
Andreas.

lmo

Thanks @jcaron and @andreas, with this vanilla firmware the crash is gone. But I'm having some problems with bluetooth on newer firmware versions. I'll ask about that in another post. For anyone looking for example of just using the wlan, here is the code for a simple one time send and receive a response.

Client:

from network import WLAN
import time
import machine
import socket

wlan = WLAN(mode=WLAN.STA)
time.sleep(6)

wlan.connect('test', auth=(WLAN.WPA2, 'testpass'), timeout=5000)
while not wlan.isconnected():
    machine.idle()
print('wlan connected '+str(wlan.ifconfig()))

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = socket.getaddrinfo('192.168.4.1', 1234)[0][-1]
sock.connect(server_address)

message = 'test message'
sock.sendall(message)
print("sent "+message)

bytes_received = 0
while bytes_received < 3:
    data = sock.recv(3)
    bytes_received += len(data)
    print("received "+str(data.decode()))

sock.close()

Server:

from network import WLAN
import time
import machine
import socket

wlan = WLAN(mode=WLAN.STA_AP, ssid='test', auth=(WLAN.WPA2,'testpass'))
time.sleep(6)

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = socket.getaddrinfo('0.0.0.0', 1234)[0][-1]
sock.bind(server_address)
sock.listen(1)
print("listening")

connection, client_address = sock.accept()
print("connection from "+str(client_address))

while True:
    data = connection.recv(16)
    if data:
        print("received "+str(data))
        response = "ok!"
        connection.sendall(response)
        print("sent "+response)
        break

connection.close()
sock.close()

nestorld

@jcaron Thank you! This helped me to figure out that when starting we have to use the scan in a try-except logic inside a while because the firsts attempts of scan get nothing and rather than returning an empty array, it arises an exception like this:
OSError: Scan operation Failed!

busytha busytha

it works on the 1.19.x firmware, but not on the 1.20.x, wl.scan() finds 0 networks.

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

Wlan communication problems

Pycom on Twitter