errno 12 ENOMEM ?

[soren]

EDIT: I did not use "gc.enable", so figuing that might be the cause. Haven't had chance to try it out yet though.

I have 4 LoPy's scanning for 4 MAC addresses of 4 different estimote beacons, everything runs fine in short runs - but I often get this error:
OSERROR: [Errno 12] ENOMEM on the line where I use "sendto" to send (via UDP) to a raspberry pie server.

I have tried including a gc.collect in my main loop, but this did not fix the issue.
I'm in a building with many wireless signals going on, and I'm getting bluetooth devices with the bluetooth.get_advertisements() - so wondering if this list could somehow grow to an extent that explodes memory?? seems ver strange though, even with gc.collect.

Here's my code:

from network import Bluetooth
from network import WLAN
import socket
#from network import socket
import pycom
import gc
import time
import machine
import binascii
from machine import Timer
pycom.heartbeat(False)
pycom.rgbled(0x000000) # turn off led

lopyID = 3 # set this unique to each lopy
print('started! lopy id: ', lopyID)
#for breaking out of while(True) loops. Removed when program is finished
chrono = Timer.Chrono() #
chrono.start()

wlan = WLAN(mode=WLAN.STA)
nets = wlan.scan()

pycom.rgbled(0x7f0770)
time.sleep(0.5)
pycom.rgbled(0x000000) # turn off led
time.sleep(1)
pycom.rgbled(0x7f0770)
time.sleep(0.5)
pycom.rgbled(0x000000) # turn off led
time.sleep(1)
pycom.rgbled(0x7f0770)
time.sleep(0.5)
pycom.rgbled(0x000000) # turn off led
time.sleep(1)
#pycom.rgbled(0x7f0770)
#time.sleep(0.5)
#pycom.rgbled(0x000000) # turn off led
#time.sleep(1)
#pycom.rgbled(0x7f0000) # red
#time.sleep(0.5)
#pycom.rgbled(0x000000) # turn off led
#time.sleep(1)

#pycom.rgbled(0x7f0000)
gc.collect()
for net in nets: # connect to rpi wifi
     if net.ssid == 'cooppifi':
         print('rpi found!')
         wlan.connect(net.ssid, auth=(net.sec, 'cooppifi2017'), timeout=4000)
         for _ in range(20):
             time.sleep(1)
             if wlan.isconnected():
                 print("connected to rpi")
                 break
             print('.', end='')
         else:
             print("could not connect to wifi, resetting lopy...")
             machine.reset()

bluetooth = Bluetooth()
bluetooth.start_scan(-1)
adv = None # bluetooth advertisementa

time.sleep(1)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

def sendToSocket(beaconID, rssi):
    RSSIpositive = rssi * -1
    string = str(lopyID) + "," + str(beaconID) + "," + str(RSSIpositive) + ","
    data = bytearray(string)
    addr = socket.getaddrinfo('10.42.0.1', 7007)[0][-1]
    s.sendto(data, addr)

# new mac adresses:
#C7:C3:83:EE:AA:A4
#FB:6F:11:25:79:9B
#F7:F0:BB:A6:ED:F7
#C3:C7:E7:73:7C:3F


while True:
    gc.collect()
    pycom.rgbled(0x007f00)
    advList = bluetooth.get_advertisements()
    for adv in advList:

        macRaw = adv[0] # binary mac address
        macReal = binascii.hexlify(macRaw) # convert to hexadecimal
        if macReal == b'fb6f1125799b': # old: b'f3bc0b6feb4c':
            print('beacon A rssi: ', adv[3]) # adv[3] is rssi
            sendToSocket(beaconID = 1, rssi = adv[3])
        elif macReal == b'f7f0bba6edf7': # old: b'edbed48d0b87':
            print('beacon B rssi: ', adv[3])
            sendToSocket(beaconID = 2, rssi = adv[3])
        elif macReal == b'c7c383eeaaa4': # <-- old
            print('beacon C rssi: ', adv[3])
            sendToSocket(beaconID = 3, rssi = adv[3])
        elif macReal == b'c3c7e7737c3f': # old: b'cd87e6a38dc2':
            print('beacon D rssi: ', adv[3])
            sendToSocket(beaconID = 4, rssi = adv[3]) # adv[3]
        gc.collect()
s.close()

[alvaroav]

@alvaroav

Dear all,

Some advances by the end of the week.
I have tried from a clean code, just building a new program. I come back to the option to open the socket, send all data in a loop, and at the end of the loop, close the socket.
I am not so sure why, BUT now the ENOMEM error just appears from time to time. Moreover, I don't need any server (blocking or non-blocking) running.
Moreover, I place this exception into the loop, and allow me to "reconnect" and continue the program even if an error appers.

flag = 1
Nerrors = 0;
socketUDP = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

while (flag<200):
    time.sleep(0.05)
    data=bytes()
    data = data + b'counter = ' + bytes(str(flag),'utf-8')
    try:
        sent = socketUDP.sendto(data, addr)
        print(data)
        led.toggle()
        flag=flag+1
    except OSError:
        Nerrors+=1
        print('error ', Nerrors)
        continue
socketUDP.close()

I am thinking that the problem might come from the wifi connection to the router... I will check this next week.

BR

[alvaroav]

@alvaroav

Dear all,

More advances on the ENOMEN.
I still have random success or unsuccess data transmisios with a UDP client (Wipy) / UDP server (PC host) programs.
However, I tried to use a TCP server on the PC host, and the surprise is that I am able to send data from teh wipy without any ENOMEN error!
Unfortunately the TCP server is not able to receive the data frames (as expected, since I am using UDP for Tx)

This is the code for the TCP server:

import socket

# Listen on port 8000
# (to all IP addresses on this system)
#listen_addr = ("",8000)
listen_addr = ('192.168.1.158',8000)

# Set up a TCP server
TCPSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

TCPSock.bind(listen_addr)
TCPSock.listen()

print("Ready to listen\n")

while True:
	TCPSock.accept()		# accept the connection
	data, addr = TCPSock.recvfrom(1024)	
	while data:			        # till data is coming
		print(data)
		data, addr = TCPSock.recvfrom(1024)
	print("All Data Received")	# Will execute when all data is received
	#TCPSock.close()
	break

Any ideas with this new info?

When I tried to do the same with UDP, both socket.listen() and socket.accept() seems to be "not compatible" with the socet object.

I will still try to develop a success UDP server, based on this new advances.

Thanks to all!

[alvaroav]

@blindman2k @seb

Thank you very much for your help and support.

I've been working around this tipic on the last days, atill without a solution, but with some conclusions ready.

I work with a client continously sending data, and a "server" which is an sniffer, listeing to all UDP packets received (liek your UDP logger @blindman2k).
When th elogger is correctly working, and listeing, there is no error
When the sniffer is closed, or the wifi from the sniffer PC is unliked (sometime happens), the error appears.

It seems that the UDP sockets need the confirmation from reception in the server side... this is strange since I understood (maybe I am worng) that the UDP sockets do not rely on any reception acknowldege, and is a lossy protocol (instead of the TCP sockets where the transmission is controlled).

I will still working on the blocking UDP, I havce found some tutorials, but some of them are just for python and not micropython, with different commads not available on our wipy.

Moreover, my idea is to have an stand alone remote sensor sending UDP packets, and a receptor, just sniffering them when required... like the OSC protocol.

http://opensoundcontrol.org/spec-1_0

Thanks and BR

[blindman2k]

@alvaroav I saw the same behavior with the UDP client sockets. I was using UDP sockets to log to a server so I could debug issues remotely but it turned out the biggest issue was actually the UDP logger. I never solved it rather just removed the logger.

What's odd is I have another bidirectional UDP client that works just fine. Maybe the difference is I am reading its buffer with recvfrom() on every loop thus clearing its memory.

I also find the wlan.scan() function throws the same error when there are no networks visible. That exception is caught successfully by a try/except block. Something else you can try.

[seb]

One suggestion would be to set the socket as blocking, then send, then close the socket. This will ensure that all the data has been sent.

[alvaroav]

@jcaron

Dear jcaron

Exactly, is on the socketUDP.sendto line, but sometimes it happens some others not.

I open and close the socket every iteration on the loop, as suggested by other users.

[jcaron]

On what line does the error actually occur? Given the different behaviour when the socket is kept or closed I guess it should be on the socketUDP.sendto line, but just to be sure...

[alvaroav]

@alvaroav

I received today the new Wipy 3.0, and the problem is still the same.

Someone with some ideas?

Thanks

[alvaroav]

@livius

Thanks agian

BTW I already ordered a Wipy3.0 module, in order to check that the RAM increase from 512KB to 4MB avoids this issue...

I'll keep you updated.

Thanks

[livius]

@alvaroav said in errno 12 ENOMEM ?:

ENOMEM

then i have not clue maybe @daniel @administrators can help more
i do not see the reason of out of memory exception

[alvaroav]

@livius said in errno 12 ENOMEM ?:

print('free:', str(gc.mem_free()))
print('info:', str(machine.info()))
print('info:', str(gc.mem_alloc()))
print('info:', str(micropython.mem_info()))
gc.collect()

Thanks again for your help.
This is what I get when I place these lines:

b'counter = 1'
free: 50176
---------------------------------------------
System memory info (in bytes)
---------------------------------------------
MPTask stack water mark: 5396
ServersTask stack water mark: 980
TimerTask stack water mark: 2160
IdleTask stack water mark: 604
System free heap: 82232
---------------------------------------------
info: None
info: 16864
stack: 768 out of 7168
GC: total: 67008, used: 16896, free: 50112
 No. of 1-blocks: 91, 2-blocks: 26, max blk sz: 384, max free sz: 2854
info: None
b'counter = 2'
free: 50336
---------------------------------------------
System memory info (in bytes)
---------------------------------------------
MPTask stack water mark: 5396
ServersTask stack water mark: 980
TimerTask stack water mark: 2160
IdleTask stack water mark: 604
System free heap: 82048
---------------------------------------------
info: None
info: 16704
stack: 768 out of 7168
GC: total: 67008, used: 16736, free: 50272
 No. of 1-blocks: 85, 2-blocks: 24, max blk sz: 384, max free sz: 2854
info: None
b'counter = 3'
free: 50352
---------------------------------------------
System memory info (in bytes)
---------------------------------------------
MPTask stack water mark: 5396
ServersTask stack water mark: 980
TimerTask stack water mark: 2160
IdleTask stack water mark: 604
System free heap: 81864
---------------------------------------------
info: None
info: 16688
stack: 768 out of 7168
GC: total: 67008, used: 16720, free: 50288
 No. of 1-blocks: 89, 2-blocks: 24, max blk sz: 384, max free sz: 2854
info: None
Traceback (most recent call last):
  File "<stdin>", line 110, in <module>
OSError: [Errno 12] ENOMEM
>
MicroPython v1.8.6-849-83e2f7f on 2018-03-19; WiPy with ESP32
Type "help()" for more information.

[livius]

@alvaroav said in errno 12 ENOMEM ?:

add also print statment about (some are duplicated but now worry):

print('free:', str(gc.mem_free()))
print('info:', str(machine.info()))
print('info:', str(gc.mem_alloc()))
print('info:', str(micropython.mem_info()))
gc.collect()

to print above info add at start imports if you do not have it already

import gc
import machine
import micropython

[alvaroav]

@livius

Dear livius

Thanks for your reply.

I also include the

    gc.collect()
    print('where:', str(gc.mem_free()))

without succes.
This is what happens from time to time. Sometimes it reached couter= 1000

b'contador = 1'
where: 48336
b'contador = 2'
where: 48336
b'contador = 3'
where: 48368
Traceback (most recent call last):
  File "<stdin>", line 109, in <module>
OSError: [Errno 12] ENOMEM

It is really strange. I also use to perform machine.reset() to avoid any left opened connection or socket.

Thanks

[livius]

@alvaroav
I know that you have gc.enable()
but what if you add also to the end of this loop gc.collect()?

[alvaroav]

@robert-hh

@robert-hh

Dear Robert,

Thanks for your advise on gc.disable().

Regarding the UDP sockets used:

1. When I open the socket out of the send loop:

gc.enable()
socketUDP = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
flag = 1
while (flag<1000):
    time.sleep(0.1)
    data=bytes()
    data = data + b'contador = ' + bytes(str(flag),'utf-8')
    sent = socketUDP.sendto(data, addr)
    flag=flag+1
socketUDP.close()

the program breaks after 3 sent messages

when I include the socket open-close commands within the loop, sometimes it reaches the last 1000th message, some others breaks after some messages sent:

gc.enable()
flag = 1
while (flag<1000):
    socketUDP = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
    time.sleep(0.1)
    data=bytes()
    data = data + b'contador = ' + bytes(str(flag),'utf-8')
    sent = socketUDP.sendto(data, addr)
    flag=flag+1
    socketUDP.close()

Someone knows why sometimes it crashes or some others not?

Thanks!

[alvaroav]

This post is deleted!

[robert-hh]

@alvaroav Never call gc.disable(). garbage collection is normally done automatically, gc.collect() just enforces that and can be called to avoid the automatic collection at time critical situations. It will however not help if objects are not released. But I do not think that this is the issue here. As said, I've seen this error in cases, when sockets have been opened and not closed. So you should check the flow of you code, wheter sockets you use are either opened once and then used repeatedly, or closed after use and opened again when needed the next time.
Edit: I see you code behaves just like that. The only thing that seems a little bit unneeded is to get the addr value again every time you send.

[alvaroav]

Dear all,

I am working with a wipy 2.0, trying to send data with UDP sockets, and I still have the same problem reported on this thread.
I am able to send just 3 packets before the OSError: [Errno 12] happens.

Has anyone solved this issue?

I also include gc.enable() before the message loop, gc.collect() within the loop and gc.disable() before leaving the routine.
Moreover, the sent message is fixed, just to "Hello"

Thanks!

[robert-hh]

@soren That typically happens when you open too many sockets, or try to re-open a socket which is already in use. Please ensure, that you close sockets when you do not need them any more.
A subtle pitfall is, that a soft reset does not close open sockets. You have to either push the reset button or call machine.reset().