Very occasional hard freeze from Guru Meditation Error (StoreProhibited) on HTTP GET over LTE with Gpy



  • Hello, I will begin by saying that i am new to everything Pycom and Python.
    I'm trying to use a Gpy to upload sensor data being read with UART and then sent to a 3rd party API through LTE with urequests.

    It is ultimately meant to be running a transmission cycle and then enter deep sleep for a few hours and repeat. Currently the sleep time is set at 5 minutes for stress testing.

    It works perfectly fine most of the time. Sometimes over a week without issues (with 5 mins between cycles), until it suddenly freezes with a Guru Meditation Error (StoreProhibited) on the urequests.get(url) seen in code below.
    The only way out then is to press the reset button or power cycle the device. Machine.WDT is used but has no effect.

    Pycom firmware is updated to version 1.20.2.r1 and modem firmware is LR5.1.1.0-47510 (from factory).

    cycleNonce = self.useNonce()
    print("cycleNonce: " + str(cycleNonce))
    checksum = str(cycleNonce) + ":1;" + str(distance) + ";"
    params = {
        "hash": self.hashData(checksum),
        "identifier": self.identifier,
        "nonce": cycleNonce,
        "values[0].key": '1',
        "values[0].value": distance
    }
     url = self.dataUrl + '?' + urequests.urlencode(params)
    print(url)
    response = urequests.get(url)
    

    Regrettably I have no control over the API and the resulting URL string is usually somewhere over 200 characters long.

    Guru Meditation Error: Core  1 panic'ed (StoreProhibited). Exception was unhandled.
    Core 1 register dump:
    PC      : 0x40096d1d  PS      : 0x00060b33  A0      : 0x800958cf  A1      : 0x3ffd2580
    A2      : 0x3ffd279c  A3      : 0x00000001  A4      : 0x3ffd2794  A5      : 0x00000001
    A6      : 0x000000fe  A7      : 0x00060023  A8      : 0x01000100  A9      : 0x09045a28
    A10     : 0x04005923  A11     : 0xffffffff  A12     : 0xb33fffff  A13     : 0x0000abab
    A14     : 0x00000003  A15     : 0x00060023  SAR     : 0x00000010  EXCCAUSE: 0x0000001d
    EXCVADDR: 0x0400592b  LBEG    : 0x40091b99  LEND    : 0x40091bcd  LCOUNT  : 0x00000000
    
    ELF file SHA256: 0000000000000000000000000000000000000000000000000000000000000000
    
    Backtrace: 0x40096d1d:0x3ffd2580 0x400958cc:0x3ffd25a0 0x40094473:0x3ffd25c0 0x4008e2eb:0x3ffd2600 0x4008e8a5:0x3ffd2640 0x401688c9:0x3ffd2680 0x40110e6d:0x3ffd26c0
    

    The unpredictability of the problem makes it very hard to debug so here i am on the forums hoping that someone knows what might be causing this. Grateful for any information on the matter.



  • @SciWax said in Very occasional hard freeze from Guru Meditation Error (StoreProhibited) on HTTP GET over LTE with Gpy:

    @knaris said in Very occasional hard freeze from Guru Meditation Error (StoreProhibited) on HTTP GET over LTE with Gpy:

    That is exactly the kind of information i was hoping to get. Many thanks to both of you!

    I will try switching to 41065 and see if that solves it.

    Did it solve your issues? I'm doing pretty much the same as you. Collecting a measurement string via UART and then using urequests to send the data to my server. The modem doesnt seem to be stable though. Today, I've downgraded my modem firmware to 41065 and I'm using GPy-1.20.2.rc6-0.10.1-vanilla-squirrel at the moment for the GPy. This combination also doesn't seem to be stable for me at the moment.

    Okay, the combination runs nearly for one day stable now, sending data every minute. I had to include some more exceptions. I hope it stays this way.



  • @knaris said in Very occasional hard freeze from Guru Meditation Error (StoreProhibited) on HTTP GET over LTE with Gpy:

    That is exactly the kind of information i was hoping to get. Many thanks to both of you!

    I will try switching to 41065 and see if that solves it.

    Did it solve your issues? I'm doing pretty much the same as you. Collecting a measurement string via UART and then using urequests to send the data to my server. The modem doesnt seem to be stable though. Today, I've downgraded my modem firmware to 41065 and I'm using GPy-1.20.2.rc6-0.10.1-vanilla-squirrel at the moment for the GPy. This combination also doesn't seem to be stable for me at the moment.



  • That is exactly the kind of information i was hoping to get. Many thanks to both of you!

    I will try switching to 41065 and see if that solves it.


  • Global Moderator

    The 41065 modem firmware does seem to be a little 'more better' than the 47510 version. As the coredump does indeed seem to be related to the LTE modem UART connection. Thanks for the addition @kjm!



  • We're running brief data uploads through cat M1 on a gpy with deepsleep. Have been mercifully free of crashes for a year now with 1.20.1.r1/littleFS on the gpy & 41065 on the modem.



  • @Gijs
    Thank you very much for your response.
    I did actually print out the remaining memory on one of the frozen cycles but didn't think to include it in the post. From gc.mem_free(): 2545648, which feels like it should be plenty (but could still end up corrupted?).

    I followed the instructions in your link with the coredump posted and got this trace. From what i can tell, it seems to point at code which reads a UART response from the modem (perhaps a modem firmware issue?). The only other recorded coredump had an identical backtrace.

    uxListRemove
    /Users/ehlers/pycom/pycom-esp-idf/components/freertos/list.c:214
    vTaskPlaceOnEventList
    /Users/ehlers/pycom/pycom-esp-idf/components/freertos/tasks.c:2906
    xQueueGenericReceive
    /Users/ehlers/pycom/pycom-esp-idf/components/freertos/queue.c:1590
    prvReceiveGeneric
    /Users/ehlers/pycom/pycom-esp-idf/components/esp_ringbuf/ringbuf.c:563
    xRingbufferReceive
    /Users/ehlers/pycom/pycom-esp-idf/components/esp_ringbuf/ringbuf.c:850
    uart_read_bytes
    /Users/ehlers/pycom/pycom-esp-idf/components/driver/uart.c:1225
    TASK_UART_EVT
    /var/jenkins_home/workspace/com_pycom-micropython-sigfox_Dev/esp32/lte/lteppp.c:588
    

    I did not have the good judgement to save earlier coredumps but definitely will in the future.


  • Global Moderator

    Hi,
    Thanks a lot for the clear error description!
    From what I could find, the StoreProhobited coredump is related to a corruption in the memory (overflow / null pointer). You can try the following: https://docs.pycom.io/advance/coredump/ to figure out what exactly the coredump means (but its quite advanced), and where the error is (in the sourcecode). If it keeps pointing towards the same location for every crash, there might be something we are able to fix here.
    It is quite hard to detect any sort of memory leak we might have here, or that the memory is just full when trying to load the page.
    Sorry I could not be of more help here
    Best,
    Gijs


Log in to reply
 

Pycom on Twitter