Random Guru meditation Wipy3-1.20.2.r4



  • Hi,
    I'm having random Guru meditation.
    I'm using a WiPy3 with custom hardware and the release v1.20.2.r4 from git with custom frozen files.
    I find this thread on esp32 forum.

    I was able to log one core dump, and decoding It.

    Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
    Core 1 register dump:
    PC      : 0x40092ffb  PS      : 0x00060e33  A0      : 0x80091523  A1      : 0x3ffdebe0  
    A2      : 0x3ffdfd1c  A3      : 0x00060e20  A4      : 0x3ffdcbc8  A5      : 0x00000001  
    A6      : 0x000000fe  A7      : 0x00000001  A8      : 0x00000000  A9      : 0x3ffc3fa0  
    A10     : 0x3ffc3fa0  A11     : 0x0000abab  A12     : 0x3ffdcbc8  A13     : 0x00000001  
    A14     : 0x000000fe  A15     : 0x3ffdfd14  SAR     : 0x00000000  EXCCAUSE: 0x0000001c  
    EXCVADDR: 0x00000004  LBEG    : 0x40094354  LEND    : 0x40094382  LCOUNT  : 0xffffffff  
    
    ELF file SHA256: 0000000000000000000000000000000000000000000000000000000000000000
    
    Backtrace: 0x40092ffb:0x3ffdebe0 0x40091520:0x3ffdec00 0x400f372c:0x3ffdec20 0x400e045d:0x3ffdecb0
    
    ================= CORE DUMP START =================
    

    Decode:

    xtensa-esp32-elf-addr2line -fe build/WIPY/release/application.elf 0x40092ffb:0x3ffdebe0 0x40091520:0x3ffdec00 0x400f372c:0x3ffdec20 0x400e045d:0x3ffdecb0
    uxListRemove
    /home/luis/Desktop/Lumitio/software/pycom-esp-idf/components/freertos/list.c:218
    vTaskDelete
    /home/luis/Desktop/Lumitio/software/pycom-esp-idf/components/freertos/tasks.c:5063
    TASK_Interrupts
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/util/mpirq.c:72
    freertos_entry
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/mpthreadport.c:140
    

    Update:
    Using ExpansionBoard v2.1A with Gpy, fw release v1.20.2.r4 from git with custom frozen files. I also have the same Guru meditation.

    Best regards,



  • Yes, as @robert-hh suggested, I'm very much looking for a reproduction, or a series of things I can do that make this coredump happen (like running this micropython code in combination with pressing button x). The functions calls listed in this coredump decode happen quite a lot all over the firmware, making me think it could be related to running out of memory, or a memory leak somewhere, but again, it depends on the location and order of your function calls.

    Also, quick side note, you mention in your previous post you're using a Gpy, but have the firmware.elf path going through WIPY, is that intentional? It is possible to run Wipy firmware on a Gpy, (disabling the usage of the LTE modem). If its not intentional. it might show us the wrong function calls here.

    Let me know



  • @serafimsaudade What @Gijs probably asked is, if you can tell what the code is doing at the moment of the core dump. The decoding of the core dump does not look strange. The code seems to deal with function arguments down in a series of function calls.



  • @Gijs I'm being developing for the last year a device can read multiple sensors. And also local webserver for configuration. The sofware is custom and save into frozen folder to be compile.
    So far I have install +- 100 devices. Normally they are install and forget. But now some clients want to interact with the device locally.
    And we start receiving some complains, about random reboots.

    After some testing we also see some random coredumps. We test with different fw releases.
    The felling we get is that the device do a coredump when is in heavy load, like accessing to websever over wifi_AP.

    Update:
    New coredump

    Guru Meditation Error: Core  1 panic'ed (StoreProhibited). Exception was unhandled.
    Core 1 register dump:
    PC      : 0x4010238c  PS      : 0x00060530  A0      : 0x8010273c  A1      : 0x3ffd8c40  
    A2      : 0x00000004  A3      : 0x3ffd8c60  A4      : 0x3f9c8000  A5      : 0x3f4736d0  
    A6      : 0x3f450d0d  A7      : 0x00000f02  A8      : 0x00000000  A9      : 0x3ffd8c20  
    A10     : 0x3f9c7ff0  A11     : 0x00000005  A12     : 0x00000000  A13     : 0x3f40c13c  
    A14     : 0x3f40c13c  A15     : 0x3f40ad00  SAR     : 0x00000004  EXCCAUSE: 0x0000001d  
    EXCVADDR: 0x00000004  LBEG    : 0x400f66fc  LEND    : 0x400f6727  LCOUNT  : 0x0001f80d  
    
    ELF file SHA256: 0000000000000000000000000000000000000000000000000000000000000000
    
    Backtrace: 0x4010238c:0x3ffd8c40 0x40102739:0x3ffd8c60 0x40104a72:0x3ffd8ca0 0x400fce69:0x3ffd8cc0 0x4010833d:0x3ffd8ce0 0x40100710:0x3ffd8d80 0x400fce69:0x3ffd8df0 0x400fceed:0x3ffd8e10 0x401083c3:0x3ffd8e30 0x40100710:0x3ffd8ed0 0x400fce69:0x3ffd8f00 0x400fceed:0x3ffd8f20 0x401083c3:0x3ffd8f40 0x40100710:0x3ffd8fe0 0x400fce69:0x3ffd9010 0x400fceed:0x3ffd9030 0x401083c3:0x3ffd9050 0x40100710:0x3ffd90f0 0x400fce69:0x3ffd9120 0x400fceed:0x3ffd9140 0x401083c3:0x3ffd9160 0x40100710:0x3ffd9200 0x400fce69:0x3ffd9230 0x400fce96:0x3ffd9250 0x400e0f1a:0x3ffd9270 0x400e11c1:0x3ffd9310 0x400dfe98:0x3ffd9340
    

    Decode:

    xtensa-esp32-elf-addr2line -fe build/WIPY/release/application.elf 0x4010238c:0x3ffd8c40 0x40102739:0x3ffd8c60 0x40104a72:0x3ffd8ca0 0x400fce69:0x3ffd8cc0 0x4010833d:0x3ffd8ce0 0x40100710:0x3ffd8d80 0x400fce69:0x3ffd8df0 0x400fceed:0x3ffd8e10 0x401083c3:0x3ffd8e30 0x40100710:0x3ffd8ed0 0x400fce69:0x3ffd8f00 0x400fceed:0x3ffd8f20 0x401083c3:0x3ffd8f40 0x40100710:0x3ffd8fe0 0x400fce69:0x3ffd9010 0x400fceed:0x3ffd9030 0x401083c3:0x3ffd9050 0x40100710:0x3ffd90f0 0x400fce69:0x3ffd9120 0x400fceed:0x3ffd9140 0x401083c3:0x3ffd9160 0x40100710:0x3ffd9200 0x400fce69:0x3ffd9230 0x400fce96:0x3ffd9250 0x400e0f1a:0x3ffd9270 0x400e11c1:0x3ffd9310 0x400dfe98:0x3ffd9340
    mp_obj_new_str_from_vstr
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objstr.c:2082
    mp_obj_str_make_new
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objstr.c:2082
    type_call
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objtype.c:997
    mp_call_function_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:624
    mp_execute_bytecode
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/vm.c:919
    fun_bc_call
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objfun.c:287
    mp_call_function_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:624
    mp_call_method_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:640
    mp_execute_bytecode
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/vm.c:1002
    fun_bc_call
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objfun.c:287
    mp_call_function_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:624
    mp_call_method_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:640
    mp_execute_bytecode
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/vm.c:1002
    fun_bc_call
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objfun.c:287
    mp_call_function_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:624
    mp_call_method_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:640
    mp_execute_bytecode
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/vm.c:1002
    fun_bc_call
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objfun.c:287
    mp_call_function_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:624
    mp_call_method_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:640
    mp_execute_bytecode
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/vm.c:1002
    fun_bc_call
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/objfun.c:287
    mp_call_function_n_kw
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:624
    mp_call_function_0
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../py/runtime.c:598
    parse_compile_execute
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../lib/utils/pyexec.c:601
    pyexec_frozen_module
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/../lib/utils/pyexec.c:588
    TASK_Micropython
    /home/luis/Desktop/Lumitio/software/pycom-micropython-sigfox/esp32/mptask.c:328
    

    Best Regards,



  • Nice work decoding the coredump! Can you also describe what you are trying to achieve, and where it breaks? That would really help us to find the cause of the coredump!
    Gijs



Pycom on Twitter