A NEW TOPICCCCCCCC "maximum recursion depth exceeded"

[bitrat]

I know this has been asked elsewhere, but the answer I require hasn't appeared.

My code, which uses threads but has no (intentional) recursive calls is throwing "maximum recursion depth exceeded" exceptions.

Using this test :

def recursion_check(count):
    try:
        return recursion_check(count+1)
    except:
        return count

I can see that without threads, the recursion depth is 49 (or more likely 50).

When run inside a thread, the above code returns a paltry 10, regardless of the number of threads or the stack size, which I've tried resetting with _thread.stack_size(16 * 1024) (my board is an ESP32 with additional 64Mbit PSRAM).

I think this exception is being thrown because a call stack limit is being hit, and has nothing to do with recursion. My code paths are predictable , with few differences between ones that succeed and ones that don't.

On this basis, I'd like to increase the 'call stack limit, which would seem to be hard coded somewhere in the MP source. I've searched in the usual places, but come up empty handed.

Could someone please point the way to this mod?

Without too many suggestions along the lines of converting function calls to loops or not using threads, thanks. :-)

[robert-hh]

@bitrat SInce you raised that question both on this Forum and on the micropython.org forum, would you mind to tell, which version of the firmware you are using?
Note: If I compare the code of Pycom and MicroPython.org, I cannot tell where in the micropython.org the memory for the stack is allocated. It may be implicitly done in the esp-idf.

[robert-hh]

@robert-hh Reading further down the code and the espressif pages, I'm a little bit confused: The stack size is set in the code as stack len, means stack_size/sizeof(stack_element), which is typically a 32 bit sized. The espressif documentation however required the stack size to be given in bytes. See: https://docs.espressif.com/projects/esp ... ertos.html. But the telling in the doc is not clear. While for the paramter ulStackDepth, it tells: "The size of the task stack specified as the number of bytes. ", for the parameter pxStackBuffer it tells: *"Must point to a StackType_t array that has at least ulStackDepth indexes ", which indicates that the stack size is to be given in stack elements.

Also comparing the two ports:*

• in the micropython.org port, th->stack is set to the end of the stack, as returned by a call to pxTaskGetStackStart()
• in the Pycom port, th->stack is set to the lowest address of the stack, as allocated before.

Since the stack grows down, only one of both can be right, unless the entry is used for different purposes in both ports.

[bitrat]

@robert-hh Hi Robert,

I'm using the Pycom port on LoPy and WiPy. I'm not too familiar with the C internals of either interpreter, or how they differ in this area. I've found these constants in py/mpconfig.h that hint at a link between the python stack and the C stack

UPDATE

Missed the obvious!

mptask.h

#define MICROPY_TASK_STACK_SIZE                 (8 * 1024)
#define MICROPY_TASK_STACK_SIZE_PSRAM           (16 * 1024)

mptask.c

    if (chip_rev > 0) {
        stack_len = (MICROPY_TASK_STACK_SIZE_PSRAM / sizeof(StackType_t));
    } else {
        stack_len = (MICROPY_TASK_STACK_SIZE / sizeof(StackType_t));
    }

Just for testing, I hard coded #define MICROPY_TASK_STACK_SIZE (16 * 1024) but (to my surprise) I stall had to set the larger stack size in my thread with _thread.stack_size(16 * 1024)

Anyway, it seems that solved the problem! Thankyou.

Cheers,
bitrat

[jcaron]

@bitrat Haven't looked at any of that, but I wouldn't be surprised if there were two distinct stacks: the ESP32 stack, and the micropython stack (it would seem difficult for an interpreted language to use the CPU stack for storage of the language stack).

It would expect exceeding the ESP32 stack would result in a panic/core dump rather than just an Exception. No idea which stack any of the above controls nor whether the two are in any way linked.

[robert-hh]

@bitrat The code lines you cite are the size of the MicroPython main task, which is started at boot time. This stack size cannot be changed without recompiling the code. The size you set with _thread.stack_size(16 * 1024) will be effective for all additional threads, which you start in your python scripts.

[bitrat]

@robert-hh I have recompiled the interpreter with #define MICROPY_TASK_STACK_SIZE (16 * 1024) in mptask.h and that's fixed the problem, by increasing stack_len in the line stack_len = (MICROPY_TASK_STACK_SIZE_PSRAM / sizeof(StackType_t)); in mptask.c

Strangely, I still had to set the stack size in micropython with _thread.stack_size(16 * 1024).

Also unexpected was that my recursion check routine still returned the same values, so perhaps there actually is an interpreter recursion limit, distinct from the interpreter call stack limit.

Ultimately tho, the change prevented the exception being thrown in my code, so yay.