@Wolli01 These batteries are only good for medium to low power consumption. They are rated at ~1500 mAh, but only at 100Ohm load = 30mA. My first impression is, that these batteries are too weak. Anyhow, Better go for batteries with a higher voltage like this one: https://www.conrad.de/de/p/saft-ls-14500-spezial-batterie-mignon-aa-lithium-3-6-v-2600-mah-1-st-651153.html, or use three AA sized cells with 1.5 V each.
If you scan through the forum you'll find a lot of discussion about power consumption and power budget. The power consumption of the ESP32 is pretty high by itself, and the WiPy need comparable long to start, consuming high power during that period. Other Pycom devices need more power and longer start-up time.
The baseline Arduino UNO with its 8 bit Atmel processor at 8 MHz takes ~5 mA at full speed, compared to ~30mA for the ES32 w/o WiFi and up to 250mA peak/100mAaverage with wifi, @160 MHz clock speed.