Deepsleep current only low(er) before first wake-up?

[KenM]

I noticed that my deepsleep current is only very low the first time it goes to sleep, after that it's up to like 100-200µA.
I do have pull ups on P9, P10, P21, P22 (I2C)
and P20 is disconnected from the Expansion Board

My guess is some pulled up pins are changing behavior after the first deepsleep wake up, although my code is doing exactly the same.

Lopy4 + Expansion Board V3.1

Any advice/thoughts?