How to solve the firmware update failures

[daniel]

@LoneTech thanks for the feedback. I've added an extra step to indicate that the cable jumper must be removed before running the firmware updater again.

Thank you! Providing this patch on a weekend is a step above and beyond. :)

Your are welcome! Our first priority right now is to give the best possible support and complete the firmware as soon as we can. Our tech team is working almost 24/7 with these goals in mind.

[Batilan]

Hi Daniel,

Thanks a lot indeed!

I'm trying to run the tool in windows but I get:

C:\Users\Corné\Downloads\flash_debug_tool\flash_debug_tool>python flash_debug.py COM3 write 0x0 2
Connecting...
Traceback (most recent call last):
 File "flash_debug.py", line 183, in <module>
esp_dev = esptool.ESP32ROM(port,115200)
File "C:\Users\CornÚ\Downloads\flash_debug_tool\flash_debug_tool\esptool.py", line 167, in __init__
self.connect()
File "C:\Users\CornÚ\Downloads\flash_debug_tool\flash_debug_tool\esptool.py", line 299, in connect
raise FatalError('Failed to connect to %s: %s' % (self.CHIP_NAME, last_exception))
esptool.FatalError: Failed to connect to ESP32: Invalid head of packet ('\x08')

This is different from choosing an invalid serial port: that results in:

C:\Users\Corné\Downloads\flash_debug_tool\flash_debug_tool>python flash_debug.py COM2 write 0x0 2
Traceback (most recent call last):
  File "flash_debug.py", line 183, in <module>
    esp_dev = esptool.ESP32ROM(port,115200)
  File "C:\Users\CornÚ\Downloads\flash_debug_tool\flash_debug_tool\esptool.py", line 159, in __init__
    self._port = serial.Serial(port)
  File "C:\Python273\lib\site-packages\serial\serialwin32.py", line 31, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Python273\lib\site-packages\serial\serialutil.py", line 236, in __init__
    self.open()
  File "C:\Python273\lib\site-packages\serial\serialwin32.py", line 62, in open
    raise SerialException("could not open port {!r}: {!r}".format(self.portstr, ctypes.WinError()))
serial.serialutil.SerialException: could not open port 'COM2': WindowsError(2, 'Het systeem kan het opgegeven bestand niet vinden.')

I did close down PyMakr first.

Guess I'll switch to my Linux Box now :-)

[daniel]

@Batilan,

Did you place the cable jumper between GND and P2 and reset the board before running the script?

[Batilan]

@Batilan

Ok to myself: this happens when you forget the reset step (or push the right button on the wrong LoPy as I did :-S)
Discovered it immediately after my post, but had to wait 10 minutes because of "bad reputation" ;-)
That allowed Daniel to show off his great customer service skills :-)

Now it looks nicer:

C:\Users\Corné\Downloads\flash_debug_tool\flash_debug_tool>python flash_debug.py COM3 write 0x0 2
Connecting...
value: 0x0
length: 2 
write 0x60002008 0x00000000 0x00400000 
write 0x60002010 0x00000000 0x00000000 
write 0x60002000 0x08000000 0xffffffff
write 0x60002008 0x00000000 0x00400000
write 0x60002010 0x00000000 0x00000000
write 0x60002000 0x08000000 0xffffffff
write 0x60002008 0x00000000 0x00400000
write 0x60002010 0x00000000 0x00000000
write 0x60002000 0x08000000 0xffffffff
write 0x60002008 0x00400000 0x00400000
write 0x60002010 0x00000000 0x0000ffff
write 0x60002000 0x04000000 0xffffffff
write 0x60002008 0x00000000 0x00400000
write 0x60002010 0x00000000 0x00000000
write 0x60002000 0x08000000 0xffffffff
get status: 0x0
ESP DEV DISCONNECT...

Maybe it helps someone else :-)

[on4bhm]

running python 3.6.0a3 on Win10
I get this error:

C:\Users\guyro\Downloads\flash_debug_tool\flash_debug_tool>python flash_debug.py COM3 write 0x0 2
File "flash_debug.py", line 29
print"ESP DEV DISCONNECT..."
^
SyntaxError: invalid syntax

C:\Users\guyro\Downloads\flash_debug_tool\flash_debug_tool>python

what can I do?
even when I place a extra space between print" it will not work

kind regards

[Batilan]

Probably use Python version 2 (like 2.7)

[on4bhm]

I thought: the latest is always the greatest?

[on4bhm]

I removed my python3 and installed 2.7
now I get this:

C:\Users\guyro\Downloads\flash_debug_tool\flash_debug_tool>c:\Python27\python.exe flash_debug.py COM3 write 0x0 2
Traceback (most recent call last):
File "flash_debug.py", line 1, in <module>
import esptool
File "C:\Users\guyro\Downloads\flash_debug_tool\flash_debug_tool\esptool.py", line 25, in <module>
import serial
ImportError: No module named serial

C:\Users\guyro\Downloads\flash_debug_tool\flash_debug_tool>

what is going on?

[Batilan]

Install the PySerial package as explained also here: https://learn.adafruit.com/arduino-lesson-17-email-sending-movement-detector/installing-python-and-pyserial Disregard the version difference with latest (in this case latest is greatest it seems :-)

[littlebat]

Can you open a github issue for this at esp-idf repo so we can know what it is and when it gets fixed?

[Batilan]

Actually that isn't the best example for Windows, but just download the module from https://pypi.python.org/pypi/pyserial unpack it using a program that is also capable of unpacking .tgz files and perform the setup by typing '

python setup.py install

In the pyserial-3.2.1 directory

If this helps please give me an upvote so next time you don't have to wait 10 minutes (as I don't have 3 reputation points yet (just registered) and have to wait 10 minutes before I can post my next reply :-S)

[on4bhm]

The tool seemed to work...

but the upgrade is still not working.
I have a dos box that is constant pinging to 192.168.4.1
I have a ping reply.
I do the procedure of going into safeboot
I always get 'The upgrade failed.please check that you are connected to the LoPy wireless LAN and try again.
but I can ping him !!
my wired network is disconnected.... !!

[on4bhm]

I took my virgin LoPy. and it worked right from the first try.
my previous LoPy had been upgraded before, a few day's ago.

Just for your information....
Can someone please check this?

[digsatman]

@daniel said in How to solve the firmware update failures:

python flash_debug.py /dev/ttyUSB0 write 0x0 2

Well I'm afraid that I stil can't update either of my LoPys using the updaters for Windows. OSX or Linux. All three return a "failure" message. One of them had indeed become "write only" so I rann the flash_debug.py utility and that sorted out the problem. Just to be doubly sure, I ran it on the other as well.

As the standalone utility does not give an info on its procgress, I tried using ftp (Filezilla on Windows 7 64) and it seems to upload in fits and starts, has disconnections and retries. Tried increasing timeouts and using the filezilla "throttle" all to no avail. My most recent attempt was to copy the new firmware onto my Android phone and try an ftp to the LoPy. This also failed to both LoPy's but seemed to consistently fail after around 480-486Kb.. Any comments/help/workarounds much appreciated!

[LoneTech]

It would appear both LoPys I have at home have a different problem updating. They can write to flash, and uploading appimg.bin takes 3.5s (readonly boards accept it in 1.7s), but they do not boot 0.9.1.b1; only 0.9.0.b2 (the one I've updated them with earlier, by default or safeboot latest) or 0.9.0b (factory image, also loaded with safeboot previous). They simply don't seem to accept the update.

[daniel]

I can release an updater tool that will re-flash everything, but the problem is that it will erase the LoRa MAC address. The way to avoid that is to read the MAC address as a first step, then do the flashing and program the MAC address back again, similar to the way it's done in the factory. This is the way it will be don in order to reprogram the bootloader once the big in the IDF that is locking the flash gets fixed.

We'll work on this solution and try to provide something ASAP.

[LoneTech]

I also took the liberty of backing my flash up with esptool. I found that it contains four copies of the firmware; 8785822 , f5444c7, a78d0fa and a78d0fa. So it has clearly been written to flash, it just doesn't boot on that. Of these locations the 4th has an oddball address. The booting image resides at 0x110000, the non-booting update at 0x210000, and the surprising copy of the non-booting version appears to sit at 0x385200.

However, the firmware updates are NOT intact. Only the booting one is an exact match, suggesting the copy at 0x210000 in fact shouldn't have passed an integrity check. 224439 bytes in they differ.

>>> fwbad[0x36cb0:0x36cd0].encode('hex')
'981982120ab80002c8fca1d7ff8a8d8080b4a0ab1080aa20a909b21c3a3bc80c'
>>> fw2[0x36cb0:0x36cd0].encode('hex')
'981982120ab80982c8fca1d7ff8a8d8080b4a0ab1080aa20a909b21c3a3bc80c'

Clearly we have a stray bit cleared, suggesting perhaps the flash block wasn't erased properly. I might be able to just rewrite the chunk with esptool. I did try overwriting it directly with esptool, but that didn't make it start; however, it did make the firmware update work. Looks like there's a bug with erasing during OTA update (not sure how rarely it triggers). Anyhow, after wiping the second image with esptool I have now updated the first board. Time for the other.

[daniel]

@LoneTech thanks for the super detailed analysis and for sharing your findings. If erasing the block with esptool fixes it, it's definitely related to the IDF bug with regards to the Status Register 2 being written with odd values. See this:

https://github.com/espressif/esp-idf/commit/f988dc9587e9d39f78250c403f3ef85d03d6cd30

Especially the new filed added:

components/spi_flash/spi_flash_rom_patch.c

We are preparing a new updated that includes this fix and uploads the bootloader as well.

We are aiming to release that on Friday after exhaustive testing.

[Xykon]

I also looked through the 4MB image I made of the LoPy flash using esptool.py --chip esp32 --port COM24 --baud 115200 read_flash 0 4194304 lopy_firmware.bin

It seems the "recovery" firmware is at 0x10000, the previous version is at 0x110000 and the main firmware at 0x210000.

After flashing with esptool.py --chip esp32 --port COM24 --baud 115200 write_flash 0x210000 lopy_0.9.1.b1_868.bin -fs 4MB -z the latest version 0.9.1.b1 is starting as default.

I also flashed 0x10000 the same way and now my recovery image (when I leave the safeboot jumper on until the blue heartbeat LED starts) is now also at 0.9.1.b1.

My boot.py file and MAC address don't seem to be affected by this.

[LoneTech]

@daniel I think that bug causes full on readonly mode (I could be wrong). The data present in flash indicates that it did write, but didn't erase properly. On the second board, I did try esptool erase then OTA update; while it marked the image as updated, it caused a memory access error on boot. So I used esptool to write the firmware, and after that it booted the new revision. So it would seem, for the secondary flash position to update correctly on these particular boards, I need to get both the OTA done and write it with esptool. I haven't analyzed the bugfix you refer to to see if it would solve this, but I am certainly willing to try; I have backups of my boards now and could recover. Right now I'm happy I got them to boot the most recent release. :)