Support for 'internal' dual boot?

[mokrafoka]

Hi,

I played around with a possibility to (OTA)-flash new app image without extra serial flash memory-chip requirement.
The idea is to have a device with at least twice the flash as the app image and write a new image into some upper flash address. The boot loader would detect it and, if possible, use it to re-flash the 'old' app.

The necessary urboot changes for it are available here: https://github.com/mokrafoka/urboot/tree/dual_internal
Please keep in mind that this is a proof of concept only and shall not be used in any production environment. Especially as support for >64kB ROM devices isn't available yet.

At least for me it works very well. In my application code I use the urbootPageWrite() function to write my new app image which in turn gets activated after reboot (eg. wdt trigger).

However it works only if the app-image does fit into (flash_size - bootloader_size) / 2!
This requirement must be fulfilled in the users app code.

To build a boot loader with this 'internal' dual boot support something like this could be used:

make MCU=atmega328p F_CPU=8000000L NAME=myInternalDualBootloaderStuff \
  AUTOBAUD=1 BLINK=1 LED=AtmelPB0 LEDPOLARITY=1 EEPROM=0 URPROTOCOL=1 VBL=1 DUAL=1 DUAL_INTERN=1

Especially if DUAL_INTERN is enabled DUAL must be enabled as well.

Please let me know what do you think about it and if you would be interested to take this code over to urboot.
I could provide a simple example how to read new app (over serial for simplicity) and use it to re-flash.

-Milosz

[stefanrueger]

I think I understand what the idea is and how it's implemented.

I wonder whether a similar thing can be achieved outside the bootloader and for applications > flash/2. Say you have 2 kB (out of 128 kB) free in top flash below a 1k bootloader. The OTA mechanism makes the existing 125 k application load a staging process of 2 kB into [125 kB, 127 kB] just below the bootloader (using the pagewrite() function exported by the bootloader). A further OTA command makes the 125 kB application jump to the staging process which overwrites the vectors to point to itself as application (just in case an interrupt happens). Then the staging process is free to receive further OTA packets with the full shining new 125 kB (or less) application. Once all is there, verified and double checked the staging process overwrites the vectors to point to the new application (the last packet should be the vector table). TLDR; as the urboot bootloader exports a page write function to the application, the application can play bootloader.

BTW, I have just created a PR with a release candidate for v8.0. Have a look how you like it, and see whether you can break it.

[mokrafoka]

Sure,
that would be really cool.

However some issues would have to be solved. OTA is probably in most cases really over the air, and such communication devices tend to require you to work with interrupts (data ready) and/or specific timings (timed reply to specific requests or CSMA/CA). All that depends on the communication device in question and might get from tricky to very hard to implement if for example interrupts should be replaced by polling and timings by delays. A rfm12 radio device has a 2-Btye fifo, good luck trying to poll it ;-)

However to simplify things the small 'intermediate downloader' (the staging process) could be written into the flash together with the main image, but into the upper memory area. The, lets say, last 2-3kB would be unusable anyway and some extra code could be saved in the main app. The interrupt table would have to be adapted multiple times, though.

A possible procedure could look like this:

1. large main app overwrites the SPM_RDY vector to point to the staging process and triggers a wdt/reset
2. urboot starts, buts since there is nothing to flash it jumps into SPM_RDY which in turn starts the 'staging process'
3. Assuming that this staging process relies on interrupts for communication, the interrupt table would have to be tweaked again: probably timer interrupt and maybe some TWI or pin-change interrupts would have to be populated with new addresses that point into the staging process memory. This is something the main app can't do.
4. Now that all necessary interrupts to point into the staging process context, the image download and flash process can start.
5. Then, after all the write, check, verify and what-ever QA is done and dandy, adapt the SPM_RDY to point to the fresh written image, adapt vector 0 to point to urboot.
6. wdt-reset, jump to urboot, then jump to fresh app image

But, well there is a pitfall in between 4 and 5. In the moment the new app image is written the staging process would overwrite the interrupt vector table and loose potentially critical interrupts. Yeah, the new interrupt table could be written elsewhere and the main app would copy it every time it starts (before first sei). Well, no, eh...

I may be completely wrong, but I think this alternative dual_boot approach can work only if the OTA downloader (staging process) is capable to act without interrupts. And yes I'm sure in most cases it will be possible, but in most cases would be hard to implement as well.

-Milosz

[stefanrueger]

Good point about the interrupts for the OTA comms. I had not thought about that. But you have! So, my suggestion is to farm off some interrupt routines and other functions needed for OTA into a high flash section below the bootloader. These can still be used by the main application for all sorts of other radio-ing tasks it may have, not? And would reduce the size of the main application as it's farmed out. Think of them as a persistent library of function/interrupt calls that both the main application and the staging OTA loader can use and that always are available in high flash and stay there once compiled for the first time. This requires a bit of a .section ju-jitsu when laying out the application. The advantage of the separation of OTA staging and the application code is that the application only needs to be able to switch to the OTA staging when told so (by reassigning the bootloader vector and WDT timing out). Hopefully the staging process is a well-debugged independent process that lives unperturbed by all the flashing under the bootloader and protects itself from overwriting does all due diligence checks, error correction etc. The disadvantage of this method is that the normal application process stops during OTA upload. Cannot be done on a life-support machine in a hospital, eg, that would not be able to be switched off for uploading a new FW version.