improvements to esp32 build configuration

[UncleGrumpy]

These changes include several enhancements to the ESP32 build configuration and deployment of custom and test builds. The language flavor is determined by the offset of the main.avm partition in the partition table, simplifying Elixir supported builds and making the configuration of the image less error prone. The configuration of an Elixir build may be done without altering git tracked files by using idf.py -DAVM_ELIXIR_SUPPORTT=on set-target ${CHIP}. This is the equivalent of setting the custom partition file name to partitions-elixir.csv using idf.py menuconfig. Other existing AtomVM configuration options have been added to main/Kconfig.projbuild and are exposed in idf.py menuconfig under the AtomVM menu to make the configuration process more natural for users already familiar with ESP-IDF and its build and configuration system.

The auto discovered flavor of the esp32boot (boot.avm partition) libraries are now included in the idf.py flash task. This makes use of the build/mkimage.sh script only necessary for building release images in the CI or to create a complete image for future use, and makes the experience like most other ESP-IDF based projects. To flash only the factory (AtomVM) partition idf.py app-flash may be used or idf.py bootloader-flash app-flash if changes made affect the bootloader partition as well.

Changes the build configuration for reproducible builds, this strips leading path information (which is replaced by placeholders), and allows locally building from a release tag and generating a binary with the same sha256 checksum as official release binaries (as long as ESP-IDF and toolchain versions match those used to build the release). See: https://docs.espressif.com/projects/esp-idf/en/v5.5/esp32/api-guides/reproducible-builds.html

The nvs partition may now be generated and included in the build/mkimage.sh script (or flashed manually using the idf.py nvs-flash command) if the file nvs_partition.csv is present in the top level esp32 platform directory. An example is provided and nvs_partition.csv is added to .gitignore to prevent accidental pushes of sensitive passwords.

Closes atomvm/exatomvm#34

These changes are made under both the "Apache 2.0" and the "GNU Lesser General Public License 2.1 or later" license terms (dual license).

SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later

[petermm]

Not sure I like this bundling of commands:

1. Imho it's a coupling, implicit behaviour change - which might hinder future things, like point 2..

2. In a simulator flow I use: idf.py build && touch ~/path_to_project/atomvm_wokwi_bins/flasher_args.json (as to build and then reboot sim) - and having mkimage in there would slow things down.

But I'm also a poweruser of idf.py build && ./build/mkimage.sh - so I get the drift, but not sure I understand how it simplifies deploying custom images..

[UncleGrumpy]

@petermm, I don’t want to disrupt anyone’s workflows. My goal was just to streamline the process for newer users who need to build a custom image to include additional components, like atomvm_mqtt. It makes more sense for me to add an additional cmake job called “deploy” (or similar) that will build the complete image and flash it to the target device using the generated ./build/flasjimage.sh script, leaving the default “build” task unchanged, this will better my goal and should have no affect on any pre-existing workflows.

[pguyot]

Should we keep a way to not create the image? (it's not much longer anyway). When working on the VM, I usually only reflash with idf.py flash which is faster than flashing the whole image.

Also, one of my workflows (la machine, obviously) does not include esp32boot.avm at all because it seems to me that esp32boot.avm doesn't bring anything to production images. I'll definitely work around this improvement for newcomers, though.

Likewise, qemu-based tests only use artifacts created by idf.py build (the VM, the partition map and the bootloader) and not the full image.

[UncleGrumpy]

Should we keep a way to not create the image?

I converted this back to draft, because I realize after @petermm’s comments that could be intrusive on existing workflows, so instead I will create and document a new build task that can be used as an alternative to idf.py build that will create the ready to deploy image, or even flash the image after building so that flashing a new custom image can be achieved with one command.

[pguyot]

Should we keep a way to not create the image?

I converted this back to draft, because I realize after @petermm’s comments that could be intrusive on existing workflows, so instead I will create and document a new build task that can be used as an alternative to idf.py build that will create the ready to deploy image, or even flash the image after building so that flashing a new custom image can be achieved with one command.

This sounds like a better approach indeed and would be very welcome. I also find it frustrating to have to call the bash script to build the image when I need it.