[Rust cargo-first builds] Build the GCC toolchains for Windows with --with-gnu-ld to avoid too-long-command-line linking errors (IDFGH-6187)

[ivmarkov]

For various reasons which are unavoidable, the Rust builds driven via cargo do generate a very long linker command line, which goes above the 32767 character limit on Windows.

There is an escape hatch we are taking advantage of, which is to supply (most of) the link command-line arguments to the xtensa-esp32-elf-gcc.exe executable in a file. E.g. @F:\\... ...\\linker_args.txt

Unfortunately, when xtensa-esp32-elf-gcc.exe is itself calling the LD executable, it does expand the command line arguments supplied in the @ file, and thus we have the too-long-command-line problem again, but this time further down the process chain.

It seems the above file expansion does not happen, as long as the GCC toolchain for windows is compiled with the --with-gnu-ld flag enabled.

Therefore, we would like to request the GCC toolchains for Windows to always be build with --with-gnu-ld flag enabled.

[ivmarkov]

@N3xed, @iggr, @georgik ^^^

[antmak]

@ivmarkov There is a build with using --with-gnu-ld https://dl.espressif.com/dl/toolchains/preview/withgnuld/xtensa-esp32-elf-gcc8_4_0-esp-2021r1-7-gec835ae-win64.zip . Could you try it if helps, please?

[ivmarkov]

@ivmarkov There is a build with using --with-gnu-ld https://dl.espressif.com/dl/toolchains/preview/withgnuld/xtensa-esp32-elf-gcc8_4_0-esp-2021r1-7-gec835ae-win64.zip . Could you try it if helps, please?

As mentioned here I've checked and it indeed does work.

[antmak]

Checked, thanks! We will add this fix in the next toolchain release for Windows

[ivmarkov]

@antmak Any ETA when the next toolchain release for Windows is expected?
By the way, we need this fix for all of the toolchains (that is esp32, esp32c3, esp32s2, esp32s3, ..., ulp).

[TD-er]

Yep would really like to have this as my project now needs to generate temporary cpp files and exclude the other .cpp files to make it compile in Windows.
This is often quite a frustration as I keep then editing those temp files (I know after N times I should know better, with N probably over 100).
Also I just hope this does not cause other issues too...

So looking forward to this fix.

[georgik]

@antmak Please, can you share details about the solution. AFAIK there is an update in tools.json. Espressif-bot just closed the issue without details.

[georgik]

@JurajSadel - can you confirm the solution by results of your tests?

[ivmarkov]

@antmak @georgik @JurajSadel I assume we "just" need to do a export ESP_IDF_VERSION=master; cargo build --features native build of the demo or the esp-idf-template on Windows and confirm that it does link successfully.

The problem is that since one week or so we cannot build against ESP-IDF master (4.4 and 4.3.X work fine): https://github.com/esp-rs/esp-idf-template/actions/runs/1681595816

@N3xed ^^^

[igrr]

@ivmarkov That's right, if you pull latest master branch of IDF there should be a new version of gcc for Windows installable as usual (tools.json / idf_tools.py / Windows installer / etc...). This version is built with --with-gnu-ld flag.
We are backporting this update to release/v4.4 as well.

[antmak]

Thanks igrr.

Also you can cherry-pick commit d836163 into your IDF work directory or branch. Or download archives with -patch2 suffix from the commit manually

JFI, gcc shows its built options:

C:\> xtensa-esp32-elf-gcc -v
Using built-in specs.
COLLECT_GCC=C:\Users\mak\.espressif\tools\xtensa-esp32-elf\esp-2021r2-patch2-8.4.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-gcc.exe
COLLECT_LTO_WRAPPER=c:/users/mak/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch2-8.4.0/xtensa-esp32-elf/bin/../libexec/gcc/xtensa-esp32-elf/8.4.0/lto-wrapper.exe
Target: xtensa-esp32-elf
Configured with: /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/configure --build=x86_64-build_pc-linux-gnu --host=x86_64-host_w64-mingw32 --target=xtensa-esp32-elf --prefix=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf --with-local-prefix=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf --with-headers=/builds/idf/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/include --with-newlib --enable-threads=no --disable-shared --with-pkgversion='crosstool-NG esp-2021r2-patch2' --disable-__cxa_atexit --enable-cxx-flags=-ffunction-sections --disable-libgomp --disable-libmudflap --disable-libmpx --disable-libssp --disable-libquadmath --disable-libquadmath-support --with-gmp=/builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/buildtools/complibs-host --with-mpfr=/builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/buildtools/complibs-host --with-mpc=/builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/buildtools/complibs-host --with-isl=/builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/buildtools/complibs-host --enable-lto --enable-target-optspace --without-long-double-128 --disable-nls --enable-multiarch --enable-languages=c,c++ --disable-libstdcxx-verbose --enable-threads=posix --enable-gcov-custom-rtio --enable-libstdcxx-time=yes --with-gnu-ld
Thread model: posix
gcc version 8.4.0 (crosstool-NG esp-2021r2-patch2) 

There is --with-gnu-ld opt now

[Jason2866]

@antmak @igrr Can you upload the new toolchains to the Platformio registry?
Or are the already there? If yes can you tell us the version to use it in platformio?
Thx!

[antmak]

@Jason2866 There were some issues with uploading the update to Platformio due to its limitation. Will be updated soon