CREO2URDF – Add action for compile and load the dll on releases

[Nicogene]

It would be nice to have a github action that on release compile on a self-hosted runner the dll and load it as artifact of the release.

cc @pattacini @mfussi66 @traversaro

[Nicogene]

Today I started this activity, but we need first to install visual studio on our self-hosted runner, it should be by any user (cc @davidelasagna) because when I try to install the dependencies I get this error:

 Could not locate a complete Visual Studio instance

Moreover, I was thinking about how to perform these steps:

• https://github.com/icub-tech-iit/creo2urdf?tab=readme-ov-file#dependencies

I don't know if it is ok every time to install vcpkg from scratch or if we can install it on the machine and simply use it updating the dependencies.

Or maybe we could try with conda binaries? Do they exist compiled static? cc @traversaro

[traversaro]

Today I started this activity, but we need first to install visual studio on our self-hosted runner, it should be by any user (cc @davidelasagna) because when I try to install the dependencies I get this error:

 Could not locate a complete Visual Studio instance

Moreover, I was thinking about how to perform these steps:

• https://github.com/icub-tech-iit/creo2urdf?tab=readme-ov-file#dependencies

I don't know if it is ok every time to install vcpkg from scratch or if we can install it on the machine and simply use it updating the dependencies.

Or maybe we could try with conda binaries? Do they exist compiled static? cc @traversaro

No, only a subset of conda packages offer static packages on Windows, and anyhow we do not use simple static libraries, but a specific set of options (static with /MD if I recall correctly) so we need to use a package manager that builds from source for sure.

In theory if we do not change the compiler vcpkg should have quite good caching support, so even if we reinstall everything from vcpkg at every step it should do it quite fast (except the first time).

Note that if it is easier, we can just install vs build tools instead of the full visual studio.

[traversaro]

An alternative is to use vcpkg export or simply zip the vcpkg folder and upload it somewhere, as we did for the robotology-superbuild.

[Nicogene]

In theory if we do not change the compiler vcpkg should have quite good caching support, so even if we reinstall everything from vcpkg at every step it should do it quite fast (except the first time).

In this sense do you adise to put the vcpkg in C:/users/Public instead of the GA workspace? Because I think that the workspace is cleaned up every run of the action

[pattacini]

In this sense do you adise to put the vcpkg in C:/users/Public instead of the GA workspace? Because I think that the workspace is cleaned up every run of the action

Yes, keep things separated. Even if the workspace is retained, we may need to reinstall the self-hosted runner from scratch, deleting the GA workspace entirely.

[traversaro]

In theory if we do not change the compiler vcpkg should have quite good caching support, so even if we reinstall everything from vcpkg at every step it should do it quite fast (except the first time).

In this sense do you adise to put the vcpkg in C:/users/Public instead of the GA workspace? Because I think that the workspace is cleaned up every run of the action

The cache location is independent from the vcpkg location, by default is outside of the GH workspace: https://learn.microsoft.com/en-us/vcpkg/consume/binary-caching-default?pivots=shell-cmd .

[Nicogene]

In theory if we do not change the compiler vcpkg should have quite good caching support, so even if we reinstall everything from vcpkg at every step it should do it quite fast (except the first time).

In this sense do you adise to put the vcpkg in C:/users/Public instead of the GA workspace? Because I think that the workspace is cleaned up every run of the action

The cache location is independent from the vcpkg location, by default is outside of the GH workspace: https://learn.microsoft.com/en-us/vcpkg/consume/binary-caching-default?pivots=shell-cmd .

The first run took 32 minutes to build the dependencies but now takes just 1 minute thanks to the cache, I am facing still some issues but cloning vcpkg from scratch seems the corret solution

[Nicogene]

Today I stumbled in making the GA using the cmake embedded in visual studio, I put its path in PATH, in fact I can run it from powershell, but unfortunately I cannot see it inside the github action

Github workspace

 PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\MATLAB\MATLAB Runtime\R2023b\runtime\win64;C:\Program Files\MATLAB\R2017a\runtime\win64;C:\Program Files\MATLAB\R2017a\bin;C:\Program Files\Git\cmd;C:\Program Files\dotnet\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\ngenesio\AppData\Local\Microsoft\WindowsApps;C:\Users\ngenesio\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\ngenesio\.dotnet\tools

Powershell

PATH=C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\MATLAB\MATLAB Runtime\R2023b\runtime\win64;C:\Program Files\MATLAB\R2017a\runtime\win64;C:\Program Files\MATLAB\R2017a\bin;C:\Program Files\Git\cmd;C:\Program Files\dotnet\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\ngenesio\AppData\Local\Microsoft\WindowsApps;C:\Users\ngenesio\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\ngenesio\.dotnet\tools

I have to understand the github workspace is missing one piece of the PATH

[Nicogene]

Bingo! I had to close the powershell where run_cmd.ps1 (GA daemon) was running and open a new one

[Nicogene]

I finally managed to build creo2urdf via GA:

The workstation was missing the otk and protoolkit installation, since we were using the dll without issues it is a confirmation that compiling static prevent the user to have to install all teh dependencies.

I am getting strange linking warnings, that I am not sure I can ignore, but I think to have seen them before @mfussi66

LINK : warning LNK4286: symbol '??1xstring@@QEAA@XZ (public: __cdecl xstring::~xstring(void))' defined in 'otk_cpp_md.lib(strlibsrc.obj)' is imported by 'otk_cpp_md.lib(featcr2.obj)' [C:\Users\Public\actions-runner\_work\creo2urdf-private\creo2urdf-private\creo2urdf\build\src\creo2urdf\creo2urdf.vcxproj]
LINK : warning LNK4286: symbol '?CastToCString@xstring@@AEBAPEBDXZ (private: char const * __cdecl xstring::CastToCString(void)const )' defined in 'otk_cpp_md.lib(strlibsrc.obj)' is imported by 'otk_cpp_md.lib(helppfc.obj)' [C:\Users\Public\actions-runner\_work\creo2urdf-private\creo2urdf-private\creo2urdf\build\src\creo2urdf\creo2urdf.vcxproj]
LINK : warning LNK4286: symbol '??0xstring@@QEAA@PEBD@Z (public: __cdecl xstring::xstring(char const *))' defined in 'otk_cpp_md.lib(strlibsrc.obj)' is imported by 'otk_no222_md.lib(wfc_tk.obj)' [C:\Users\Public\actions-runner\_work\creo2urdf-private\creo2urdf-private\creo2urdf\build\src\creo2urdf\creo2urdf.vcxproj]
LINK : warning LNK4286: symbol '??0xstring@@QEAA@PEBD@Z (public: __cdecl xstring::xstring(char const *))' defined in 'otk_cpp_md.lib(strlibsrc.obj)' is imported by 'otk_cpp_md.lib(pfccmnsrc.obj)' [C:\Users\Public\actions-runner\_work\creo2urdf-private\creo2urdf-private\creo2urdf\build\src\creo2urdf\creo2urdf.vcxproj]

And more similar, all related to the xstring class

[traversaro]

I am getting strange linking warnings, that I am not sure I can ignore, but I think to have seen them before @mfussi66

From https://learn.microsoft.com/en-us/cpp/error-messages/tool-errors/linker-tools-warning-lnk4286?view=msvc-170 it seems that the problem is in Creo code.

[traversaro]

To avoid unexpected failure in the CI, I suggest to fix the vcpkg version to a given commit, for example:

git clone --depth 1 -b 9224b3bbd8df24999d85720b1d005dd6f969ade0 https://github.com/microsoft/vcpkg 

Otherwise, any run on the CI will potentially use a different version of vcpkg, resulting in unexpected failures. Ideally we may want to store the vcpkg version used inside the creo2urdf so that also users that want to compile creo2urdf use the same version. This can be done out of the box when using the vcpkg manifest mode with the builtin-baseline parameter (see https://learn.microsoft.com/en-us/vcpkg/concepts/manifest-mode), but just to start I think hardcoding the commit to use in the CI script is probably enough.

[Nicogene]

I struggled a little bit because the catch was that I was creating a release in a different repository than the one where the action is hosted, but here it is the first automatic release of creo2urdf:

• https://github.com/icub-tech-iit/creo2urdf/releases/tag/v0.5.0

For now it is run via workflow-dispatch and the new tag to be created has to be specified

I checked that the .zip contains all we need for running creo2urdf and that the dll works fine.

This is not perfect, I will open an issue for possible future improvements for both this action and the one that generates the urdf

cc @mfussi66 @pattacini @traversaro

[Nicogene]

Closing in favour of:

• CREO2URDF – Improve the automatic release action #93

[traversaro]

Great!