Enable SDK for ROS2 Rolling #1203

sgstreet · 2024-08-12T22:35:24Z

This PR tries to resurrect PR #988 for Rolling to enable OpenEmbedded SDK cross-compilation. These changes allow demos, examples and examples_interfaces packages to be built using the resulting SDK. To build the SDK use:

$ bitbake ros2-image-sdktest -c do_populate_sdk

Install the resulting SDK, setup a ROS2 workspace containing demos, examples and example_interfaces and build the workspace using the follow script:

export ROS_VERSION=2
export ROS_PYTHON_VERSION=3
export ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET
export ROS_DISTRO=rolling
export PYTHONPATH=$OECORE_NATIVE_SYSROOT/usr/lib/python3.12/site-packages:$OECORE_TARGET_SYSROOT/opt/ros/rolling/lib/python3.12/site-packages
export AMENT_PREFIX_PATH=$OECORE_TARGET_SYSROOT/opt/ros/rolling
usr/share:$OECORE_TARGET_SYSROOT/opt/ros/rolling/share:$OECORE_TARGET_SYSROOT/opt/ros/rolling/lib/cmake"

colcon build \
--merge-install \
--install-base $PWD/install/arm64 \
--cmake-args \
" -DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_FILE" \
" -DCMAKE_STAGING_PREFIX=$PWD/install/arm64" \
" -DBUILD_TESTING=OFF"

Install the $PWD/install/arm64 on the target device. Setup the device workspace:

$ . /opt/ros/rolling/setup.bash
$ . ./arm64/local_setup.bash

Run any of the rclcpp examples to demonstrate the valid cross-compilation.

sgstreet · 2024-08-12T22:39:40Z

The python nodes are not functional:

root@raspberrypi5:~# ros2 run demo_nodes_py talker
Traceback (most recent call last):
  File "/opt/ros/rolling/bin/ros2", line 33, in <module>
    sys.exit(load_entry_point('ros2cli==0.33.0', 'console_scripts', 'ros2')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/ros2cli/cli.py", line 91, in main
    rc = extension.main(parser=parser, args=args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/ros2run/command/run.py", line 70, in main
    return run_executable(path=path, argv=args.argv, prefix=prefix)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/ros2run/api/__init__.py", line 64, in run_executable
    process = subprocess.Popen(cmd)
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.12/subprocess.py", line 1955, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/root/arm64/lib/demo_nodes_py/talker'

The file /home/root/arm64/lib/demo_nodes_py/talker exists and contains the expected python code. Does someone have an idea on how to debug this?

jiaxshi · 2024-08-19T07:58:58Z

Hi @sgstreet
Greate job! I have done similar work on humble, I suggest to add TOOLCHAIN_HOST_TASK in a include file

+require conf/distro/include/ros2-sdk.inc

Could you please full build https://github.com/ros2/examples or https://github.com/ros2/demos with SDK env?

sgstreet · 2024-08-19T18:06:53Z

Could you please full build https://github.com/ros2/examples or https://github.com/ros2/demos with SDK env?

Yes. I can build demos, example_interfaces and example ros2 packages using the SDK env. The C++ nodes are functional,unfortunately the python packages are not, see above.

jiaxshi · 2024-08-22T08:02:32Z

The python nodes are not functional:

root@raspberrypi5:~# ros2 run demo_nodes_py talker
Traceback (most recent call last):
  File "/opt/ros/rolling/bin/ros2", line 33, in <module>
    sys.exit(load_entry_point('ros2cli==0.33.0', 'console_scripts', 'ros2')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/ros2cli/cli.py", line 91, in main
    rc = extension.main(parser=parser, args=args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/ros2run/command/run.py", line 70, in main
    return run_executable(path=path, argv=args.argv, prefix=prefix)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/ros2run/api/__init__.py", line 64, in run_executable
    process = subprocess.Popen(cmd)
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.12/subprocess.py", line 1955, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/root/arm64/lib/demo_nodes_py/talker'

The file /home/root/arm64/lib/demo_nodes_py/talker exists and contains the expected python code. Does someone have an idea on how to debug this?

Hi @sgstreet
You can check the python shebang in python executor. HOST path is included like:

#!/local/mnt2/workspace/install_dir/sysroots/x86_64-qcomsdk-linux/usr/bin/python3

jiaxshi · 2024-08-26T07:13:58Z

meta-ros-common/classes/ros_cmake.bbclass

@@ -13,3 +13,8 @@ EXTRA_OECMAKE:prepend:class-native = "\
    -DCMAKE_PREFIX_PATH='${STAGING_DIR_NATIVE}${ros_prefix};${STAGING_DIR_NATIVE}${prefix}' \
    -DCMAKE_INSTALL_PREFIX:PATH='${ros_prefix}' \
 "
+
+EXTRA_OECMAKE:prepend:class-nativesdk = "\
+    -DCMAKE_PREFIX_PATH='${STAGING_DIR_NATIVE}/opt/ros/${ROS_DISTRO};${STAGING_DIR_NATIVE}${ros_prefix};${STAGING_DIR_NATIVE}${prefix}' \


Use ${ros_prefix} to replace opt/ros/${ROS_DISTRO}

Thanks! I will test and feedback. Sorry for the delay, my attention has been distracted.

This change still allows the SDK to build and run, but I suspect it is not used in my case.

jiaxshi · 2024-08-26T07:19:34Z

meta-ros2/classes/ros_ament_cmake.bbclass

@@ -25,11 +25,13 @@ export AMENT_PREFIX_PATH="${STAGING_DIR_HOST}${prefix};${STAGING_DIR_NATIVE}${pr

 inherit cmake python3native

+PYTHONPATH:prepend:class-nativesdk = "${STAGING_DIR_NATIVE}/opt/ros/${ROS_DISTRO}/lib/python3.12/site-packages:"


Use ${ros_prefix} to replace opt/ros/${ROS_DISTRO}

Use ${PYTHON_BASEVERSION} instead of python3.12 is better.

Thanks! I will test and feedback. Sorry for the delay, my attention has been distracted.

Using 1. fails to build the SDK with the following error for the packages rosidl-typesupport-interface, rosidl-cmake, rcutils and rosidl-generator-type-description

| DEBUG: Python function extend_recipe_sysroot finished | DEBUG: Executing shell function do_configure | -- Found ament_cmake: 2.6.0 (/home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake/cmake) | -- Found Python3: /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/usr/bin/python3-native/python3 (found version "3.12.3") found components: Interpreter | Traceback (most recent call last): | File "/home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/package_templates/templates_2_cmake.py", line 21, in <module> | from ament_package.templates import get_environment_hook_template_path | ModuleNotFoundError: No module named 'ament_package' | CMake Error at /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/ament_cmake_package_templates-extras.cmake:41 (message): | | execute_process(/home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/usr/bin/python3-native/python3 | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/package_templates/templates_2_cmake.py | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/build/ament_cmake_package_templates/templates.cmake) | returned error code 1 | Call Stack (most recent call first): | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/ament_cmake_coreConfig.cmake:41 (include) | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake/cmake/ament_cmake_export_dependencies-extras.cmake:15 (find_package) | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-cmake/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake/cmake/ament_cmakeConfig.cmake:41 (include) | CMakeLists.txt:5 (find_package)

As a corollary, remove ${STAGING_DIR_NATIVE}/opt/ros/${ROS_DISTRO} from EXTRA_OECMAKE:prepend:class-nativesdk -D-DCMAKE_PREFIX_PATH line results in similar errors:

| CMake Error at CMakeLists.txt:5 (find_package): | By not providing "Findament_cmake.cmake" in CMAKE_MODULE_PATH this project | has asked CMake to find a package configuration file provided by | "ament_cmake", but CMake did not find one. | | Could not find a package configuration file provided by "ament_cmake" with | any of the following names: | | ament_cmakeConfig.cmake | ament_cmake-config.cmake | | Add the installation prefix of "ament_cmake" to CMAKE_PREFIX_PATH or set | "ament_cmake_DIR" to a directory containing one of the above files. If | "ament_cmake" provides a separate development package or SDK, be sure it | has been installed.

for rosidl/rosidl-typesupport-interface and rosidl-cmake while the following error generated for rcutils:

| CMake Error at CMakeLists.txt:16 (find_package): | By not providing "Findament_cmake_python.cmake" in CMAKE_MODULE_PATH this | project has asked CMake to find a package configuration file provided by | "ament_cmake_python", but CMake did not find one. | | Could not find a package configuration file provided by | "ament_cmake_python" with any of the following names: | | ament_cmake_pythonConfig.cmake | ament_cmake_python-config.cmake | | Add the installation prefix of "ament_cmake_python" to CMAKE_PREFIX_PATH or | set "ament_cmake_python_DIR" to a directory containing one of the above | files. If "ament_cmake_python" provides a separate development package or | SDK, be sure it has been installed.

The appear to be a problem with the ${ros_prefix} value.

Using 2. generates the variation of the following error for packages rosidl-typesupport-interface, rosidl-cmake, rcutils, rosidl-parser and rosidl-generator-type-description:

| DEBUG: Python function extend_recipe_sysroot finished | DEBUG: Executing shell function do_configure | -- Found ament_cmake: 2.6.0 (/home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake/cmake) | -- Found Python3: /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/usr/bin/python3-native/python3 (found version "3.12.3") found components: Interpreter | Traceback (most recent call last): | File "/home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/package_templates/templates_2_cmake.py", line 21, in <module> | from ament_package.templates import get_environment_hook_template_path | ModuleNotFoundError: No module named 'ament_package' | CMake Error at /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/ament_cmake_package_templates-extras.cmake:41 (message): | | execute_process(/home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/usr/bin/python3-native/python3 | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/package_templates/templates_2_cmake.py | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/build/ament_cmake_package_templates/templates.cmake) | returned error code 1 | Call Stack (most recent call first): | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake_core/cmake/ament_cmake_coreConfig.cmake:41 (include) | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake/cmake/ament_cmake_export_dependencies-extras.cmake:15 (find_package) | /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling/share/ament_cmake/cmake/ament_cmakeConfig.cmake:41 (include) | CMakeLists.txt:5 (find_package)

Investigating value of '${ros_prefix}' for the native-sdk

$ bitbake-getvar -r nativesdk-rosidl-typesupport-interface --value ros_prefix /usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-oesdk-linux/opt/ros/rolling

while for the target:

$ bitbake-getvar -r rosidl-typesupport-interface --value ros_prefix /opt/ros/rolling

These values are prefixed with STAGING_DIR_NATIVE

bitbake-getvar -r rosidl-typesupport-interface --value STAGING_DIR_NATIVE /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/cortexa76-oe-linux/rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native

The actual value being used is /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-oesdk-linux/opt/ros/rolling which is not present in the build path for the package

ls -al /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native//usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-oesdk-linux/opt/ros/rolling ls: cannot access '/home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native//usr/local/oe-sdk-hardcoded-buildpath/sysroots/x86_64-oesdk-linux/opt/ros/rolling': No such file or directory

while the path /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/cortexa76-oe-linux/rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native//opt/ros/rolling is present:

$ ls -al /home/stephen/Workspace/yocto/pi5/build/tmp-glibc/work/cortexa76-oe-linux/rosidl-typesupport-interface/4.7.0-1/recipe-sysroot-native/opt/ros/rolling total 16 drwxr-xr-x 4 stephen stephen 4096 Aug 27 09:46 . drwxr-xr-x 3 stephen stephen 4096 Aug 27 09:46 .. drwxr-xr-x 3 stephen stephen 4096 Aug 27 09:46 lib drwxr-xr-x 20 stephen stephen 4096 Aug 27 09:46 share

My interpretation is that '${ros_prefix}' is incorrect for the building the nativesdk recipes.

Search meta-ros for ros_prefix yields the following definition in meta-ros-common/classes/ros_opt_prefix.bbclass:

ros_prefix ?= "${base_prefix}/opt/ros/${ROS_DISTRO}"

I did not find (could be wrong here) any other prior definitions of ros_prefix so base_prefix must be the governing variable. I found base_prefix in openembedded-core/meta/classes-recipe/nativesdk.bbclass:

base_prefix = "${SDKPATHNATIVE}"

Tracking SDKPATHNATIVE to openembedded-core/meta/conf/bitbake.conf the definition is

SDK_SYS = "${SDK_ARCH}${SDK_VENDOR}-${SDK_OS}" SDKPATH = "/usr/local/oe-sdk-hardcoded-buildpath" SDKPATHNATIVE = "${SDKPATH}/sysroots/${SDK_SYS}"

Which starts to make sense in explaining the observed behavior. I think that the current value of ros_prefix is wrong for building the SDK. Unfortunately I'm not sure the correct way to fix this would be?

Thanks for your details. I didn't realize the prefix_xx are changed in nativesdk.bbclass.

base_prefix = "${SDKPATHNATIVE}" prefix = "${SDKPATHNATIVE}${prefix_nativesdk}"

My intention is to use the variables defined in ros_opt_prefix.bbclass.

That seems right, may be we need to add an variable with it's default value as /opt/ros/${ROS_DISTRO}. This would allow the some flexibility when constructing paths for the nativesdk.

sgstreet · 2024-08-27T22:31:22Z

@jiaxshi Still hacking on the shebang path.

jiaxshi · 2024-08-28T02:10:45Z

@jiaxshi Still hacking on the shebang path.

python executor can be found after changing shebang path. And one more thing, if you build python-library, you should add PYTHON_SOABI to rename library which is named with "x86".

-DPYTHON_SOABI=cpython-310-aarch64-linux-gnu

You can refer to #1204 for details.

sgstreet · 2024-08-28T15:28:53Z

@jiaxshi here is the contents of my talker executor (are these correct words?)

#!/home/stephen/Workspace/ros2-dev/sdk/sysroots/x86_64-oesdk-linux/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'demo-nodes-py==0.34.2','console_scripts','talker'
import re
import sys

# for compatibility with easy_install; see #2198
__requires__ = 'demo-nodes-py==0.34.2'

try:
    from importlib.metadata import distribution
except ImportError:
    try:
        from importlib_metadata import distribution
    except ImportError:
        from pkg_resources import load_entry_point


def importlib_load_entry_point(spec, group, name):
    dist_name, _, _ = spec.partition('==')
    matches = (
        entry_point
        for entry_point in distribution(dist_name).entry_points
        if entry_point.group == group and entry_point.name == name
    )
    return next(matches).load()


globals().setdefault('load_entry_point', importlib_load_entry_point)


if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(load_entry_point('demo-nodes-py==0.34.2', 'console_scripts', 'talker')())

As you pointed out the shebang is clearly wrong. Can you point out the code which generates this talker file? Maybe I can figure out which path is wrong.

sgstreet · 2024-08-28T15:33:50Z

python executor can be found after changing shebang path. And one more thing, if you build python-library, you should add PYTHON_SOABI to rename library which is named with "x86".
-DPYTHON_SOABI=cpython-310-aarch64-linux-gnu
You can refer to #1204 for details.

I tried this, how should I verify the impact?

Further, for none python ROS2 packages the following warnings are generated:

CMake Warning:
  Manually-specified variables were not used by the project:

    PYTHON_SOABI

which is harmless but annoying. I think we need another place to add the PYTHON_SOABI.

jiaxshi · 2024-08-29T01:58:48Z

I think the shebang path is added by native python interpreter.

CMake Warning:
  Manually-specified variables were not used by the project:

    PYTHON_SOABI

The warning is because is not used by none-python packages. If there's way to add the default value to native interpreter is better.

sgstreet · 2024-08-29T02:36:20Z

I think the shebang path is added by native python interpreter.
Are you saying the native python3 is adding the shebang? I would have thought either the python setup or some ament_python make script was doing this but I'm not sure where to look. Maybe I'm confused? I have limit python skills.
CMake Warning:

  Manually-specified variables were not used by the project:



    PYTHON_SOABI
The warning is because is not used by none-python packages. If there's way to add the default value to native interpreter is better.

Or maybe superflores could add during recipe generation.

sgstreet · 2024-10-02T22:45:41Z

@jiaxshi here is the contents of my talker executor (are these correct words?)

#!/home/stephen/Workspace/ros2-dev/sdk/sysroots/x86_64-oesdk-linux/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'demo-nodes-py==0.34.2','console_scripts','talker'
import re
import sys

# for compatibility with easy_install; see #2198
__requires__ = 'demo-nodes-py==0.34.2'

try:
    from importlib.metadata import distribution
except ImportError:
    try:
        from importlib_metadata import distribution
    except ImportError:
        from pkg_resources import load_entry_point


def importlib_load_entry_point(spec, group, name):
    dist_name, _, _ = spec.partition('==')
    matches = (
        entry_point
        for entry_point in distribution(dist_name).entry_points
        if entry_point.group == group and entry_point.name == name
    )
    return next(matches).load()


globals().setdefault('load_entry_point', importlib_load_entry_point)


if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(load_entry_point('demo-nodes-py==0.34.2', 'console_scripts', 'talker')())

As you pointed out the shebang is clearly wrong. Can you point out the code which generates this talker file? Maybe I can figure out which path is wrong.

I patched colcon build to generate a better/correct shebang for python packages builds. See 5965ca6

Signed-off-by: Stephen Street <[email protected]>

…ions Signed-off-by: Stephen Street <[email protected]>

Signed-off-by: Stephen Street <[email protected]>

jiaxshi · 2024-10-08T02:07:22Z

@jiaxshi here is the contents of my talker executor (are these correct words?)

#!/home/stephen/Workspace/ros2-dev/sdk/sysroots/x86_64-oesdk-linux/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'demo-nodes-py==0.34.2','console_scripts','talker'
import re
import sys

# for compatibility with easy_install; see #2198
__requires__ = 'demo-nodes-py==0.34.2'

try:
    from importlib.metadata import distribution
except ImportError:
    try:
        from importlib_metadata import distribution
    except ImportError:
        from pkg_resources import load_entry_point


def importlib_load_entry_point(spec, group, name):
    dist_name, _, _ = spec.partition('==')
    matches = (
        entry_point
        for entry_point in distribution(dist_name).entry_points
        if entry_point.group == group and entry_point.name == name
    )
    return next(matches).load()


globals().setdefault('load_entry_point', importlib_load_entry_point)


if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(load_entry_point('demo-nodes-py==0.34.2', 'console_scripts', 'talker')())

As you pointed out the shebang is clearly wrong. Can you point out the code which generates this talker file? Maybe I can figure out which path is wrong.

I patched colcon build to generate a better/correct shebang for python packages builds. See 5965ca6

Hi @sgstreet
Is the code based on your 5965ca6 commit? If so, it's not correct.
As you see,

#!/home/stephen/Workspace/ros2-dev/sdk/sysroots/x86_64-oesdk-linux/usr/bin/python3

Talker executor is running on target(e.g. RPI-4 board) device, the path python3 executor is /usr/bin/python3 not /home/stephen/Workspace/ros2-dev/sdk/sysroots/x86_64-oesdk-linux/usr/bin/python3 which is your HOST python3 executor of SDK.
The right output is

#!/usr/bin/python3

without HOST path prefix(/home/stephen/Workspace/ros2-dev/sdk/sysroots/x86_64-oesdk-linux) included.

sgstreet · 2024-10-08T15:26:44Z

@jiaxshi Sorry for the confusing and incomplete post. Building python nodes with 5965ca results in the following talker code:

#!/usr/bin/env python3
# EASY-INSTALL-ENTRY-SCRIPT: 'demo-nodes-py==0.34.2','console_scripts','talker'
import re
import sys

# for compatibility with easy_install; see #2198
__requires__ = 'demo-nodes-py==0.34.2'

try:
    from importlib.metadata import distribution
except ImportError:
    try:
        from importlib_metadata import distribution
    except ImportError:
        from pkg_resources import load_entry_point


def importlib_load_entry_point(spec, group, name):
    dist_name, _, _ = spec.partition('==')
    matches = (
        entry_point
        for entry_point in distribution(dist_name).entry_points
        if entry_point.group == group and entry_point.name == name
    )
    return next(matches).load()


globals().setdefault('load_entry_point', importlib_load_entry_point)


if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(load_entry_point('demo-nodes-py==0.34.2', 'console_scripts', 'talker')())

Which has the correct SHEBANG line.

sgstreet · 2024-10-08T22:15:27Z

Joy, cpp and python code rolling [email protected] and [email protected] are functional against origin/scarthgap-next using oe scarthgap release on a raspberry pi 5.

sgstreet · 2024-10-08T22:23:59Z

@robwoolley @jiaxshi This is almost ready to go, what are the next steps for getting this pull request merged? Jazzy, Humble? What about Kirkstone, master, Scarthgap? What addition changes are needed?

jiaxshi · 2024-10-09T07:13:41Z

Joy, cpp and python code rolling [email protected] and [email protected] are functional against origin/scarthgap-next using oe scarthgap release on a raspberry pi 5.

That's cool.

To merge the PR, my opinion is that we should test on other ROS2 distros (Humble, Jazzy, Iron, Rolling) and then propagate to the other branch (Kirkstone).

And one more topic is image recipe: meta-ros-common/recipes-core/images/ros2-image-sdktest.bb . How to generate SDK based on customized image recipe with platform dependency included?
For example, if i want to build a board/platform(raspberry Pi, RB3 gen2, Nvidia) specific image recipe and generate SDK with both ROS and platform support(e.g. GPU SDK) included.
@robwoolley Do you have any thoughts on this topic? If the scope of SDK is platform independent, we can ignore this.

sgstreet · 2024-10-09T17:56:06Z

To merge the PR, my opinion is that we should test on other ROS2 distros (Humble, Jazzy, Iron, Rolling) and then propagate to the other branch (Kirkstone).

I will work on this over the next couple of days, I suspect that Humble and Jazzy will need some specific patches for fastrtp and tinyxml. I was planning to ignore Iron as it will EOL in November. @robwoolley Thoughts?

sgstreet marked this pull request as draft August 12, 2024 22:35

jiaxshi reviewed Aug 26, 2024

View reviewed changes

sgstreet added 6 commits October 3, 2024 09:38

Add native sdk class extentions

c7aeb40

Signed-off-by: Stephen Street <[email protected]>

Patch fastrtps to use find TinyXML2

59c45e0

Signed-off-by: Stephen Street <[email protected]>

Ensure EXTRA_OECMAKE overrides correctly handle multiple cmake -D opt…

75fe390

…ions Signed-off-by: Stephen Street <[email protected]>

Add ros2 sdktest

9cd1246

Signed-off-by: Stephen Street <[email protected]>

Use yocto PYTHON_BASEVERSION variable to build PYTHONPATH

2734272

Signed-off-by: Stephen Street <[email protected]>

Make colcon python package builds use correct shebang for merge installs

6c93749

Signed-off-by: Stephen Street <[email protected]>

sgstreet force-pushed the ros2-rolling-sdk-enablement branch from 5965ca6 to 6c93749 Compare October 8, 2024 22:10

sgstreet mentioned this pull request Oct 24, 2024

Enable SDK for ROS2 on Scarthgap #1241

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable SDK for ROS2 Rolling #1203

Enable SDK for ROS2 Rolling #1203

sgstreet commented Aug 12, 2024 •

edited

Loading

sgstreet commented Aug 12, 2024 •

edited

Loading

jiaxshi commented Aug 19, 2024 •

edited

Loading

sgstreet commented Aug 19, 2024

jiaxshi commented Aug 22, 2024 •

edited

Loading

jiaxshi Aug 26, 2024

sgstreet Aug 27, 2024

sgstreet Aug 27, 2024

jiaxshi Aug 26, 2024

sgstreet Aug 27, 2024

sgstreet Aug 27, 2024

sgstreet Aug 27, 2024

sgstreet Aug 27, 2024

sgstreet Aug 27, 2024

sgstreet Aug 27, 2024 •

edited

Loading

sgstreet Aug 27, 2024 •

edited

Loading

jiaxshi Aug 28, 2024

sgstreet Aug 28, 2024 •

edited

Loading

sgstreet commented Aug 27, 2024

jiaxshi commented Aug 28, 2024

sgstreet commented Aug 28, 2024

sgstreet commented Aug 28, 2024 •

edited

Loading

jiaxshi commented Aug 29, 2024

sgstreet commented Aug 29, 2024

sgstreet commented Oct 2, 2024 •

edited

Loading

jiaxshi commented Oct 8, 2024

sgstreet commented Oct 8, 2024

sgstreet commented Oct 8, 2024 •

edited

Loading

sgstreet commented Oct 8, 2024 •

edited

Loading

jiaxshi commented Oct 9, 2024

sgstreet commented Oct 9, 2024 •

edited

Loading

		@@ -25,11 +25,13 @@ export AMENT_PREFIX_PATH="${STAGING_DIR_HOST}${prefix};${STAGING_DIR_NATIVE}${pr

		inherit cmake python3native

		PYTHONPATH:prepend:class-nativesdk = "${STAGING_DIR_NATIVE}/opt/ros/${ROS_DISTRO}/lib/python3.12/site-packages:"

Enable SDK for ROS2 Rolling #1203

Are you sure you want to change the base?

Enable SDK for ROS2 Rolling #1203

Conversation

sgstreet commented Aug 12, 2024 • edited Loading

sgstreet commented Aug 12, 2024 • edited Loading

jiaxshi commented Aug 19, 2024 • edited Loading

sgstreet commented Aug 19, 2024

jiaxshi commented Aug 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sgstreet Aug 27, 2024 • edited Loading

Choose a reason for hiding this comment

sgstreet Aug 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sgstreet Aug 28, 2024 • edited Loading

Choose a reason for hiding this comment

sgstreet commented Aug 27, 2024

jiaxshi commented Aug 28, 2024

sgstreet commented Aug 28, 2024

sgstreet commented Aug 28, 2024 • edited Loading

jiaxshi commented Aug 29, 2024

sgstreet commented Aug 29, 2024

sgstreet commented Oct 2, 2024 • edited Loading

jiaxshi commented Oct 8, 2024

sgstreet commented Oct 8, 2024

sgstreet commented Oct 8, 2024 • edited Loading

sgstreet commented Oct 8, 2024 • edited Loading

jiaxshi commented Oct 9, 2024

sgstreet commented Oct 9, 2024 • edited Loading

sgstreet commented Aug 12, 2024 •

edited

Loading

sgstreet commented Aug 12, 2024 •

edited

Loading

jiaxshi commented Aug 19, 2024 •

edited

Loading

jiaxshi commented Aug 22, 2024 •

edited

Loading

sgstreet Aug 27, 2024 •

edited

Loading

sgstreet Aug 27, 2024 •

edited

Loading

sgstreet Aug 28, 2024 •

edited

Loading

sgstreet commented Aug 28, 2024 •

edited

Loading

sgstreet commented Oct 2, 2024 •

edited

Loading

sgstreet commented Oct 8, 2024 •

edited

Loading

sgstreet commented Oct 8, 2024 •

edited

Loading

sgstreet commented Oct 9, 2024 •

edited

Loading