AsyncRobotState consumes too much CPU

[k-okada]

AsyncRobotState consumes too much CPU load.
The strange thing is that just stopping the call to get_robot_state_async() does not work, but stopping the call to AsyncPeriodcQuery.__init__() does.

--

stop calling AsyncPeriodcQuery.__init__()

stop calling get_robot_state_async()

Profiling results

cc: @sktometometo

[sktometometo]

Maybe not related. But there may be some polling problems of GRPC calling which has significant effect on CPU usages.

• Increased CPU consumption in gRPC version 1.47.0 grpc/grpc#30428
• High CPU usage is found  grpc/grpc#31082
• Constant 100% CPU load on armv7 architectures on 1.51.1 grpc/grpc#32044 )

[mqcmd196]

@k-okada @sktometometo

I develop some Spot demos in spot-core-io.

obinata@core-io:~/ros/rocker/spot_ws/src $ vcs export
repositories:
  jsk-ros-pkg/jsk_robot:
    type: git
    url: https://github.com/k-okada/jsk_robot.git
    version: spot_arm
  jsk_pr2eus:
    type: git
    url: https://github.com/jsk-ros-pkg/jsk_pr2eus.git
    version: master
  spot-ros:
    type: git
    url: [email protected]:heuristicus/spot_ros.git
    version: arm
obinata@core-io:~/ros/rocker/spot_ws/src $ vcs diff
...
=== ./jsk-ros-pkg/jsk_robot (git) ===
diff --git a/jsk_spot_robot/jsk_spot_startup/launch/include/driver.launch b/jsk_spot_robot/jsk_spot_startup/launch/include/driver.launch
index 6da02bab..8b2591d1 100644
--- a/jsk_spot_robot/jsk_spot_startup/launch/include/driver.launch
+++ b/jsk_spot_robot/jsk_spot_startup/launch/include/driver.launch
@@ -113,14 +113,14 @@


   <!-- japanese speech node -->
-  <include if="$(arg use_voice_text)" file="$(find voice_text)/launch/voice_text.launch">
-    <arg name="launch_sound_play" value="$(arg launch_sound_play)" />
-    <arg name="sound_play_respawn" value="true" />
-  </include>
-  <include unless="$(arg use_voice_text)" file="$(find aques_talk)/launch/aques_talk.launch">
-    <arg name="launch_sound_play" value="$(arg launch_sound_play)" />
-    <arg name="sound_play_respawn" value="true" />
-  </include>
+  <!-- <include if="$(arg use_voice_text)" file="$(find voice_text)/launch/voice_text.launch"> -->
+  <!--   <arg name="launch_sound_play" value="$(arg launch_sound_play)" /> -->
+  <!--   <arg name="sound_play_respawn" value="true" /> -->
+  <!-- </include> -->
+  <!-- <include unless="$(arg use_voice_text)" file="$(find aques_talk)/launch/aques_talk.launch"> -->
+  <!--   <arg name="launch_sound_play" value="$(arg launch_sound_play)" /> -->
+  <!--   <arg name="sound_play_respawn" value="true" /> -->
+  <!-- </include> -->

   <!-- Notifier/Visualization -->

@@ -177,8 +177,8 @@
   </include>

   <!-- gps -->
-  <include if="$(arg use_gps)" file="$(find ublox_gps)/launch/ublox_device.launch" >
-    <arg name="param_file_name" value="zed_f9p" />
-  </include>
+  <!-- <include if="$(arg use_gps)" file="$(find ublox_gps)/launch/ublox_device.launch" > -->
+  <!--   <arg name="param_file_name" value="zed_f9p" /> -->
+  <!-- </include> -->

 </launch>
diff --git a/jsk_spot_robot/jsk_spot_startup/launch/jsk_spot_bringup.launch b/jsk_spot_robot/jsk_spot_startup/launch/jsk_spot_bringup.launch
index 16234a44..aa99e07a 100644
--- a/jsk_spot_robot/jsk_spot_startup/launch/jsk_spot_bringup.launch
+++ b/jsk_spot_robot/jsk_spot_startup/launch/jsk_spot_bringup.launch
@@ -27,8 +27,8 @@
   <include file="$(find jsk_spot_startup)/launch/include/interaction.launch"
           if="$(arg launch_interaction)" />

-  <include file="$(find switchbot_ros)/launch/switchbot.launch">
-      <arg name="token" value="/var/lib/robot/credentials/switchbot_token.txt"/>
-  </include>
+  <!-- <include file="$(find switchbot_ros)/launch/switchbot.launch"> -->
+  <!--     <arg name="token" value="/var/lib/robot/credentials/switchbot_token.txt"/> -->
+  <!-- </include> -->

 </launch>
=== ./jsk_pr2eus (git) ===
=== ./spot-ros (git) ===

And build the docker container below

FROM ros:noetic-robot-focal

ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && apt install -y -qq python3-catkin-tools python3-pip
RUN rosdep update

# pre-install some ros packages commonly used to reduce build time
RUN apt install -y -qq ros-noetic-jsk-pr2eus ros-noetic-jsk-recognition ros-noetic-jsk-model-tools
RUN pip3 install protobuf==3.20 bosdyn-client==4.0.3 bosdyn-mission==4.0.3 bosdyn-api==4.0.3 bosdyn-core==4.0.3 bosdyn-choreography-client==4.0.3

COPY src src
RUN rosdep install -iqry --from-paths src || true

RUN rm -rf src

cd rocker/spot_ws
docker build . -t jsk-spot-core-io
rocker --nvidia --user --home --network host --host 133.11.216.108 --privileged jsk-spot-core-io:latest /bin/bash # exec shell in docker

In rocker container

source /opt/ros/noetic/setup.bash 
catkin build spoteus jsk_spot_startup
source ./devel/setup.bash
roslaunch jsk_spot_startup jsk_spot_bringup.launch credential_config:=/home/obinata/spot_credential.yaml 2>&1 | grep -v "TF_REPEATED_DATA" | grep -v " at line 278"

It seems that the topics stable.

In my laptop

❯ rostopic hz /spot/camera/hand_color/image
average rate: 7.081
        min: 0.117s max: 0.181s std dev: 0.02535s window: 6
average rate: 6.299
        min: 0.117s max: 0.230s std dev: 0.03438s window: 11
average rate: 6.006
        min: 0.117s max: 0.230s std dev: 0.03054s window: 17
average rate: 5.602
        min: 0.117s max: 0.284s std dev: 0.04091s window: 21
average rate: 5.413
        min: 0.117s max: 0.284s std dev: 0.03921s window: 26
average rate: 5.136
        min: 0.117s max: 0.319s std dev: 0.04744s window: 30

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────❯ rostopic hz /spot/status/battery_states
subscribed to [/spot/status/battery_states]
average rate: 17.481
        min: 0.025s max: 0.083s std dev: 0.01552s window: 15
average rate: 16.685
        min: 0.019s max: 0.114s std dev: 0.02195s window: 32
average rate: 16.835
        min: 0.019s max: 0.114s std dev: 0.02097s window: 48
average rate: 16.760
        min: 0.019s max: 0.114s std dev: 0.02133s window: 65