TlmPacketizer in v3.1.1

[VR314]

I'm using fprime v3.1.1 and building a deployment that incorporates TlmPacketizer for telemetry packetization.

In the only examples I can find online, they define a {Deployment}Packets.xml file that gets autocoded, and the autocoded file is used in the topology build-up to set the packet list of the TlmPacketizer. However, it looks like that autocoding capability was only included in v3.2.0. How was this done in v3.1.1 and earlier?

For added context, it looks like the Ref/ application on v3.1.1 has a RefPacketsAi.xml but it isn't included in the build anywhere.

[thomas-bc]

However, it looks like that autocoding capability was only included in v3.2.0

Can you clarify what you mean there? Packets.xml doesn’t get autocoded, it has to be hand-written by projects, since they need to specify the packet spec. fprime-util new —deployment does include a baseline packets.xml to make is easier to get started, but that’s the extent of it I believe.

*Packets.xml indeed is required for when using TlmPacketizer. Ref and all the tutorials/references I know about use TlmChan, so technically this Packets.xml file is not required. The reason we keep it in sync is so that projects can swap TlmChan for TlmPacketizer in the instances.fpp and things should work out of the box.

[VR314]

From this portion of the RefTopology.cpp, it seems like the packets file gets autocoded into a RefPacketsAc.hpp file, whereas an equivalent file does not get autocoded in v3.1.1
https://github.com/nasa/fprime/blob/c02f35145edca1746f55fc39219319d0e8419f96/Ref/Top/RefTopology.cpp#L13C1-L13C36

The TlmPacketizer is unusable in v3.1.1 because the following line requires objects that are defined in the RefPacketsAc.hpp file, which does not get generated before v3.2 (from my understanding). TlmPacketizer does not function without that line included in the topology.
https://github.com/nasa/fprime/blob/c02f35145edca1746f55fc39219319d0e8419f96/Ref/Top/RefTopology.cpp#L121C1-L121C66

[LeStarch]

You are right the telemetry packet autocoder was integrated after we delivered the TlmPacketizer component. The autocoded files are required for the telemtry packetizer to work correctly. You have several options to consider on how to resolve this.

1. Upgrade F´. Not always possible, but something to consider.
2. Generate these files by hand. Acquire the script from F´ devel denoted here https://github.com/nasa/fprime/blob/a9989ceca6ba26807814230c119fd1cdef7fce30/cmake/autocoder/packets.cmake#L10C1-L10C97 and run it to generate these files whenever you need to update the packets.xml
3. Integrate a new autocoder: In your project, create cmake/autocoder, copy in the packets.cmake from the above devel link and the script it uses. Adjust the script path and add the line:

   fprime/cmake/FPrime.cmake

   Line 114 in a9989ce

   register_fprime_build_autocoder(autocoder/packets)

   to your top-level cmake. Now you've registered a project autocoder that is a backport of the supported one in devel

[VR314]

Option 3 is most realistic for our project. I have two questions:

1. I placed the python script in cmake/autocoder folder as well, so what should I set the script path to? I was thinking "./tlm_packet_gen.py" since they are in the same directory, but I am not 100% sure if relative paths play nice with the CMake toolchain, or if I need to qualify it relative to CMake environment variables.
2. I added the register_fprime_build_autocoder call to the deployment CMakeLists.txt (attached). After running fprime-util purge, fprime-util generate yields the following error:

CMake Error at CMakeLists.txt:76 (register_fprime_build_autocoder):
  Unknown CMake command "register_fprime_build_autocoder".


CMake Error at /home/rvaroon/fprime-devel/visors-software/fprime/cmake/prescan.cmake:91 (message):
-- Configuring incomplete, errors occurred!
  Failed to run prescan, cannot continue.
Call Stack (most recent call first):
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/FPrime.cmake:68 (perform_prescan)
  CMakeLists.txt:46 (include)


[ERROR] CMake erred with return code 1. Partial build cache remains. Run purge to clean-up.

How should I proceed?
CMakeLists.txt

[EDIT] Looks like the register_fprime_build_autocoder function was only introduced in v3.2.0: a946f1d

Let me know if you think there still a way forward with option 3, but if not we can just use option 2.

[LeStarch]

Right....build autocoders is a more recent thing. I would try the following next:

1. Run the autocoder in the Top module directly
2. Add the resulting source files to your Top module

Add this to your Top module after register_fprime_module.

#Step 1:
run_ac_set("${CMAKE_CURRENT_LIST_DIR}/RefPackets.xml" autocoder/packets)
#Step 2:
target_sources(${PROJECT_NAME}_Top PUBLIC "${AC_GENERATED}")

Yes you should offset paths using CMake variables as ./ is not reliable.

Bear with us, much of what we are discussing here was fixed and made easier in newer F´ versions.

[VR314]

I am trying to implement this in the Ref application now.

I fixed the path in packet.cmake to be set(PACKETS_AUTOCODER_SCRIPT "${CMAKE_CURRENT_LIST_DIR}/tlm_packet_gen.py").
I tried to run the autocoder (I assume you mean the python file) from the Ref/Top directory. Running python3 ../cmake/autocoder/tlm_packet_gen.py RefPacketsAi.xml yields:

Processing packet file /home/rvaroon/fprime-devel/visors-software/fprime/Ref/Top/RefPacketsAi.xml
Traceback (most recent call last):
  File "../cmake/autocoder/tlm_packet_gen.py", line 619, in <module>
    main()
  File "../cmake/autocoder/tlm_packet_gen.py", line 603, in main
    set_build_roots(os.environ.get("BUILD_ROOT"))
  File "/home/rvaroon/fprime-devel/python-fprime-extras/src/fprime_ac/utils/buildroot.py", line 21, in set_build_roots
    lambda path: os.path.normpath(os.path.realpath(path)), build_root_arg.split(":")
AttributeError: 'NoneType' object has no attribute 'split'

This seems to mean that the autocoder cannot find the build root.
I ran this both after purging my build cache, and after re-building fully. Both times gave the same error.

I also put cmake/autocoders in the Ref/ folder. I added the two lines you included to the end of the Top/CMakeLists.txt file as specified and the output of fprime-util generate is as follows:

CMake Error at /home/rvaroon/fprime-devel/visors-software/fprime/cmake/utilities.cmake:115 (include):
  include could not find requested file:

    autocoder/packets
Call Stack (most recent call first):
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:55 (plugin_include_helper)
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:31 (run_ac)
  Top/CMakeLists.txt:24 (run_ac_set)


CMake Error at /home/rvaroon/fprime-devel/visors-software/fprime/cmake/utilities.cmake:115 (include):
  include could not find requested file:

    autocoder/packets
Call Stack (most recent call first):
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:55 (plugin_include_helper)
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:31 (run_ac)
  Top/CMakeLists.txt:24 (run_ac_set)


CMake Error at /home/rvaroon/fprime-devel/visors-software/fprime/cmake/utilities.cmake:115 (include):
  include could not find requested file:

    autocoder/packets
Call Stack (most recent call first):
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:55 (plugin_include_helper)
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:31 (run_ac)
  Top/CMakeLists.txt:24 (run_ac_set)


CMake Error at /home/rvaroon/fprime-devel/visors-software/fprime/cmake/utilities.cmake:115 (include):
  include could not find requested file:

    autocoder/packets
Call Stack (most recent call first):
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:55 (plugin_include_helper)
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:31 (run_ac)
  Top/CMakeLists.txt:24 (run_ac_set)


CMake Error at /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:194 (packets_is_supported):
  Unknown CMake command "packets_is_supported".
Call Stack (most recent call first):
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:194 (cmake_language)
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:58 (_filter_sources)
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/autocoder/autocoder.cmake:31 (run_ac)
  Top/CMakeLists.txt:24 (run_ac_set)


CMake Error at /home/rvaroon/fprime-devel/visors-software/fprime/cmake/prescan.cmake:91 (message):
-- Performing CMake source prescan
-- Configuring incomplete, errors occurred!
  Failed to run prescan, cannot continue.
Call Stack (most recent call first):
  /home/rvaroon/fprime-devel/visors-software/fprime/cmake/FPrime.cmake:68 (perform_prescan)
  CMakeLists.txt:32 (include)


[ERROR] CMake erred with return code 1. Partial build cache remains. Run purge to clean-up.

Thanks for your support so far, I know it is not a trivial task to context-switch back multiple versions to 'hack-in' new features into old versions.

[LeStarch]

Try using a full path to the autocoder.cmake. Something like:

run_ac_set("${CMAKE_CURRENT_LIST_DIR}/RefPackets.xml" "${CMAKE_CURRENT_LIST_DIR}/../../cmake/autocoder/packets.cmake")

....or something similar.