fix(userspace/libsinsp): multiple fixes related to rawargs. #2130

FedeDP · 2024-10-23T09:18:58Z

What type of PR is this?

/kind bug

Any specific area of the project related to this PR?

/area libsinsp

Does this PR require a change in the driver versions?

What this PR does / why we need it:

Firstly, properly refresh m_arginfo and m_customfield type and print format given current event while extracting rawarg values.
Secondly, properly support PT_FLAGS, PT_ENUMFLAGS, PT_UID and PT_GID types in rawval_to_json and rawval_to_string.
Lastly, honor PF_HEX print format for 8,16,32bits types.

Also, added a test.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

These issues were found by @Andreagit97 while working on the syscall enter dropping (#2068)

Does this PR introduce a user-facing change?:

fix(userspace/libsinsp): multiple fixes related to rawargs.

FedeDP · 2024-10-23T09:19:04Z

/milestone 0.19.0

FedeDP · 2024-10-23T09:20:03Z

userspace/libsinsp/sinsp_filtercheck.cpp

@@ -139,6 +139,8 @@ Json::Value sinsp_filter_check::rawval_to_json(uint8_t* rawval,

 	case PT_L4PROTO:  // This can be resolved in the future
 	case PT_UINT8:
+	case PT_FLAGS8:


We did not support FLAGS and ENUMFLAGS in rawval_to_{json,string}.

FedeDP · 2024-10-23T09:20:29Z

userspace/libsinsp/sinsp_filtercheck.cpp

 		if(print_format == PF_OCT) {
 			prfmt = (char*)"%" PRIo8;
 		} else if(print_format == PF_DEC || print_format == PF_ID) {
 			prfmt = (char*)"%" PRIu8;
 		} else if(print_format == PF_HEX) {
-			prfmt = (char*)"%" PRIu8;
+			prfmt = (char*)"%" PRIX8;


Fix print format for 8,16,32bits types for PF_HEX.

FedeDP · 2024-10-23T09:21:33Z

userspace/libsinsp/sinsp_filtercheck_event.cpp

@@ -1192,7 +1194,7 @@ uint8_t* sinsp_filter_check_event::extract_single(sinsp_evt* evt,
 		m_val.u16 = evt->get_cpuid();
 		RETURN_EXTRACT_VAR(m_val.u16);
 	case TYPE_ARGRAW:
-		return extract_argraw(evt, len, m_arginfo->name);
+		return extract_argraw(evt, len, m_argname.c_str());


Use m_argname here that is exactly the same as m_arginfo->name.

FedeDP · 2024-10-23T09:44:51Z

userspace/libsinsp/sinsp_filtercheck_event.cpp

+		// to allow `eval_filter` expressions to be properly compiled.
+		// Note: even if they'll be overridden,
+		// it is safe to guess that eg: `size` param will always be a numeric type.
+		// Of course if we had a `size` param that is string, it would be troublesome.
 		m_customfield.m_type = m_arginfo->type;


Per-comment: store the first one found to allow proper compilation of filters on this field; this works fine unless we have an event param, eg: flags, that in an event is int type, and in another is string type.
That's not the case right now.

FedeDP · 2024-10-23T09:45:10Z

userspace/libsinsp/sinsp_filtercheck_event.cpp


 	if(pi != NULL) {
+		m_arginfo = pi->get_info();
+		m_customfield.m_type = m_arginfo->type;


Store correct field type and print format from the being-extracted event.

FedeDP · 2024-10-23T09:46:14Z

/cc @jasondellaluce @Andreagit97 @LucaGuerra

github-actions · 2024-10-23T09:55:26Z

Perf diff from master - unit tests

     9.25%     -2.50%  [.] sinsp::next
     0.52%     +0.80%  [.] sinsp_evt::get_ts
    10.22%     -0.74%  [.] sinsp_parser::reset
     4.34%     +0.74%  [.] next
     5.16%     -0.73%  [.] sinsp_evt::get_type
     1.24%     -0.71%  [.] sinsp_evt::get_param
     1.55%     +0.60%  [.] sinsp::fetch_next_event
     0.93%     -0.57%  [.] scap_file_test_helpers::capture_search_evt_by_num
     1.19%     -0.50%  [.] std::_Hashtable<long, std::pair<long const, std::shared_ptr<sinsp_threadinfo> >, std::allocator<std::pair<long const, std::shared_ptr<sinsp_threadinfo> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::find
     0.72%     -0.47%  [.] std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libsinsp::state::dynamic_struct::field_info>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libsinsp::state::dynamic_struct::field_info> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node

Heap diff from master - unit tests

peak heap memory consumption: -384B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            +0.0991         +0.0991           155           170           155           170
BM_sinsp_split_median                                          +0.1007         +0.1007           155           170           155           170
BM_sinsp_split_stddev                                          -0.7647         -0.7648             3             1             3             1
BM_sinsp_split_cv                                              -0.7859         -0.7860             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  +0.0201         +0.0201            56            57            56            57
BM_sinsp_concatenate_paths_relative_path_median                +0.0228         +0.0228            56            57            56            57
BM_sinsp_concatenate_paths_relative_path_stddev                +0.2933         +0.2932             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_cv                    +0.2678         +0.2676             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     -0.0284         -0.0284            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_median                   -0.0296         -0.0296            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_stddev                   +1.8731         +1.8587             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_cv                       +1.9572         +1.9423             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  +0.0131         +0.0131            56            56            56            56
BM_sinsp_concatenate_paths_absolute_path_median                +0.0150         +0.0149            55            56            55            56
BM_sinsp_concatenate_paths_absolute_path_stddev                -0.5197         -0.5182             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_cv                    -0.5259         -0.5244             0             0             0             0
BM_sinsp_split_container_image_mean                            -0.0133         -0.0133           398           393           398           393
BM_sinsp_split_container_image_median                          -0.0140         -0.0140           399           393           399           393
BM_sinsp_split_container_image_stddev                          -0.2666         -0.2679             4             3             4             3
BM_sinsp_split_container_image_cv                              -0.2567         -0.2580             0             0             0             0

codecov · 2024-10-23T10:00:45Z

Codecov Report

Attention: Patch coverage is 79.48718% with 8 lines in your changes missing coverage. Please review.

Project coverage is 74.56%. Comparing base (b296470) to head (d0c002f).
Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
userspace/libsinsp/sinsp_filtercheck.cpp	50.00%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2130      +/-   ##
==========================================
+ Coverage   74.46%   74.56%   +0.10%     
==========================================
  Files         254      254              
  Lines       33333    33369      +36     
  Branches     5733     5720      -13     
==========================================
+ Hits        24820    24881      +61     
+ Misses       8509     8485      -24     
+ Partials        4        3       -1

Flag	Coverage Δ
libsinsp	`74.56% <79.48%> (+0.10%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

FedeDP · 2024-10-23T11:09:25Z

userspace/libsinsp/test/filterchecks/evt.cpp

+	evt = add_event_advance_ts(increasing_ts(), 1, PPME_SYSCALL_BRK_4_E, 1, addr);
+	// UINT64_MAX is FFFFFFFFFFFFFFFF
+	ASSERT_EQ(get_field_as_string(evt, "evt.rawarg.addr"), "FFFFFFFFFFFFFFFF");
+	ASSERT_ANY_THROW(eval_filter(evt, "evt.rawarg.addr > 0"));  // PT_SOCKADDR is not comparable


This issue is still present: even f the addr arg for brk event is an uint64_t, the initial filter arginfo is set to the first addr found in the event_table, ie: PPME_SOCKET_BIND_X that has a PT_SOCKADDR addr argument.
PT_SOCKADDR is not a comparable type (see filter_compare.cpp flt_is_comparable function) therefore we are not able to compile any filter against it.
NOTE: this was an issue already present on master.

yep not sure what to do here, one thing could be to rename some parameters to avoid the issue. We should reach a point where params with the same name have the same type, but yes this would be a breaking change...

FedeDP · 2024-10-23T11:12:11Z

/hold

Andreagit97 · 2024-10-23T14:42:50Z

userspace/libsinsp/sinsp_filtercheck_event.cpp

+		// it is safe to guess that eg: `size` param will always be a numeric type.
+		// Of course if we had a `size` param that is string, it would be troublesome.


i would just remove this since we have already found a corner case (addr). Maybe we could list here all the cases we are aware of, for example for now addr

Yep i am afraid that we forgot once again 😆

Andreagit97 · 2024-10-23T14:43:25Z

userspace/libsinsp/test/filterchecks/evt.cpp

+	evt = add_event_advance_ts(increasing_ts(), 1, PPME_SYSCALL_BRK_4_E, 1, addr);
+	// UINT64_MAX is FFFFFFFFFFFFFFFF
+	ASSERT_EQ(get_field_as_string(evt, "evt.rawarg.addr"), "FFFFFFFFFFFFFFFF");
+	ASSERT_ANY_THROW(eval_filter(evt, "evt.rawarg.addr > 0"));  // PT_SOCKADDR is not comparable


yep not sure what to do here, one thing could be to rename some parameters to avoid the issue. We should reach a point where params with the same name have the same type, but yes this would be a breaking change...

Firstly, properly refresh m_arginfo and m_customfield type and print format given current event while extracting rawarg values. Secondly, propelry support PT_FLAGS, PT_ENUMFLAGS, PT_GID and PT_UID types in `rawval_to_json` and `rawval_to_string`. Lastly, honor PF_HEX print format for 8,16,32bits types. Signed-off-by: Federico Di Pierro <[email protected]> Co-authored-by: Andrea Terzolo <[email protected]>

FedeDP · 2024-10-29T11:16:40Z

Rebased on top of latest master to make use of new cncf provided arm64 runners.

Andreagit97

/approve

poiana · 2024-10-29T13:05:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Andreagit97, FedeDP

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

FedeDP · 2024-10-30T16:04:06Z

/unhold

poiana added release-note kind/bug Something isn't working dco-signoff: yes area/libsinsp labels Oct 23, 2024

poiana added this to the 0.19.0 milestone Oct 23, 2024

poiana added size/M approved labels Oct 23, 2024

poiana requested review from gnosek and hbrueckner October 23, 2024 09:19

FedeDP force-pushed the fix/rawarg_madness branch from a351bb7 to d8c26f3 Compare October 23, 2024 09:19

FedeDP commented Oct 23, 2024

View reviewed changes

FedeDP changed the title ~~fix(userspace/libsinsp): multiple fixes related to rawargs.~~ wip: fix(userspace/libsinsp): multiple fixes related to rawargs. Oct 23, 2024

poiana added the do-not-merge/work-in-progress label Oct 23, 2024

FedeDP force-pushed the fix/rawarg_madness branch from d8c26f3 to a02a0ec Compare October 23, 2024 09:43

FedeDP commented Oct 23, 2024

View reviewed changes

FedeDP changed the title ~~wip: fix(userspace/libsinsp): multiple fixes related to rawargs.~~ fix(userspace/libsinsp): multiple fixes related to rawargs. Oct 23, 2024

poiana removed the do-not-merge/work-in-progress label Oct 23, 2024

poiana requested review from Andreagit97, jasondellaluce and LucaGuerra October 23, 2024 09:46

FedeDP changed the title ~~fix(userspace/libsinsp): multiple fixes related to rawargs.~~ wip: fix(userspace/libsinsp): multiple fixes related to rawargs. Oct 23, 2024

poiana added the do-not-merge/work-in-progress label Oct 23, 2024

FedeDP force-pushed the fix/rawarg_madness branch from a02a0ec to e4543d0 Compare October 23, 2024 09:49

FedeDP changed the title ~~wip: fix(userspace/libsinsp): multiple fixes related to rawargs.~~ fix(userspace/libsinsp): multiple fixes related to rawargs. Oct 23, 2024

poiana removed the do-not-merge/work-in-progress label Oct 23, 2024

FedeDP commented Oct 23, 2024

View reviewed changes

poiana added the do-not-merge/hold label Oct 23, 2024

Andreagit97 reviewed Oct 23, 2024

View reviewed changes

FedeDP force-pushed the fix/rawarg_madness branch from e4543d0 to d0c002f Compare October 29, 2024 11:16

Andreagit97 approved these changes Oct 29, 2024

View reviewed changes

poiana assigned Andreagit97 Oct 29, 2024

poiana added the lgtm label Oct 29, 2024

Andreagit97 mentioned this pull request Oct 30, 2024

[WIP] new(proposal): drop support for syscall enter events #2068

Open

leogr approved these changes Oct 30, 2024

View reviewed changes

poiana assigned leogr Oct 30, 2024

poiana removed the do-not-merge/hold label Oct 30, 2024

poiana merged commit 15e24ae into master Oct 30, 2024
49 checks passed

poiana deleted the fix/rawarg_madness branch October 30, 2024 16:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(userspace/libsinsp): multiple fixes related to rawargs. #2130

fix(userspace/libsinsp): multiple fixes related to rawargs. #2130

FedeDP commented Oct 23, 2024 •

edited

Loading

FedeDP commented Oct 23, 2024

FedeDP Oct 23, 2024

FedeDP Oct 23, 2024

FedeDP Oct 23, 2024

FedeDP Oct 23, 2024

FedeDP Oct 23, 2024 •

edited

Loading

FedeDP commented Oct 23, 2024

github-actions bot commented Oct 23, 2024 •

edited

Loading

codecov bot commented Oct 23, 2024 •

edited

Loading

FedeDP Oct 23, 2024

Andreagit97 Oct 23, 2024

FedeDP commented Oct 23, 2024

Andreagit97 Oct 23, 2024 •

edited

Loading

FedeDP Oct 23, 2024

Andreagit97 Oct 23, 2024

FedeDP commented Oct 29, 2024

Andreagit97 left a comment

poiana commented Oct 29, 2024

FedeDP commented Oct 30, 2024

		// it is safe to guess that eg: `size` param will always be a numeric type.
		// Of course if we had a `size` param that is string, it would be troublesome.

fix(userspace/libsinsp): multiple fixes related to rawargs. #2130

fix(userspace/libsinsp): multiple fixes related to rawargs. #2130

Conversation

FedeDP commented Oct 23, 2024 • edited Loading

FedeDP commented Oct 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FedeDP Oct 23, 2024 • edited Loading

Choose a reason for hiding this comment

FedeDP commented Oct 23, 2024

github-actions bot commented Oct 23, 2024 • edited Loading

Perf diff from master - unit tests

Heap diff from master - unit tests

Heap diff from master - scap file

Benchmarks diff from master

codecov bot commented Oct 23, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FedeDP commented Oct 23, 2024

Andreagit97 Oct 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FedeDP commented Oct 29, 2024

Andreagit97 left a comment

Choose a reason for hiding this comment

poiana commented Oct 29, 2024

FedeDP commented Oct 30, 2024

FedeDP commented Oct 23, 2024 •

edited

Loading

FedeDP Oct 23, 2024 •

edited

Loading

github-actions bot commented Oct 23, 2024 •

edited

Loading

codecov bot commented Oct 23, 2024 •

edited

Loading

Andreagit97 Oct 23, 2024 •

edited

Loading