Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new(userspace): plugin api to dump async events #2152

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

FedeDP
Copy link
Contributor

@FedeDP FedeDP commented Nov 7, 2024

What type of PR is this?

/kind feature

Any specific area of the project related to this PR?

/area libsinsp

Does this PR require a change in the driver versions?

What this PR does / why we need it:

This PR adds a new dump() API under the async events plugin capability.
It is used to dump plugin state (as list of PPME_ASYNCEVENT_E events) when a scap file dump is requested.
Note:

  • only PPME_ASYNCEVENT_E can be dumped
  • PPME_ASYNCEVENT_E dumped must have a name supported by the plugin (see get_async_events() API)

Moreover, enable initialstate consuming of PPME_ASYNCEVENT_E events while opening the inspector.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

new(userspace): plugin api to dump async events

@FedeDP
Copy link
Contributor Author

FedeDP commented Nov 7, 2024

/milestone 0.19.0
/cc @jasondellaluce @mrgian

@poiana poiana added this to the 0.19.0 milestone Nov 7, 2024
@poiana poiana added the size/L label Nov 7, 2024
@poiana
Copy link
Contributor

poiana commented Nov 7, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: FedeDP

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@@ -159,7 +159,7 @@ bool sinsp::is_initialstate_event(scap_evt* pevent) const {
return pevent->type == PPME_CONTAINER_E || pevent->type == PPME_CONTAINER_JSON_E ||
pevent->type == PPME_CONTAINER_JSON_2_E || pevent->type == PPME_USER_ADDED_E ||
pevent->type == PPME_USER_DELETED_E || pevent->type == PPME_GROUP_ADDED_E ||
pevent->type == PPME_GROUP_DELETED_E;
pevent->type == PPME_GROUP_DELETED_E || pevent->type == PPME_ASYNCEVENT_E;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial state events parsing enablement for PPME_ASYNCEVENT_E.

};

// scenario: a plugin with dump capability is requested a dump and then the capture file is read.
TEST_F(sinsp_with_test_input, plugin_dump) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very simple test:

  • register a plugin with async event capability
  • open inspector in no driver mode
  • request a scap file dump to a temporary text file
  • at this stage, the plugin will be requested to dump its state; in our test case, the plugin will just dump 10 fake events
  • open a replay inspector to read the generated scap file
  • register our event processor that just counts number of PPME_ASYNC_EVENT_E
  • remove the test scap file

Copy link

github-actions bot commented Nov 7, 2024

Perf diff from master - unit tests

     3.22%     +1.33%  [.] sinsp_parser::process_event
     3.63%     -1.01%  [.] sinsp_thread_manager::find_thread
     1.81%     -0.71%  [.] libsinsp::sinsp_suppress::process_event
     1.44%     -0.67%  [.] 0x00000000000ea364
     9.90%     -0.63%  [.] sinsp_parser::reset
     0.31%     +0.62%  [.] std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libsinsp::state::dynamic_struct::field_info>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libsinsp::state::dynamic_struct::field_info> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_emplace<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libsinsp::state::dynamic_struct::field_info> >
     1.02%     -0.61%  [.] std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libsinsp::state::dynamic_struct::field_info>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, libsinsp::state::dynamic_struct::field_info> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node
     0.67%     +0.51%  [.] scap_event_encode_params_v
     0.52%     +0.49%  [.] sinsp_utils::find_longest_matching_evt_param
     3.16%     +0.48%  [.] sinsp_evt::load_params

Heap diff from master - unit tests

peak heap memory consumption: -378B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            -0.0376         -0.0376           146           141           146           141
BM_sinsp_split_median                                          -0.0360         -0.0360           146           141           146           141
BM_sinsp_split_stddev                                          +2.4398         +2.4432             1             2             1             2
BM_sinsp_split_cv                                              +2.5742         +2.5777             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  -0.0490         -0.0490            60            57            60            57
BM_sinsp_concatenate_paths_relative_path_median                -0.0499         -0.0499            60            57            60            57
BM_sinsp_concatenate_paths_relative_path_stddev                +0.1895         +0.1915             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_cv                    +0.2509         +0.2529             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     -0.0115         -0.0115            25            25            25            25
BM_sinsp_concatenate_paths_empty_path_median                   -0.0134         -0.0134            25            25            25            25
BM_sinsp_concatenate_paths_empty_path_stddev                   +0.4423         +0.4414             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_cv                       +0.4591         +0.4582             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  -0.0983         -0.0983            62            56            62            56
BM_sinsp_concatenate_paths_absolute_path_median                -0.1073         -0.1073            62            55            62            55
BM_sinsp_concatenate_paths_absolute_path_stddev                -0.7039         -0.7040             1             0             1             0
BM_sinsp_concatenate_paths_absolute_path_cv                    -0.6716         -0.6717             0             0             0             0
BM_sinsp_split_container_image_mean                            +0.0154         +0.0154           373           378           373           378
BM_sinsp_split_container_image_median                          +0.0173         +0.0173           372           379           372           379
BM_sinsp_split_container_image_stddev                          -0.1683         -0.1684             3             2             3             2
BM_sinsp_split_container_image_cv                              -0.1810         -0.1810             0             0             0             0

@FedeDP
Copy link
Contributor Author

FedeDP commented Nov 7, 2024

/hold
fixing a couple of things.

@FedeDP
Copy link
Contributor Author

FedeDP commented Nov 7, 2024

/unhold

Copy link

codecov bot commented Nov 7, 2024

Codecov Report

Attention: Patch coverage is 72.54902% with 28 lines in your changes missing coverage. Please review.

Project coverage is 74.79%. Comparing base (dd8d508) to head (5dbd02e).
Report is 26 commits behind head on master.

Files with missing lines Patch % Lines
userspace/libsinsp/plugin.cpp 53.48% 20 Missing ⚠️
userspace/libsinsp/dumper.cpp 50.00% 3 Missing ⚠️
userspace/libsinsp/test/plugins/syscall_async.cpp 83.33% 3 Missing ⚠️
userspace/libsinsp/test/plugins.ut.cpp 94.11% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2152      +/-   ##
==========================================
+ Coverage   74.70%   74.79%   +0.08%     
==========================================
  Files         254      255       +1     
  Lines       33557    33606      +49     
  Branches     5737     5778      +41     
==========================================
+ Hits        25070    25135      +65     
- Misses       8458     8471      +13     
+ Partials       29        0      -29     
Flag Coverage Δ
libsinsp 74.79% <72.54%> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Also, ASYNCEVENT_E will not be correctly pre-parsed at init time while reading from captures.

Signed-off-by: Federico Di Pierro <[email protected]>
We now expect `PPME_ASYNCEVENT_E` whose `name` matches one of the
plugin supported ones (get_async_events() API).

The new API is not required for async capability.

Added also a test.

Signed-off-by: Federico Di Pierro <[email protected]>
Other tests using no_driver engine and async plugin are also disabled on it.

Signed-off-by: Federico Di Pierro <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants