Software Defined Events

On this page:

Enabling Software Defined Events
Reading SDEs from within application code or performance tools
Adding SDEs to a library

Enabling Software Defined Events

To enable reading Software Defined Events (SDEs), the user needs to link against a PAPI library that was configured with the sde component enabled. As an example the following command: ./configure --with-components="sde" is sufficient to enable the component.

Reading SDEs from within application code or performance tools

If the PAPI library is configured with the sde component enabled, then SDEs can be read just like any other event supported by a PAPI component. The same API (PAPI_start()/PAPI_read()/PAPI_stop()) applies unmodified as with hardware events. The only caveat relates to types. PAPI_read() will always store the results in an array of type long long int. However, a library might export a counter that is of a different type, e.g., double. PAPI will not cast between the different types. Instead, it will pack the bits of the library counter into the long long int in the result. To extract a double from a counter variable cntr cast it as such: *((double *)&cntr).

Adding SDEs to a library

In the following text, we will provide examples of different features and explain how to use them. All these examples pertain to libraries (or other software layers) that wish to add SDEs in their code. For merely reading SDEs generated by a library, see the previous section.

SDE counters come in a total of six flavors:

The most common is a program variable that is being ${{\color{Goldenrod}{\textsf{registered}}}}$ with PAPI as an SDE. Registered counters incur zero overhead since they are program variables whose values are being modified as part of the normal execution of the library without requiring any SDE-specific API calls.
${{\color{Goldenrod}{\textsf{Created\ counters}}}}$, are variables that are created and managed internally by PAPI. Modifying their value requires SDE-specific API calls, which causes overhead, but PAPI is always aware of their value and therefore can deliver accurate overflow notifications.
${{\color{Goldenrod}{\textsf{Callback\ counters}}}}$, allow libraries to associate a callback with a counter. Every time a user code reads the counter, the callback is invoked, allowing the library to generate complex dynamic values that might not reside in any particular variable.
${{\color{Goldenrod}{\textsf{Counter\ groups}}}}$, enable users to read the sum, minimum, or maximum value of all counters in the group by treating the counter group as a new counter. Counter groups are first-class citizens and can be recursively combined with other counters or groups into larger groups.
${{\color{Goldenrod}{\textsf{Recorders}}}}$, are multi-value counters that are also managed internally by PAPI. Recorders enable libraries to record an arbitrarily long sequence of data.
${{\color{Goldenrod}{\textsf{Counting\ sets}}}}$, enable libraries to store objects, where each object is unique, just as a traditional set data-structure. In addition to the traditional data-structure, however, counting sets also keep track of the number of times each object was added to the counting set. In other words, counting sets enable libraries to automatically create histograms of library-specific objects with arbitrary data types.

Header files and linking

Consider that you are the developer of the project SomeName which you distribute as the library libSomeName.so. To add SDEs into your library, you need to link it against libsde.so (or libsde.a) and your source code needs to include sde_lib.h. If PAPI is installed in your system, then these files will be installed in the same location where libpapi.so (libpapi.a) and papi.h are installed, respectively. You do not need libpapi to export SDEs; this is only needed to read SDEs from an application that uses libSomeName.

Minimal example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Minimal/Minimal_Test.c.

The following code is the minimum necessary library code for creating an SDE. Let's call this library mintest.

long long local_var;

void mintest_init(void){
    local_var = 0;
    papi_handle_t *handle = papi_sde_init("Min Example Code");
    papi_sde_register_counter(handle, "Example Event", PAPI_SDE_RO|PAPI_SDE_DELTA, PAPI_SDE_long_long, &local_var);
}

Note: If choosing to copy the code block instead of visiting the full example code located at src/components/sde/tests/Minimal/Minimal_Test.c make sure you have included sde_lib.h, papi.h, papi_test.h and the appropriate system headers. As well as passed the sde library to the linker when compiling.

This code assumes that mintest has an internal variable (local_var) that counts some event that occurs inside the library, which the developers wish to export to the outside world. To do so, the library needs to call some papi_sde functions.

First, the function papi_sde_init() must be called. The parameter is a string (const char *) that contains the name of the library. The developers can choose an arbitrary string; PAPI does not parse or process it in any way. However, this string will be appended as a prefix to all SDEs exported by this library, so the colon character : should be avoided, due to its special meaning in PAPI events. This function returns an opaque handle that must be passed to all future papi_sde calls made by this library.

After obtaining the handle, mintest calls: papi_sde_register_counter(handle, "Example Event", PAPI_SDE_RO|PAPI_SDE_DELTA, PAPI_SDE_long_long, &local_var);

handle is the opaque handle returned by papi_sde_init().
The second parameter is an arbitrary string containing the name of the SDE being registered. Once again, only the colon character : should be avoided.
The third parameter specifies the mode of the event. The mode is a bitwise-or of two flags:
1. The flag that specifies the access mode of the event counter, which can be PAPI_SDE_RO or PAPI_SDE_RW (for read-only and read-write counters, respectively);
2. The flag that specifies whether this event is a delta (PAPI_SDE_DELTA) or an instant (PAPI_SDE_INSTANT) event. For delta events, PAPI reports the difference in the value of the counter between PAPI_start() and PAPI_read() (i.e., as in the hardware events that count instructions executed). For instant events, PAPI reports the actual value of the counter when PAPI_read() is called (i.e., as in the hardware events that report power usage).
The fourth parameter specifies the counter type and has to be one of: PAPI_SDE_long_long, PAPI_SDE_int, PAPI_SDE_double, PAPI_SDE_float
The last parameter is a pointer to the counter. I.e., a pointer to the library variable that counts the occurrences of the SDE that is being registered.

Counter Group and Callback-counter example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Simple2/Simple2_Lib.c.

The following code demonstrates the use of counter groups and callback counters that register a function that computes the value of an event at run-time. Let's call this example library Simple.

static double comp_value;
static long long int total_iter_cnt, low_wtrmrk, high_wtrmrk;
static papi_handle_t handle;

static const char *ev_names[4] = { 
    "COMPUTED_VALUE",
    "TOTAL_ITERATIONS",
    "LOW_WATERMARK_REACHED",
    "HIGH_WATERMARK_REACHED"
};

long long counter_accessor_function( void *param ){
    long long ll; 
    double *dbl_ptr = (double *)param;

    // Scale the variable by a factor of two. Real libraries will do meaningful work here.
    double value = *dbl_ptr * 2.0;

    // Copy the bits of the result in a long long int.
    (void)memcpy(&ll, &value, sizeof(double));

    return ll; 
}

void simple_init(void){
    // Initialize library specific variables
    comp_value = 0.0;
    total_iter_cnt = 0;
    low_wtrmrk = 0;
    high_wtrmrk = 0;

    // Initialize PAPI SDEs
    handle = papi_sde_init("Simple2");
    papi_sde_register_counter_cb(handle, ev_names[0], PAPI_SDE_RO|PAPI_SDE_INSTANT, PAPI_SDE_double, counter_accessor_function, &comp_value);
    papi_sde_register_counter(handle, ev_names[1], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &total_iter_cnt);
    papi_sde_register_counter(handle, ev_names[2], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &low_wtrmrk);
    papi_sde_register_counter(handle, ev_names[3], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &high_wtrmrk);
    papi_sde_add_counter_to_group(handle, ev_names[2], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);
    papi_sde_add_counter_to_group(handle, ev_names[3], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);

    return;
}

Note: If choosing to copy the code block instead of visiting the full example code located at src/components/sde/tests/Simple2/Simple2_Lib.c make sure you have included sde_lib.h, and relevant system headers. As well as passed the sde library to the linker when compiling.

In addition to registering counters as the example in the previous section did, this code uses two more SDE features. First, it registers a callback counter by calling the function:
papi_sde_register_counter_cb(handle, ev_names[0], PAPI_SDE_RO|PAPI_SDE_INSTANT, PAPI_SDE_double, counter_accessor_function, &comp_value);

The first four parameters of this call have the same semantics as in the case of papi_sde_register_counter(). However, the fifth parameter is a pointer to a callback function (instead of a pointer to a counter). This function will be called by PAPI when the application makes a call to PAPI_read(), or PAPI_stop(). Therefore, using this functionality a library can create events whose value is computed at run-time and does not necessarily correspond to a program variable. The last parameter is a user-specified pointer that is opaque to PAPI, and it will be passed by PAPI to the callback when it is called.

In addition to registering counters, this example code calls the function papi_sde_add_counter_to_group() to add two counters (that have already been registered) to the group named ANY_WATERMARK_REACHED. The parameters of this function are:

The handle returned by papi_sde_init().
The name of the counter which is being added to the group.
The name of the group.
The operation that PAPI will perform on the members of the group when the group is read. This can be one of the following:
1. PAPI_SDE_SUM
2. PAPI_SDE_MAX
3. PAPI_SDE_MIN

Counter groups are first class citizens and can be recursively combined with other counters or groups into larger groups. The following limitations apply to the way counters can be organized in groups:

Groups can be formed out of counters of type int, long long int, float, and double, but all the counters in a single group must be of the same type. If counters of different types are grouped together the behavior is undefined.
All the counters added in a group must use the same operation (PAPI_SDE_SUM, PAPI_SDE_MAX, or PAPI_SDE_MIN).

Listing SDEs through the papi_native_avail utility

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Simple2/Simple2_Lib.c.

Since the introduction of the sde component in PAPI, the utility papi_native_avail accepts the flag -sde followed by a path to either a library that contains SDEs or an executable that is linked against libraries that contain SDEs. When this flag and an appropriate path are specified, papi_native_avail will list all the SDEs found in the library (or all libraries linked against the executable). To enable this behavior, the library must implement the hook function:
papi_handle_t papi_sde_hook_list_events(papi_sde_fptr_struct_t *fptr_struct)
This function will be called by papi_native_avail and the parameter it takes is a pointer to a structure that contains function pointers to all SDE functions. The following code shows an example of how this function could be implemented. Note that this function is not supposed to be called by other library functions or normal applications. It is only a hook for the utility papi_native_avail to be able to discover the SDEs that are exported by a library. Therefore the pointers to the actual counters can be NULL, since they will never be dereferenced.

static double comp_value;
static long long int total_iter_cnt, low_wtrmrk, high_wtrmrk;
static papi_handle_t handle;

static const char *ev_names[4] = { 
    "COMPUTED_VALUE",
    "TOTAL_ITERATIONS",
    "LOW_WATERMARK_REACHED",
    "HIGH_WATERMARK_REACHED"
};

long long counter_accessor_function( void *param ){
    long long ll; 
    double *dbl_ptr = (double *)param;

    // Scale the variable by a factor of two. Real libraries will do meaningful work here.
    double value = *dbl_ptr * 2.0;

    // Copy the bits of the result in a long long int.
    (void)memcpy(&ll, &value, sizeof(double));

    return ll; 
}

papi_handle_t papi_sde_hook_list_events( papi_sde_fptr_struct_t *fptr_struct){
    handle = fptr_struct->init("Simple2");
    fptr_struct->register_counter_cb(handle, ev_names[0], PAPI_SDE_RO|PAPI_SDE_INSTANT, PAPI_SDE_double, counter_accessor_function, &comp_value);
    fptr_struct->register_counter(handle, ev_names[1], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &total_iter_cnt);
    fptr_struct->register_counter(handle, ev_names[2], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &low_wtrmrk);
    fptr_struct->register_counter(handle, ev_names[3], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &high_wtrmrk);
    fptr_struct->add_counter_to_group(handle, ev_names[2], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);
    fptr_struct->add_counter_to_group(handle, ev_names[3], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);

    fptr_struct->describe_counter(handle, ev_names[0], "Sum of values that are within the watermarks.");
    fptr_struct->describe_counter(handle, ev_names[1], "Total iterations executed by the library.");
    fptr_struct->describe_counter(handle, ev_names[2], "Number of times a value was below the low watermark.");
    fptr_struct->describe_counter(handle, ev_names[3], "Number of times a value was above the high watermark.");
    fptr_struct->describe_counter(handle, "ANY_WATERMARK_REACHED",  "Number of times a value was not between the two watermarks.");

    return handle;
}

Note: If choosing to copy the code block instead of visiting the full example code located at src/components/sde/tests/Simple2/Simple2_Lib.c make sure you have included sde_lib.h, and relevant system headers. As well as passed the sde library to the linker when compiling.

Created Counter example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Created_Counter/Lib_With_Created_Counter.c.

Created Counters (CCs) are SDEs that are managed internally by PAPI, in contrast with Registered Counters, which are library variables whose address has been registered with PAPI. The benefit of using a CC is that PAPI is always aware of its value, so it can notify "listeners" immediately after the value exceeds a threshold set by the listener. This is enabled through the Overflow interface of PAPI and is discussed in the following section. The downside of CCs is that their value has to be updated through an SDE API call, so using CCs incurs overhead, in contrast with registered counters.

The following example shows the creation and update of a CC.

#define MY_EPSILON 0.0001

static const char *event_names[1] = {
    "epsilon_count"
};

void *cntr_handle;

void cclib_init(void){
    papi_handle_t sde_handle;

    sde_handle = papi_sde_init("Lib_With_CC");
    papi_sde_create_counter(sde_handle, event_names[0], PAPI_SDE_DELTA, &cntr_handle);

    return;
}

void cclib_do_work(void){
    int i;

    for(i=0; i<100*1000; i++){
        double r = (double)random() / (double)RAND_MAX;
        if( r < MY_EPSILON ){
            papi_sde_inc_counter(cntr_handle, 1);
        }
        // Do some usefull work here
        if( !(i%100) )
            (void)usleep(1);
    }

    return;
}

Note: If choosing to copy the code block instead of visiting the full example code located at src/components/sde/tests/Created_Counter/Lib_With_Created_Counter.c make sure you have included sde_lib.h, and relevant system headers. As well as passed the sde library to the linker when compiling.

Using the Overflow mechanism to monitor a Created Counter

A user program can invoke the PAPI function PAPI_overflow() to specify a callback function and a threshold value for a particular event. When the event counter exceeds this threshold, PAPI will invoke the callback function. This mechanism is not SDE specific but has always been part of PAPI. The SDE component supports overflowing for all types of counters, but when overflowing is used with Created Counters, the callback is invoked immediately after the CC exceeds the threshold, since PAPI is always aware of the value of the CC.

No special treatment is needed to read SDEs through the overflow mechanism, which means that 3rd-party tools that are based on sampling should work out of the box.

The preferred type of overflowing for the sde component is PAPI_OVERFLOW_HARDWARE, which signifies that the component will be in charge of supporting the overflowing mechanism, instead of the PAPI framework.

Recorder example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Recorder/Lib_With_Recorder.c.

Recorders go beyond the notion of a single event counter in that they can record an indefinite number of values. In other words, a library can use recorders to store multiple values of interest, and the user application can retrieve the whole sequence of stored values, not just the latest one. PAPI provides the necessary storage using a dynamic data structure that tries to keep the amount of allocated space at each given time low, while also minimizing the amount of time spent in allocating additional space.

The following example code calls the function:
papi_sde_create_recorder(tmp_handle, event_names[0], sizeof(long long), papi_sde_compare_long_long, &rcrd_handle)
to create a recorder. The parameters of this function are as follows:

The handle returned by papi_sde_init().
The name of the recorder.
The size of each of the elements that will be recorded.
An optional function pointer for comparing elements. This function pointer (if not NULL) will be used to automatically compute the quartiles of the recorded elements. It can be one of the following:
1. NULL (if quartiles should not be computed automatically).
2. papi_sde_compare_long_long(const void *p1, const void *p2)
3. papi_sde_compare_int(const void *p1, const void *p2)
4. papi_sde_compare_double(const void *p1, const void *p2)
5. papi_sde_compare_float(const void *p1, const void *p2)
6. A custom, user-provided function which takes two const void * parameters and returns an int less than, equal to, or greater than zero if the first argument is considered to be respectively less than, equal to, or greater than the second.
A pointer to a void * variable. This is an output parameter, and after the return of this function it will contain a handle associated with the newly created recorder.

static const char *event_names[1] = {
    "simple_recording"
};

void *rcrd_handle;

void recorder_init_(void){
    papi_handle_t tmp_handle;

    tmp_handle = papi_sde_init("Lib_With_Recorder");
    papi_sde_create_recorder(tmp_handle, event_names[0], sizeof(long long), papi_sde_compare_long_long, &rcrd_handle);

    return;
}

void recorder_do_work_(void){
    long long r = random()%123456;
    papi_sde_record(rcrd_handle, sizeof(r), &r);
    return;
}

Note: If choosing to copy the code block instead of visiting the full example code located at src/components/sde/tests/Recorder/Lib_With_Recorder.c make sure you have included sde_lib.h, and relevant system headers. As well as passed the sde library to the linker when compiling.

Just as is the case with registering counters, the creation of a recorder happens only once. However, to record elements, the library code must call the function papi_sde_record() for every element that must be recorded. This function takes as parameters:

The recorder handle (i.e., the last parameter passed to papi_sde_create_recorder()).
The size of the element being recorded.
A pointer to the element being recorded.

It is worth noting that PAPI treats the recorded element as opaque, which means that a library can use recorders for any object that is stored in contiguous memory, from basic types to structures to whole matrices.

Reading the elements in a Recorder

PAPI_read() can only read one value per event. If the SDE that is being read is a recorder, then PAPI stores in the read value a pointer to a contiguous buffer that contains all the recorder elements. Additionally, every time a recorder is created, PAPI automatically creates an additional SDE whose value is equal to the number of elements stored in the recorder. The name of this auxiliary SDE is formed by appending the suffix :CNT to the name of the recorder. This auxiliary SDE is automatically updated every time a new element is recorded. An example of reading the elements stored in a recorder can be found in the PAPI repo under: src/components/sde/tests/Recorder/Recorder_Driver.c.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly