Merge pull request #277 from JiriPavela/feature-better-profile-sorting

Improved profile sorting and fields in status

docs/config.rst

@@ -110,13 +110,13 @@ List of Supported Options
     will yield the following status when running ``perun status`` (both for stored and pending
     profiles)::
 
-        ═══════════════════════════════════════════════════════════════════════════════▣
-          id ┃   type  ┃  cmd   ┃ workload ┃  args  ┃ collector  ┃         time        ┃
-        ═══════════════════════════════════════════════════════════════════════════════▣
-         0@p ┃ [mixed] ┃ target ┃ hello    ┃        ┃ complexity ┃ 2017-09-07 14:41:49 ┃
-         1@p ┃ [time ] ┃ perun  ┃          ┃ status ┃ time       ┃ 2017-10-19 12:30:29 ┃
-         2@p ┃ [time ] ┃ perun  ┃          ┃ --help ┃ time       ┃ 2017-10-19 12:30:31 ┃
-        ═══════════════════════════════════════════════════════════════════════════════▣
+        ══════════════════════════════════════════════════════════════════════▣
+          id ┃   type  ┃  cmd   ┃ workload ┃ collector  ┃         time        ┃
+        ══════════════════════════════════════════════════════════════════════▣
+         0@p ┃ [mixed] ┃ target ┃ hello    ┃ complexity ┃ 2017-09-07 14:41:49 ┃
+         1@p ┃ [time ] ┃ perun  ┃ status   ┃ time       ┃ 2017-10-19 12:30:29 ┃
+         2@p ┃ [time ] ┃ perun  ┃ --help   ┃ time       ┃ 2017-10-19 12:30:31 ┃
+        ══════════════════════════════════════════════════════════════════════▣
 
 .. confkey:: format.shortlog
 
@@ -175,6 +175,10 @@ List of Supported Options
         Placeholder for workload that was supplied to the profiled command (refer to
         :munit:`workloads` or :doc:`jobs` for more details).
 
+    ``%label%``:
+
+        Placeholder for custom user-supplied string label associated with a profile.
+
     ``%type%``:
 
         Placeholder for global type of the resources of the profile, i.e. `memory`, `time`,
@@ -205,6 +209,11 @@ List of Supported Options
 .. currentmodule:: perun.profile.helpers
 .. autoattribute:: ProfileInfo.valid_attributes
 
+.. confkey:: format.sort_profiles_order
+
+    ``[recursive]`` Specifies the order which will be used for sorting the output of the
+    ``perun status`` commands. Can be either ``ascending`` or ``descending``.
+
 .. confunit:: execute
 
    Groups various list of commands, that can be executed before specific phases. Currently this

docs/logs.rst

@@ -34,20 +34,20 @@ E.g. the following formatting string::
 will yield the following status when running ``perun status`` (both for stored and pending
 profiles)::
 
-    ═══════════════════════════════════════════════════════════════════════════════▣
-      id ┃   type  ┃  cmd   ┃ workload ┃  args  ┃ collector  ┃         time        ┃
-    ═══════════════════════════════════════════════════════════════════════════════▣
-     0@p ┃ [mixed] ┃ target ┃ hello    ┃        ┃ complexity ┃ 2017-09-07 14:41:49 ┃
-     1@p ┃ [time ] ┃ perun  ┃          ┃ status ┃ time       ┃ 2017-10-19 12:30:29 ┃
-     2@p ┃ [time ] ┃ perun  ┃          ┃ --help ┃ time       ┃ 2017-10-19 12:30:31 ┃
-    ═══════════════════════════════════════════════════════════════════════════════▣
+    ══════════════════════════════════════════════════════════════════════▣
+      id ┃   type  ┃  cmd   ┃ workload ┃ collector  ┃         time        ┃
+    ══════════════════════════════════════════════════════════════════════▣
+     0@p ┃ [mixed] ┃ target ┃ hello    ┃ complexity ┃ 2017-09-07 14:41:49 ┃
+     1@p ┃ [time ] ┃ perun  ┃ status   ┃ time       ┃ 2017-10-19 12:30:29 ┃
+     2@p ┃ [time ] ┃ perun  ┃ --help   ┃ time       ┃ 2017-10-19 12:30:31 ┃
+    ══════════════════════════════════════════════════════════════════════▣
 
 The first column of the ``perun status`` output, ``id``, has a fixed position and defines a tag for
-the given, which can be used in ``add``, ``rm``, ``show`` and ``postprocessby`` commands as a quick
-wildcard for concrete profiles, e.g. ``perun add 0@p`` would register the first profile stored in
-the pending ``.perun/jobs`` directory to the index of current head. Tags are always in form of
-``i@p`` (for pending profiles) and ``i@i`` for profiles registered in index, where ``i`` stands for
-position in the corresponding storage, index from zero.
+the given profile, which can be used in ``add``, ``rm``, ``show`` and ``postprocessby`` commands as
+a quick wildcard for concrete profiles, e.g. ``perun add 0@p`` would register the first profile
+stored in the pending ``.perun/jobs`` directory to the index of current head. Tags are always in
+form of ``i@p`` (for pending profiles) and ``i@i`` for profiles registered in index, where ``i``
+stands for position in the corresponding storage, index from zero.
 
 The specification of the formatting string can contain the following special tags:
 
@@ -65,6 +65,11 @@ The specification of the formatting string can contain the following special tag
     program, script or binary. Refer to :ref:`jobs-overview` for more information about profiling
     jobs and command workloads.
 
+``%label%``:
+    Lists the optional, user-defined string label associated with the profile. Labels may be used
+    to further distinguish profiles, e.g., collected using different environment configuration or
+    collected on differently configured machines.
+
 ``%collector%``:
     Lists the collector which was used to obtain the given profile. Refer to :doc:`collectors` for
     list of supported collectors and more information about collection of profiles.
@@ -76,10 +81,12 @@ The specification of the formatting string can contain the following special tag
     Original source of the profile. This corresponds to the name of the generated profile
     and the original path.
 
-By default the profiles are sorted according to the timestamp. The sort order can be modified by
-setting either the :ckey:`format.sort_profiles_by` or the :doc:`cli` option ``--sort-by`` to a
-valid profile information attribute. Setting the command line option ``--sort-by`` has higher
-priority than the key set in the :ckey:`format.sort_profiles_by`.
+By default the profiles are sorted in ascending order according to the timestamp. The sort order
+can be modified by setting either the :ckey:`format.sort_profiles_by` and
+:ckey:`format.sort_profiles_order` configuration options, or the :doc:`cli` options ``--sort-by``
+and ``--sort-order`` to a valid profile information attribute and a valid sort order. Setting the
+command line options ``--sort-by`` and ``--sort-order`` have higher priority than the keys set in
+the :ckey:`format.sort_profiles_by` and :ckey:`format.sort_profiles_order`.
 
 .. _logs-log:
 

perun/cli.py

@@ -63,7 +63,7 @@
     MissingConfigSectionException,
     ExternalEditorErrorException,
 )
-from perun.utils.structs.common_structs import Executable
+from perun.utils.structs.common_structs import Executable, SortOrder
 from perun.utils.structs.diff_structs import HeaderDisplayStyle
 from perun import fuzz as fuzz
 import perun.postprocess
@@ -398,8 +398,8 @@ def add(profile: list[str], minor: Optional[str], **kwargs: Any) -> None:
     pending profiles, then all the non-existing pending profiles will
     be obviously skipped.
     Run ``perun status`` to see the `tag` annotation of pending profiles.
-    Tags consider the sorted order as specified by the following option
-    :ckey:`format.sort_profiles_by`.
+    Tags consider the sorted order as specified by the options
+    :ckey:`format.sort_profiles_by` and :ckey:`format.sort_profiles_order`.
 
     Example of adding profiles:
 
@@ -486,8 +486,8 @@ def remove(
     to represent the removed profile. If the path points to existing profile in
     pending jobs (i.e. ``.perun/jobs`` directory) the profile is removed from the
     jobs, otherwise it is looked-up in the index.
-    Tags consider the sorted order as specified by the following option
-    :ckey:`format.sort_profiles_by`.
+    Tags consider the sorted order as specified by the options
+    :ckey:`format.sort_profiles_by` and :ckey:`format.sort_profiles_order`.
 
     Examples of removing profiles:
 
@@ -565,9 +565,22 @@ def log(head: Optional[str], **kwargs: Any) -> None:
     type=click.Choice(profiles.ProfileInfo.valid_attributes),
     callback=cli_kit.set_config_option_from_flag(pcs.local_config, "format.sort_profiles_by", str),
     help=(
-        "Sets the <key> in the local configuration for sorting profiles. "
-        "Note that after setting the <key> it will be used for sorting which is "
-        "considered in pending and index tags!"
+        "Sets the sort key in the local configuration as 'format.sort_profiles_by' that will be "
+        "used for sorting both pending and index profiles."
+    ),
+)
+@click.option(
+    "--sort-order",
+    "-so",
+    "format__sort_profiles_order",
+    nargs=1,
+    type=click.Choice(SortOrder.supported()),
+    callback=cli_kit.set_config_option_from_flag(
+        pcs.local_config, "format.sort_profiles_order", str
+    ),
+    help=(
+        "Sets the sort order in the local configuration as 'format.sort_profiles_order' that will "
+        "be used for sorting both pending and index profiles."
     ),
 )
 def status(**kwargs: Any) -> None:
@@ -585,17 +598,17 @@ def status(**kwargs: Any) -> None:
     default using ``less``).
 
     An error is raised if the command is executed outside of range of any
-    perun, or configuration misses certain configuration keys
-    (namely ``format.status``).
+    perun, or configuration misses certain configuration keys (namely
+    ``format.status``).
 
     Profiles (both registered in index and stored in pending directory) are sorted
-    according to the :ckey:`format.sort_profiles_by`. The option ``--sort-by``
-    sets this key in the local configuration for further usage. This means that
+    according to the key :ckey:`format.sort_profiles_by` and order
+    :ckey:`format.sort_profiles_order`. The options ``--sort-by`` and ``--sort-order``
+    set these keys in the local configuration for further usage. This means that
     using the pending or index tags will consider this order.
 
     Refer to :ref:`logs-status` for information how to customize the outputs of
-    ``status`` or how to set :ckey:`format.status` in nearest
-    configuration.
+    ``status`` or how to set :ckey:`format.status` in nearest configuration.
     """
     try:
         commands.try_init()
@@ -651,8 +664,8 @@ def show(ctx: click.Context, profile: Profile, **_: Any) -> None:
         5. Otherwise, the directory is walked for any match. Each found match
            is asked for confirmation by user.
 
-    Tags consider the sorted order as specified by the following option
-    :ckey:`format.sort_profiles_by`.
+    Tags consider the sorted order as specified by the options
+    :ckey:`format.sort_profiles_by` and :ckey:`format.sort_profiles_order`.
 
     Example 1. The following command will show the first profile registered at
     index of ``HEAD~1`` commit. The resulting graph will contain bars
@@ -742,8 +755,8 @@ def showdiff(ctx: click.Context, **kwargs: Any) -> None:
         5. Otherwise, the directory is walked for any match. Each found match
            is asked for confirmation by user.
 
-    Tags consider the sorted order as specified by the following option
-    :ckey:`format.sort_profiles_by`.
+    Tags consider the sorted order as specified by the options
+    :ckey:`format.sort_profiles_by` and :ckey:`format.sort_profiles_order`.
 
     Example 1. The following command will show the difference first two profiles
     registered at index of ``HEAD~1`` commit::
@@ -813,8 +826,8 @@ def postprocessby(ctx: click.Context, profile: Profile, **_: Any) -> None:
         5. Otherwise, the directory is walked for any match. Each found match
            is asked for confirmation by user.
 
-    Tags consider the sorted order as specified by the following option
-    :ckey:`format.sort_profiles_by`.
+    Tags consider the sorted order as specified by the options
+    :ckey:`format.sort_profiles_by` and :ckey:`format.sort_profiles_order`.
 
     For checking the associated `tags` to profiles run ``perun status``.
 

perun/cli_groups/import_cli.py

@@ -80,6 +80,23 @@
     default=False,
     help="Saves the imported profile to index.",
 )
+@click.option(
+    "--profile-name",
+    "-pn",
+    nargs=1,
+    type=str,
+    help=(
+        "Specifies the name of the resulting imported profile, which will be stored in .perun/jobs."
+    ),
+)
+@click.option(
+    "--profile-label",
+    "-pl",
+    nargs=1,
+    type=str,
+    default="",
+    help="An optional custom label to associate with the imported profile.",
+)
 @click.pass_context
 def import_group(ctx: click.Context, **kwargs: Any) -> None:
     """Imports Perun profiles from different formats.

perun/logic/commands.py

@@ -1094,6 +1094,7 @@ def get_untracked_profiles() -> list[ProfileInfo]:
                     "type": index_entry.type,
                     "cmd": index_entry.cmd,
                     "workload": index_entry.workload,
+                    "label": index_entry.label,
                 },
                 "collector_info": {"name": index_entry.collector},
                 "postprocessors": [{"name": p} for p in index_entry.postprocessors],

perun/logic/config.py

@@ -11,7 +11,7 @@
 from __future__ import annotations
 
 # Standard Imports
-from typing import Any, Iterable, Optional
+from typing import Any, Iterable, Optional, TypeVar
 import dataclasses
 import os
 import re
@@ -27,6 +27,9 @@
 from perun.utils.exceptions import SuppressedExceptions
 
 
+T = TypeVar("T")
+
+
 def is_valid_key(key: str) -> bool:
     """Validation function for key representing one option in config section.
 
@@ -211,6 +214,7 @@ def init_shared_config_at(path: str) -> None:
     output_profile_template: "%collector%-%cmd%-%workload%-%date%"
     output_show_template: "%collector%-%cmd%-%workload%-%date%"
     sort_profiles_by: time
+    sort_profiles_order: asc
 
 degradation:
     apply: all
@@ -468,3 +472,38 @@ def gather_key_recursively(key: str) -> list[Any]:
         except exceptions.MissingConfigSectionException:
             continue
     return gathered_values
+
+
+def safely_lookup_key_recursively(key: str, allowed_values: Iterable[T], default: T) -> T:
+    """Safely recursively looks up the key in runtime, local and shared config.
+
+    By safely, we mean that if the function returns a value, that value will be one of the allowed
+    values. If no allowed values are provided, or the default value itself is not one of the allowed
+    values, an AssertionError is raised.
+
+    :param key: the looked up key
+    :param allowed_values: a nonempty set of allowed values
+    :param default: a default value (must be one of allowed value) to use in case of any issues
+    """
+    # A set of allowed values must be provided and the default value must be one of the allowed
+    # values to ensure the resulting value is always valid.
+    assert allowed_values and default in allowed_values
+    error_desc: str = ""
+    value = default
+    try:
+        value = lookup_key_recursively(key)
+        if value not in allowed_values:
+            # If the stored key is invalid, we use the default value instead
+            error_desc = f"invalid value '{value}' of key '{key}' in config. "
+            value = default
+    except exceptions.MissingConfigSectionException:
+        # If there is no value for the looked up key, we use the default value instead
+        error_desc = f"missing config value for key '{key}'. "
+    if error_desc:
+        perun_log.warn(
+            error_desc + f"The default value '{default}' will be used instead.\n\n"
+            f"Please run 'perun config edit' and set '{key}' to one of "
+            f"({', '.join(map(str, allowed_values))}). "
+            f"Consult the documentation (Configuration and Logs) for more information."
+        )
+    return value

perun/logic/config_templates.py

@@ -153,9 +153,10 @@
 #   register_after_run: true
 {% endif %}
 
-## Be default, we sort the profiles by time
+## Be default, we sort the profiles by time in ascending order
 format:
   sort_profiles_by: time
+  sort_profiles_order: asc
 {% if format is defined and format.output_profile_template is defined %}
 ## The following changes the automatically generated name of the profiles
   output_profile_template: "{{ format.output_profile_template }}"