IA-3433 Tasks view: Add extra filters on type, status and username (launcher) #1889

bramj · 2024-12-20T13:12:04Z

This PR improves the tasks. Specifically, it:

Adds a new column created_by on the Task model to differentiate between who did the initial launch, and who did a potential re-launch.
Adds the following filters to the tasks page:
- type (the task's "name")
- status
- user (both creator as well as last launcher are take into consideration)
- start and end date (of any of the timestamps created_at, started_at and ended_at
General refactoring:
- File structure of Tasks api endpoint, extracted filters.py andserializers.py
- DRY up some Django API filtering code
- Typescriptify some JS
Add test for Tasks API filtering

Screenshot

And add some indexes to support filtering.

between who did the initial launch, and who did a potential re-launch.

if different from the creator.

add filter on users (creator + launcher).

Extracted datetime helpers into general Iaso utils file.

kemar

Good job.

I'd say that you can safely remove 2 redundant indexes and add a relation in the select_related method.

The rest is optional.

kemar · 2025-01-08T14:00:30Z

iaso/api/deduplication/filters.py

-                start_date_dt = timezone.make_aware(start_date_dt, timezone.get_default_timezone())
-                start_date_dt = start_date_dt.replace(hour=0, minute=0, second=0)
-                queryset = queryset.filter(analyze__created_at__gte=start_date_dt)
+                queryset = queryset.filter(analyze__created_at__gte=date_string_to_start_of_day(start_date))


We may be able to use a string and only filter on the date part of the datetime field.

E.g. (assuming the date format is correct):

queryset.filter(analyze__created_at__date__gte="2021-05-23") queryset.filter(analyze__created_at__date__lte="2021-05-23")

This may be easier to read and may save some code.

kemar · 2025-01-08T14:09:21Z

iaso/models/base.py

@@ -274,11 +275,18 @@ class Task(models.Model):

    class Meta:
        ordering = ["-created_at"]
+        indexes = [
+            models.Index(fields=["account"]),


Suggested change

models.Index(fields=["account"]),

This is not necessary because a database index is automatically created on the ForeignKey (unless db_index=False is set).

kemar · 2025-01-08T14:09:36Z

iaso/models/base.py

+        indexes = [
+            models.Index(fields=["account"]),
+            models.Index(fields=["created_at"]),
+            models.Index(fields=["created_by"]),


Suggested change

models.Index(fields=["created_by"]),

Same thing as above.

kemar · 2025-01-08T14:12:39Z

beanstalk_worker/__init__.py

@@ -56,6 +56,7 @@ def wrapper(*args, **kwargs):
                task = Task()
                user = kwargs.pop("user")
                task.account = user.iaso_profile.account
+                task.created_by = user
                task.launcher = user


What I understand is that we assume that the creator and the launcher are always the same person. Is this correct?

kemar · 2025-01-08T14:13:51Z

iaso/tests/api/test_tasks.py

 class TaskSerializerSerializerTestCase(TestCase):
    """
    Test Task serializer.
    """

+    maxDiff = None


TBH I'm always tempted to put that in the base class…

kemar · 2025-01-08T14:16:52Z

iaso/api/tasks/views.py

    serializer_class = TaskSerializer
    results_key = "tasks"
    queryset = Task.objects.all()
    http_method_names = ["get", "patch", "head", "options", "trace"]

    def get_queryset(self):
        profile = self.request.user.iaso_profile
-        order = self.request.query_params.get("order", "created_at").split(",")
-        return Task.objects.select_related("launcher").filter(account=profile.account).order_by(*order)
+        return Task.objects.select_related("created_by", "launcher").filter(account=profile.account)


We could add user__iaso_profile because it's used in ordering_fields.

Suggested change

return Task.objects.select_related("created_by", "launcher").filter(account=profile.account)

return Task.objects.select_related("created_by", "launcher", "user__iaso_profile").filter(account=profile.account)

kemar · 2025-01-08T14:23:26Z

iaso/api/tasks/views.py

+        UsersFilterBackend,
+        StartEndDateFilterBackend,
+        StatusFilterBackend,
+    ]


Just for you information (no need to rewrite that) I like to do that via django-filter.

It has built-in validation and you don't need a bunch of different classes.

You have various examples in the code base.

kemar · 2025-01-08T14:25:55Z

hat/assets/js/apps/Iaso/domains/app/translations/fr.json

@@ -1357,7 +1357,8 @@
    "iaso.tasks.killed": "Arrêtée",
    "iaso.tasks.killSignalSent": "Signal d'arrêt envoyé",
    "iaso.tasks.killTask": "Arrêter la tâche",
-    "iaso.tasks.launcher": "Utilisateur",
+    "iaso.tasks.user": "Utilisateur",
+    "iaso.tasks.last_launch_by": "Dernière execution :",


Excuse my French 😄

Suggested change

"iaso.tasks.last_launch_by": "Dernière execution :",

"iaso.tasks.last_launch_by": "Dernière exécution :",

bramj force-pushed the IA-3433 branch from d04b556 to 36ed25b Compare January 7, 2025 10:45

bramj added 11 commits January 7, 2025 12:15

IA-3433 Change tasks index page to Typescript

2723463

IA-3433 Add created_by user to Task model

3dfb5c0

And add some indexes to support filtering.

IA-3433 Use created_by on Task to differentiate...

1f098e5

between who did the initial launch, and who did a potential re-launch.

IA-3433 Display last launcher in task interface...

7d77606

if different from the creator.

IA-3433 Improve ordering on tasks endpoint and ...

535e2cb

add filter on users (creator + launcher).

IA-3433 Allow date filtering on tasks API

5fb80ba

Extracted datetime helpers into general Iaso utils file.

IA-3433 Refact: Use datetime utils function to DRY up code

83d6ae3

IA-3433 Add filtering on status for Tasks API

8618405

IA-3433 Refac: Move tasks api to views.py and serializers.py

56ffc3d

IA-3433 Add filtering on task "type" (name)

f3f9272

IA-3433 Cleanup, fix tests and add task filtering test

f865156

bramj force-pushed the IA-3433 branch from 36ed25b to f865156 Compare January 7, 2025 11:16

IA-3433 Allow override of date_format for filters

0e402bc

bramj marked this pull request as ready for review January 7, 2025 14:02

bramj requested a review from kemar January 8, 2025 09:02

bramj self-assigned this Jan 8, 2025

kemar requested changes Jan 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IA-3433 Tasks view: Add extra filters on type, status and username (launcher) #1889

IA-3433 Tasks view: Add extra filters on type, status and username (launcher) #1889

bramj commented Dec 20, 2024 •

edited

Loading

kemar left a comment

kemar Jan 8, 2025

kemar Jan 8, 2025

kemar Jan 8, 2025

kemar Jan 8, 2025

kemar Jan 8, 2025

kemar Jan 8, 2025

kemar Jan 8, 2025

kemar Jan 8, 2025

	return Task.objects.select_related("created_by", "launcher").filter(account=profile.account)
	return Task.objects.select_related("created_by", "launcher", "user__iaso_profile").filter(account=profile.account)

	"iaso.tasks.last_launch_by": "Dernière execution :",
	"iaso.tasks.last_launch_by": "Dernière exécution :",

IA-3433 Tasks view: Add extra filters on type, status and username (launcher) #1889

Are you sure you want to change the base?

IA-3433 Tasks view: Add extra filters on type, status and username (launcher) #1889

Conversation

bramj commented Dec 20, 2024 • edited Loading

Screenshot

kemar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bramj commented Dec 20, 2024 •

edited

Loading