Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flows: improve sorting of listed flows by event times #604

Open
Tracked by #58
zaychenko-sergei opened this issue Apr 17, 2024 · 0 comments · May be fixed by #1022
Open
Tracked by #58

Flows: improve sorting of listed flows by event times #604

zaychenko-sergei opened this issue Apr 17, 2024 · 0 comments · May be fixed by #1022
Assignees
Labels
enhancement New feature or request

Comments

@zaychenko-sergei
Copy link
Contributor

zaychenko-sergei commented Apr 17, 2024

See the following UI example of flow listing per dataset:

image

The compacting flow was scheduled later than the planned update.
Since it did not have to wait for anything much, it got executed earlier than the planned update, which is still waiting for the next schedule tick in almost an hour.
Still, we display the compacting above the in-progress update.

If we take a look at in-memory repository implementation that backs this listing, there is no sorting at all, and the order is dependent on the order of flow creation (time of initiation event):

    #[tracing::instrument(level = "debug", skip_all, fields(%dataset_id, ?filters, ?pagination))]
    fn get_all_flow_ids_by_dataset(
        &self,
        dataset_id: &DatasetID,
        filters: DatasetFlowFilters,
        pagination: FlowPaginationOpts,
    ) -> FlowIDStream {
        let flow_ids_page: Vec<_> = {
            let state = self.inner.as_state();
            let g = state.lock().unwrap();
            g.all_flows_by_dataset
                .get(dataset_id)
                .map(|dataset_flow_ids| {
                    dataset_flow_ids
                        .iter()
                        .rev()
                        .filter(|flow_id| g.matches_dataset_flow(**flow_id, &filters))
                        .skip(pagination.offset)
                        .take(pagination.limit)
                        .map(|flow_id| Ok(*flow_id))
                        .collect()
                })
                .unwrap_or_default()
        };

        Box::pin(futures::stream::iter(flow_ids_page))
    }

The expectations are:

  • finished flows should be below any waiting or running
  • currently running flows should be below any waiting
  • 2 or more completed flows should be sorted by completion time, earliest below
  • 2 or more running flows should be ordered by the time running started, earliest below
  • 2 or more waiting flows should be ordered by the time the latest event was recorded (such as condition change), earliest below

Update: now we should also account for this in the Postgres & Sqlite implementations.

@zaychenko-sergei zaychenko-sergei added the enhancement New feature or request label Apr 17, 2024
@demusdev demusdev linked a pull request Dec 30, 2024 that will close this issue
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants