refactor!: run data generators only for visible columns

[ugur-vaadin]

Description

Warning

Possibly behaviour altering change.
Unlikely to break an existing app.

Currently, the value provider in any type of created grid column runs regardless of its visibility. This can cause unnecessary load if the tasks are heavy.

This PR updates the rendering logic in the columns to run data generators only if the column is visible before the client response. This also avoids sending unnecessary data to the client.

The downside is that there is an extra refresh when a column is set visible.

Fixes #6737

Type of change

• Bugfix
• Feature
• Refactor

Checklist

• I have read the contribution guide: https://vaadin.com/docs/latest/contributing/overview
• I have added a description following the guideline.
• The issue is created in the corresponding repository and I have referenced it.
• I have added tests to ensure my change is effective and works as intended.
• New and existing tests are passing locally with my change.
• I have performed self-review and corrected misspellings.

[sissbruecker]

Regarding the utility of this change: This sort of moves the performance issue to a different use-case, which is showing and hiding columns. In that case it now needs to reload all data whenever a single column is made visible. Previously it would load everything up front, but then doesn't have to reload everything if you show a single column.

What is everyone's thought on this?

[tomivirkki]

In theory, hiding and unhiding columns shouldn't need to result in a data provider call, but only the data generators should need to be rerun for the already loaded items.

[sissbruecker]

I'm pretty sure that whoever created this issue wouldn't expect additional data provider calls as part of this change.

Since there's a trade-off, maybe it could be turned into an option. Though it's a bit challenging to communicate what this even does.

[tomivirkki]

This nasty prototype method in Grid does the job without additional data provider calls, but it requires reflection. I hope there's a better way (maybe storing the value in Grid.setRequestedRange):

private void regenerateData() {
        try {
        var communicator = getDataCommunicator();
        var requestedRangeField = communicator.getClass()
                .getDeclaredField("requestedRange");
        requestedRangeField.setAccessible(true);
        var requestedRange = (Range) requestedRangeField
                .get(communicator);

        for (int i = requestedRange.getStart(); i < requestedRange.getEnd(); i++) {
            var item = communicator.getItem(i);
            if (item != null) {
                getDataCommunicator().refresh(item);
            }
        }
    } catch (Exception e) {
        // ignore
    }
}

[vursen]

Might be related: #6713

[sissbruecker]

If the app calls refreshAll before making a column visible, that results in two calls to the data provider now.

In general I'm uncertain about calling refreshAll in beforeClientResponse, which is the same phase that data communicator uses to prepare data to send to the client. Is there a reason that this has to be called in beforeClientResponse, or could this already be called in setVisible?

[ugur-vaadin]

I agree that it would be better to do it that way. However, directly calling in setVisible does not work and I'm not sure what would be a good way to refactor the refresh.

[sissbruecker]

When adding a column, it sends two full updates to the client. One based on the old column config, and then another one with the new config. Also calls the data provider twice.

[ugur-vaadin]

Added a fix and the test for it to handle this case.

[tomivirkki]

Would making the data generator run conditionally based on the column's visibility fix the original issue?
Proto: 1a5cb52

[ugur-vaadin]

Would making the data generator run conditionally based on the column's visibility fix the original issue? Proto: 1a5cb52

Adapted the suggestion. It works nicely and is cleaner.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud