fix(native filters): rendering performance improvement by reduce overrendering

[justinpark]

SUMMARY

Following up #25282, rendering performance can still be improved by reducing unnecessary rerenders (also known as over-rendering).
This commit improves performance in four areas.

1. Inefficient uniqWith and object equality check in uniqueOptions

superset/superset-frontend/src/filters/components/Select/SelectFilterPlugin.tsx

Lines 221 to 224 in d6fde3c

	const uniqueOptions = useMemo(() => {
	const allOptions = [...data];
	return uniqWith(allOptions, isEqual).map(row => {
	const [value] = groupby.map(col => row[col]);

To get the list of unique values, it compares the entire data list (arrayIncludesWith) using object equality comparison (isEqual). This is slow because the data contains several attributes, and object equality comparison is slow.

As it shown in the above metric, uniqueOptions operation triggers four times every time a filter component re-renders, which takes more than 2 seconds per re-render for 10 filters (i.e. 50ms * 4 * 10 filters = 2s).

Using a hashmap approach (with unnecessary equality check) reduced the delay from ~200m(=50ms * 4) to 10ms (95% reduction)

2. Redundant filter re-renders due to the dynamic sizing

Initially renders at a minimum width and then expands to its full width.

Since the native filter's allocated to 100% of its width using SuperChart, which is wrapped by @vx/responsive, it triggers re-renders every time the screen size is updated.
This commit removes the resize observer (by assigning the static width) and updates the filter component wrapper to the full width using CSS instead. As a result, the filters no longer re-render when the page size changes.

3. descendants over-rendering

React's inherent behavior of rerendering all descendant components, coupled with the Redux container's rerendering upon state changes, leads to excessive rerendering of dashboard components, including filter components, whenever the chart fetch status updates the Redux store.
This commit replaces DashboardBuilder to the identical descendant (props.children) which prevents the rerenders when the container component rerenders.
To further optimize performance, the dashboard state synchronization logic was moved to a separate component, effectively decoupling it from the DashboardPage component and preventing unnecessary rerenders of the DashboardPage.

4. Redundant __cache attribute

Due to superfluous setDataMask calls triggered by __cache update, each setState update causes four unnecessary uniqWith executions

An unnecessary __cache value was introduced in the SelectFilterPlugin in PR#14869, but it was never utilized and resulted in superfluous rerenders of filter components.
This commit removes this redundant __cache value to resolve the problem.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Successfully optimized the application's performance, resulting in a 10% decrease in rendering count (255 to 228) and a remarkable 60% reduction in loading time (10.9 seconds to 4.5 seconds).

Before	After

TESTING INSTRUCTIONS

1. Go to any dashboard
2. Add more than 10 select filters
3. Reload dashboard
4. See that filters take a lot of time for before render.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[rusackas]

CC @kgabryje who has looked into this sort of thing in this area before.

[kgabryje]

Shouldn't this be a hook rather than a component?

[justinpark]

The reason is related to the topic no.3 3. descendants over-rendering

To further optimize performance, the dashboard state synchronization logic was moved to a separate component, effectively decoupling it from the DashboardPage component and preventing unnecessary rerenders of the DashboardPage.

• In order to observe state updates and sync them to local storage, it is necessary to include a component with the hook. The most effective approach would be to implement a middleware. In this case, it is skipped to minimize any potential regression.

[kgabryje]

I see, thank you for explanation!

[kgabryje]

What's the advantage of using this RESPONSIVE_WIDTH instead of 100%?

[justinpark]

The RESPONSE_WIDTH change is related to the topic no.2 (2. Redundant filter re-renders due to the dynamic sizing)

[kgabryje]

Amazing improvement! Left 2 minor comments for the code.
As often is the case in such refactors, there's a high risk of introducing regressions. Is there any chance we could get some QA help @sadpandajoe @jinghua-qa ?

[kgabryje]

/testenv up

[github-actions]

@kgabryje Ephemeral environment spinning up at http://35.87.180.132:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

[jinghua-qa]

Amazing improvement! Left 2 minor comments for the code. As often is the case in such refactors, there's a high risk of introducing regressions. Is there any chance we could get some QA help @sadpandajoe @jinghua-qa ?

I will take a look for the regression

[jinghua-qa]

Do some regression test around native filter , did not find issues, LGTM

[kgabryje]

Thanks @jinghua-qa! @justinpark Would you rather address my comments in this PR or merge it and do a follow up if needed?

[justinpark]

Thanks jinghua-qa! justinpark Would you rather address my comments in this PR or merge it and do a follow up if needed?

@kgabryje please see my comments.

[kgabryje]

LGTM

[github-actions]

Ephemeral environment shutdown and build artifacts deleted.