Unable to disable PA (using cluster API) during startup

[sruti1312]

During startup, cluster API to disable PA does not work.

This is a problem only during startup. After re-enabling PA (cluster_state=3), the disable option for PA works fine.

[ec2-user@ip-10-212-47-103 logs]$ curl localhost:9200/_opendistro/_performanceanalyzer/cluster/config -H 'Content-Type: application/json' -d '{"enabled": false}'
{"currentPerformanceAnalyzerClusterState":3,"shardsPerCollection":0,"batchMetricsRetentionPeriodMinutes":7}
[ec2-user@ip-10-212-47-103 logs]$ curl -XGET localhost:9200/_opendistro/_performanceanalyzer/cluster/config
{"currentPerformanceAnalyzerClusterState":3,"shardsPerCollection":0,"batchMetricsRetentionPeriodMinutes":7}

From logs:

[2020-10-30T16:05:08,008][INFO ][o.e.c.s.ClusterSettings  ] [c4d4a248a38f4230f583d5cd1cf475ee] updating [cluster.metadata.perf_analyzer.state] from [] to [0]

[ec2-user@ip-10-212-47-103 logs]$ curl -s 'localhost:9200/_cluster/settings?pretty&include_defaults&flat_settings' | grep perf
  "cluster.metadata.perf_analyzer.state" : "0",
  "cluster.metadata.perf_analyzer.config.overrides" : "",
  "cluster.metadata.perf_analyzer.pa_node_stats_setting" : "1",

Intermittent solution:

1. Disabling RCA followed by disabling PA works.
2. Use Node API's.