Tso client is not ready after create PD client

[okJiang]

Enhancement Task

If getClusterInfo fails due to the network problem, the tso client will not be initiated. In this way, if the caller calls tso related API, like GetMinTS(), it will report tso client is nil error.

pd/client/servicediscovery/pd_service_discovery.go

Lines 512 to 514 in 5ad4301

	if err := c.checkServiceModeChanged(); err != nil {
	log.Warn("[pd] failed to check service mode and will check later", zap.Error(err))
	}

pd/client/servicediscovery/pd_service_discovery.go

Lines 857 to 883 in 5ad4301

	func (c *pdServiceDiscovery) checkServiceModeChanged() error {
	leaderURL := c.getLeaderURL()
	if len(leaderURL) == 0 {
	return errors.New("no leader found")
	}
	clusterInfo, err := c.getClusterInfo(c.ctx, leaderURL, c.option.Timeout)
	if err != nil {
	if strings.Contains(err.Error(), "Unimplemented") {
	// If the method is not supported, we set it to pd mode.
	// TODO: it's a hack way to solve the compatibility issue.
	// we need to remove this after all maintained version supports the method.
	if c.serviceModeUpdateCb != nil {
	c.serviceModeUpdateCb(pdpb.ServiceMode_PD_SVC_MODE)
	}
	return nil
	}
	return err
	}
	if clusterInfo == nil || len(clusterInfo.ServiceModes) == 0 {
	return errors.WithStack(errs.ErrNoServiceModeReturned)
	}
	if c.serviceModeUpdateCb != nil {
	c.serviceModeUpdateCb(clusterInfo.ServiceModes[0])
	}
	return nil
	}

We can apply retry to this function.

[okJiang]

related issue:

• add index failed with error "Error 1105 (HY000): get TSO failed, tso client is nil" when injection pd leader network fault pingcap/tidb#55248
• br restore doesn't tolerate error "[PD:client:ErrClientGetTSO]get TSO failed, tso client is nil" pingcap/tidb#58239