[6.5]retry as stale reads and remove back off for server busy errors

[mittalrishabh]

It contains two changes

1. Backport of this PR retry as stale read on replica #1509.
2. Remove back off for server busy errors. It is already there in master.
3. Try reads as stale reads if leader is not accessible

Unit test

1. Injected skewed traffic and confirmed that reads are not retried as follower read by looking at read index messages.

[ti-chi-bot]

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.

📝 Please follow instructions in the contributing guide to update your commits with the DCO

Full details of the Developer Certificate of Origin can be found at developercertificate.org.

The list of commits missing DCO signoff:

• 0832080 retry as stale reads
• 6b4a6b5 build failure
• 6b81169 fixed test cases
• ec97c19 allow stale read if leader is not even tried
• 1e34922 fixed test cases
• ba8c21b disable test
• 7fe8732 fixed test cases

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[ti-chi-bot]

@cfzjywxk: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ti-chi-bot]

@crazycs520: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: cfzjywxk, crazycs520

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mittalrishabh]

I don't have access to merge. Please merge it.