InstantTimeProviderTest.testInstantCurrentTimeNanos is consistently failing

[ejona86]

Inside Google:

expected         : 1730533931622822000
but was          : 1730533931609013000
outside tolerance: 10000000
	at io.grpc.internal.InstantTimeProviderTest.testInstantCurrentTimeNanos(InstantTimeProviderTest.java:48)

Another arbitrary failure:

expected         : 1731147454282900000
but was          : 1731147454264246000

@vinodhabib
CC @kannanjgithub

[ejona86]

These failures are when running with --config=tsan, which runs slower.

[vinodhabib]

@ejona86 @shivaspeaks @kannanjgithub I went through the details and found like above scenarios mentioned in the issue details failed because the given tolerance time of 10 milliseconds exceeded in UT ( I,e first run/failure completed in 13 milliseconds and second completed in 18 milliseconds as mentioned below).

1730533931622822000 -> November 2, 2024 1:22:11.622 PM
1730533931609013000 -> November 2, 2024 1:22:11.609 PM
tolerance time - 13 milliseconds

1731147454282900000 -> November 9, 2024 3:47:34.282 PM
1731147454264246000 -> November 9, 2024 3:47:34.264 PM
tolerance time - 18 milliseconds

As per my Analysis we can update the tolerance value to 50 from 10 milliseconds to resolve this issue, Please confirm?

[kannanjgithub]

With 50ms it still failed 3 / 1000 times (Google internal test run)

With 75ms it still failed but only for 10 / 10000 (Google internal test run).

I think we can set the tolerance as 75ms.

[ejona86]

How about setting it to 100 ms or 1 second, so we don't deal with it again? 75 ms is already too large to notice precision issues, so I'm not sure what we gain by using 75 ms instead of 1 second.

[ejona86]

This recently flaked when not running with TSAN. http://sponge2/973b69a4-998c-457f-a496-c49bc8c3eef0

[kannanjgithub]

Yes, lets make it 1s. @vinodhabib