Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mdoc guards to test-runtime.md. #4097

Open
wants to merge 2 commits into
base: series/3.5.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -1063,4 +1063,12 @@ lazy val stressTests = project
)
.enablePlugins(NoPublishPlugin, JCStressPlugin)

lazy val docs = project.in(file("site-docs")).dependsOn(core.jvm).enablePlugins(MdocPlugin)
lazy val docs = project
.in(file("site-docs"))
.dependsOn(core.jvm, testkit.jvm)
.enablePlugins(MdocPlugin)
.settings(
libraryDependencies += "org.typelevel" %% "munit-cats-effect" % "2.0.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately it would be bad practice add this dependency to the Cats Effect build. This will create a cycle: munit-cats-effect depends on cats-effect. So then we might not be able to update cats-effect without first updating munit-cats-effect, but we wouldn't be able to update munit-cats-effect without first updating cats-effect.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless we update first munit-cats-effect! 🧌

)


121 changes: 73 additions & 48 deletions docs/core/test-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ libraryDependencies += "org.typelevel" %% "cats-effect-testkit" % "3.5.4" % Test

For the remainder of this page, we will be writing tests which verify the behavior of the following function:

```scala mdoc
```scala mdoc:silent
import cats.effect.IO
import cats.effect.std.Random
import scala.concurrent.duration._
Expand All @@ -59,25 +59,30 @@ This is *exactly* the sort of functionality for which `TestControl` was built to

The first sort of test we will attempt to write comes in the form of a *complete* execution. This is generally the most common scenario, in which most of the power of `TestControl` is unnecessary and the only purpose to the mock runtime is to achieve deterministic and fast time:

```scala
test("retry at least 3 times until success") {
case object TestException extends RuntimeException
```scala mdoc:silent
import munit.CatsEffectSuite
import cats.effect.testkit.TestControl

var attempts = 0
val action = IO {
attempts += 1
class TestSuite extends CatsEffectSuite {
test("retry at least 3 times until success") {
case object TestException extends RuntimeException

if (attempts != 3)
throw TestException
else
"success!"
}
var attempts = 0
val action = IO {
attempts += 1

val program = Random.scalaUtilRandom[IO] flatMap { random =>
retry(action, 1.minute, 5, random)
}
if (attempts != 3)
throw TestException
else
"success!"
}

val program = Random.scalaUtilRandom[IO] flatMap { random =>
retry(action, 1.minute, 5, random)
}

TestControl.executeEmbed(program).assertEquals("success!")
TestControl.executeEmbed(program).assertEquals("success!")
}
}
```

Expand All @@ -95,33 +100,38 @@ For more advanced cases, `executeEmbed` may not be enough to properly measure th

Fortunately, `TestControl` provides a more general function, `execute`, which provides precisely the functionality needed to handle such cases:

```scala
test("backoff appropriately between attempts") {
case object TestException extends RuntimeException
```scala mdoc:nest:silent
import cats.syntax.all._
import cats.effect.Outcome

val action = IO.raiseError(TestException)
val program = Random.scalaUtilRandom[IO] flatMap { random =>
retry(action, 1.minute, 5, random)
}
class TestSuite extends CatsEffectSuite {
test("backoff appropriately between attempts") {
case object TestException extends RuntimeException

TestControl.execute(program) flatMap { control =>
for {
_ <- control.results.assertEquals(None)
_ <- control.tick
val action = IO.raiseError(TestException)
val program = Random.scalaUtilRandom[IO] flatMap { random =>
retry(action, 1.minute, 5, random)
}

_ <- 0.until(4) traverse { i =>
for {
_ <- control.results.assertEquals(None)
TestControl.execute(program) flatMap { control: TestControl[Random[IO]] =>
for {
_ <- control.results.assertEquals(None)
_ <- control.tick

interval <- control.nextInterval
_ <- IO(assert(interval >= 0.nanos))
_ <- IO(assert(interval < (1 << i).minute))
_ <- control.advanceAndTick(interval)
} yield ()
}
_ <- List.range(0, 4) traverse { i =>
for {
_ <- control.results.assertEquals(None)

_ <- control.results.assertEquals(Some(Outcome.failed(TestException)))
} yield ()
interval <- control.nextInterval
_ <- IO(assert(interval >= 0.nanos))
_ <- IO(assert(interval < (1 << i).minute))
_ <- control.advanceAndTick(interval)
} yield ()
}

_ <- control.results.assertEquals(Some(Outcome.errored[cats.Id, Throwable, Random[IO]](TestException)))
} yield ()
}
}
}
```
Expand All @@ -145,7 +155,13 @@ More usefully, we could ask what the `nextInterval` is. When all active fibers a

Since we know we're going to retry five times and ultimately fail (since the `action` never succeeds), we take advantage of `traverse` to write a simple loop within our test. For each retry, we test the following:

```scala
```scala mdoc:invisible
import munit.CatsEffectAssertions._
import cats.effect.unsafe.IORuntime
val control: TestControl[Random[IO]] = TestControl.execute(Random.scalaUtilRandom[IO]).unsafeRunSync()(IORuntime.global)
val i: Int = 0
```
```scala mdoc:silent
for {
_ <- control.results.assertEquals(None)

Expand All @@ -172,12 +188,18 @@ We finally have `results`, since the `program` will have terminated with an exce

As you might now expect, `executeEmbed` is actually implemented in terms of `execute`:

```scala
```scala mdoc:invisible
import cats.effect.unsafe.IORuntimeConfig
import cats.{Id, ~>}
import scala.concurrent.CancellationException
import TestControl.NonTerminationException
```
```scala mdoc:silent
def executeEmbed[A](
program: IO[A],
config: IORuntimeConfig = IORuntimeConfig(),
seed: Option[String] = None): IO[A] =
execute(program, config = config, seed = seed) flatMap { c =>
TestControl.execute(program, config = config, seed = seed) flatMap { c =>
val nt = new (Id ~> IO) { def apply[E](e: E) = IO.pure(e) }

val onCancel = IO.defer(IO.raiseError(new CancellationException()))
Expand All @@ -187,6 +209,9 @@ def executeEmbed[A](
c.tickAll *> embedded
}
```
```scala mdoc:invisible
executeEmbed(IO.unit)
```

If you ignore the messy `map` and `mapK` lifting within `Outcome`, this is actually a relatively simple bit of functionality. The `tickAll` effect causes `TestControl` to `tick` until a `sleep` boundary, then `advance` by the necessary `nextInterval`, and then repeat the process until either `isDeadlocked` is `true` or `results` is `Some`. These results are then retrieved and embedded within the outer `IO`, with cancelation and non-termination being reflected as exceptions.

Expand All @@ -196,7 +221,7 @@ It is very important to remember that `TestControl` is a *mock* runtime, and thu

To give an intuition for the type of program which behaves strangely under `TestControl`, consider the following pathological example:

```scala
```scala mdoc:silent
IO.cede.foreverM.start flatMap { fiber =>
IO.sleep(1.second) *> fiber.cancel
}
Expand All @@ -208,23 +233,23 @@ Under `TestControl`, this program will execute forever and never terminate. What

Another common pitfall with `TestControl` is the fact that you need to be careful to *not* advance time *before* a `IO.sleep` happens! Or rather, you are perfectly free to do this, but it probably won't do what you think it will do. Consider the following:

```scala
```scala mdoc:silent
TestControl.execute(IO.sleep(1.second) >> IO.realTime) flatMap { control =>
for {
_ <- control.advanceAndTick(1.second)
_ <- control.results.assertEquals(Some(Outcome.succeeded(1.second)))
_ <- control.results.assertEquals(Some(Outcome.succeeded[Id, Throwable, FiniteDuration](1.second)))
} yield ()
}
```

The above is very intuitive! Unfortunately, it is also wrong. The problem becomes a little clearer if we desugar `advanceAndTick`:

```scala
```scala mdoc:silent
TestControl.execute(IO.sleep(1.second) >> IO.realTime) flatMap { control =>
for {
_ <- control.advance(1.second)
_ <- control.tick
_ <- control.results.assertEquals(Some(Outcome.succeeded(1.second)))
_ <- control.results.assertEquals(Some(Outcome.succeeded[Id, Throwable, FiniteDuration](1.second)))
} yield ()
}
```
Expand All @@ -233,13 +258,13 @@ We're instructing `TestControl` to advance the clock *before* we `sleep`, and th

The solution is to add an additional `tick` to execute the "beginning" of the program (from the start up until the `sleep`(s)):

```scala
```scala mdoc:silent
TestControl.execute(IO.sleep(1.second) >> IO.realTime) flatMap { control =>
for {
_ <- control.tick
_ <- control.advance(1.second)
_ <- control.tick
_ <- control.results.assertEquals(Some(Outcome.succeeded(1.second)))
_ <- control.results.assertEquals(Some(Outcome.succeeded[Id, Throwable, FiniteDuration](1.second)))
} yield ()
}
```
Expand Down
Loading