Timeout Issue with Image Pulling in Micronaut-Test-Resources in GitHub Actions

[willdickerson]

Expected Behavior

Docker images are pulled without causing timeouts.

Actual Behaviour

Pulling of certain images takes longer than expected, leading to build timeouts. We noticed this consistently with the confluentinc/cp-kafka:7.0.4 image. Example error stack trace:

  Message: Test resources service wasn't able to revolve expression 'kafka.bootstrap.servers': Failed to get Docker client for confluentinc/cp-kafka:7.0.4
  Path Taken: new KafkaConsumerProcessor(ExecutorService executorService,ApplicationConfiguration applicationConfiguration,BeanContext beanContext,AbstractKafkaConsumerConfiguration defaultConsumerConfiguration,ConsumerRecordBinderRegistry binderRegistry,BatchConsumerRecordsBinderRegistry batchBinderRegistry,SerdeRegistry serdeRegistry,ProducerRegistry producerRegistry,KafkaListenerExceptionHandler exceptionHandler,ExecutorService schedulerService,TransactionalProducerRegistry transactionalProducerRegistry,ApplicationEventPublisher startedEventPublisher,ApplicationEventPublisher subscribedEventPublisher,ConditionalRetryBehaviourHandler conditionalRetryBehaviourHandler) --> new KafkaConsumerProcessor(ExecutorService executorService,ApplicationConfiguration applicationConfiguration,BeanContext beanContext,[AbstractKafkaConsumerConfiguration defaultConsumerConfiguration],ConsumerRecordBinderRegistry binderRegistry,BatchConsumerRecordsBinderRegistry batchBinderRegistry,SerdeRegistry serdeRegistry,ProducerRegistry producerRegistry,KafkaListenerExceptionHandler exceptionHandler,ExecutorService schedulerService,TransactionalProducerRegistry transactionalProducerRegistry,ApplicationEventPublisher startedEventPublisher,ApplicationEventPublisher subscribedEventPublisher,ConditionalRetryBehaviourHandler conditionalRetryBehaviourHandler) --> new DefaultKafkaConsumerConfiguration([KafkaDefaultConfiguration defaultConfiguration])
      at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2326)
      at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2281)
      at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2293)
      at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3095)
      at app//io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
      at app//io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2997)
      at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2958)
      at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2732)
      at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1731)
      at app//io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:89)
      at app//io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2161)
      at app//io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1324)
      at app//io.micronaut.configuration.kafka.config.$DefaultKafkaConsumerConfiguration$Definition.instantiate(Unknown Source)
      at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2311)
      at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2281)
      at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2293)
      at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3095)
      at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2985)
      at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2732)
      at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1731)
      at app//io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:89)
      at app//io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2161)
      at app//io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1324)
      at app//io.micronaut.configuration.kafka.processor.$KafkaConsumerProcessor$Definition.instantiate(Unknown Source)
      at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2311)
      at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2281)
      at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2293)
      at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3095)
      at app//io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
      at app//io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2997)
      at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2958)
      at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2932)
      at app//io.micronaut.context.DefaultBeanContext.addCandidateToList(DefaultBeanContext.java:3521)
      at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistrations(DefaultBeanContext.java:3476)
      at app//io.micronaut.context.DefaultBeanContext.getBeanRegistrations(DefaultBeanContext.java:3450)
      at app//io.micronaut.context.DefaultBeanContext.streamOfType(DefaultBeanContext.java:980)
      at app//io.micronaut.context.DefaultBeanContext.streamOfType(DefaultBeanContext.java:906)
      at app//io.micronaut.context.DefaultBeanContext.streamOfType(DefaultBeanContext.java:883)
      at app//io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:2020)
      at app//io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:294)
      at app//io.micronaut.context.DefaultBeanContext.readAllBeanDefinitionClasses(DefaultBeanContext.java:3348)
      at app//io.micronaut.context.DefaultBeanContext.finalizeConfiguration(DefaultBeanContext.java:3692)
      at app//io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:342)
      at app//io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:202)
      at app//io.micronaut.test.extensions.AbstractMicronautExtension.startApplicationContext(AbstractMicronautExtension.java:507)
      at app//io.micronaut.test.extensions.AbstractMicronautExtension.beforeClass(AbstractMicronautExtension.java:346)
      at app//io.micronaut.test.extensions.kotest5.MicronautKotest5Context.<init>(MicronautKotest5Context.kt:41)
      at app//io.micronaut.test.extensions.kotest5.MicronautKotest5Extension.instantiate(MicronautKotest5Extension.kt:110)
      at app//io.kotest.engine.spec.InstantiateSpecKt.createAndInitializeSpec(instantiateSpec.kt:30)
      at app//io.kotest.engine.spec.InstantiateSpecKt.instantiate(instantiateSpec.kt:11)
      at app//io.kotest.engine.spec.SpecRefKt.instance(SpecRef.kt:14)
      at app//io.kotest.engine.spec.SpecExecutor.createInstance-gIAlu-s(SpecExecutor.kt:63)
      at app//io.kotest.engine.spec.SpecExecutor.access$createInstance-gIAlu-s(SpecExecutor.kt:24)
      at app//io.kotest.engine.spec.SpecExecutor$execute$innerExecute$1.invokeSuspend(SpecExecutor.kt:40)
      at app//io.kotest.engine.spec.SpecExecutor$execute$innerExecute$1.invoke(SpecExecutor.kt)
      at app//io.kotest.engine.spec.SpecExecutor$execute$innerExecute$1.invoke(SpecExecutor.kt)
      at app//io.kotest.engine.spec.interceptor.ref.FinalizeSpecInterceptor.intercept-0E7RQCE(FinalizeSpecInterceptor.kt:24)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.BeforeSpecStateInterceptor.intercept-0E7RQCE(BeforeSpecStateInterceptor.kt:25)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.PrepareSpecInterceptor.intercept-0E7RQCE(PrepareSpecInterceptor.kt:25)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.ApplyExtensionsInterceptor.intercept-0E7RQCE(ApplyExtensionsInterceptor.kt:36)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.SpecFinishedInterceptor.intercept-0E7RQCE(SpecFinishedInterceptor.kt:22)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.SpecStartedInterceptor.intercept-0E7RQCE(SpecStartedInterceptor.kt:20)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.SpecRefExtensionInterceptor$intercept$inner$1.invokeSuspend(SpecRefExtensionInterceptor.kt:28)
      at app//io.kotest.engine.spec.interceptor.ref.SpecRefExtensionInterceptor$intercept$inner$1.invoke(SpecRefExtensionInterceptor.kt)
      at app//io.kotest.engine.spec.interceptor.ref.SpecRefExtensionInterceptor$intercept$inner$1.invoke(SpecRefExtensionInterceptor.kt)
      at app//io.kotest.engine.spec.interceptor.ref.SpecRefExtensionInterceptor.intercept-0E7RQCE(SpecRefExtensionInterceptor.kt:31)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.RequiresTagInterceptor.intercept-0E7RQCE(RequiresTagInterceptor.kt:35)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.TagsInterceptor.intercept-0E7RQCE(TagsInterceptor.kt:38)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.SystemPropertySpecFilterInterceptor.intercept-0E7RQCE(SystemPropertySpecFilterInterceptor.kt:48)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.SpecFilterInterceptor.intercept-0E7RQCE(SpecFilterInterceptor.kt:38)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.IgnoredSpecInterceptor.intercept-0E7RQCE(IgnoredSpecInterceptor.kt:51)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.EnabledIfInterceptor.intercept-0E7RQCE(EnabledIfInterceptor.kt:41)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.ref.RequiresPlatformInterceptor.intercept-0E7RQCE(RequiresPlatformInterceptor.kt:32)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invokeSuspend(SpecRefInterceptorPipeline.kt:49)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline$execute$3$1.invoke(SpecRefInterceptorPipeline.kt)
      at app//io.kotest.engine.spec.interceptor.SpecRefInterceptorPipeline.execute-0E7RQCE(SpecRefInterceptorPipeline.kt:50)
      at app//io.kotest.engine.spec.SpecExecutor.execute(SpecExecutor.kt:42)
      at app//io.kotest.engine.ConcurrentTestSuiteScheduler$schedule$8$1$1$2.invokeSuspend(ConcurrentTestSuiteScheduler.kt:72)
      at app//kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
      at app//kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
      at app//kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:280)
      at app//kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
      at app//kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
      at app//kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
      at app//kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
      at app//kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
      at app//io.kotest.common.RunBlockingKt.runBlocking(runBlocking.kt:3)
      at app//io.kotest.engine.TestEngineLauncher.launch(TestEngineLauncher.kt:253)
      at app//io.kotest.runner.junit.platform.KotestJunitPlatformTestEngine.execute(KotestJunitPlatformTestEngine.kt:84)
      at app//io.kotest.runner.junit.platform.KotestJunitPlatformTestEngine.execute(KotestJunitPlatformTestEngine.kt:54)
  Caused by: io.micronaut.testresources.client.TestResourcesException: Test resources service wasn't able to revolve expression 'kafka.bootstrap.servers': Failed to get Docker client for confluentinc/cp-kafka:7.0.4
      at app//io.micronaut.testresources.client.TestResourcesClientPropertyExpressionResolver$DelegateResolver.withErrorHandling(TestResourcesClientPropertyExpressionResolver.java:130)
      at app//io.micronaut.testresources.client.TestResourcesClientPropertyExpressionResolver$DelegateResolver.callClient(TestResourcesClientPropertyExpressionResolver.java:112)
      at app//io.micronaut.testresources.client.TestResourcesClientPropertyExpressionResolver$DelegateResolver.resolve(TestResourcesClientPropertyExpressionResolver.java:99)
      at app//io.micronaut.testresources.core.LazyTestResourcesExpressionResolver.resolve(LazyTestResourcesExpressionResolver.java:45)
      at app//io.micronaut.context.env.DefaultPropertyPlaceholderResolver.resolveExpression(DefaultPropertyPlaceholderResolver.java:209)
      at app//io.micronaut.context.env.DefaultPropertyPlaceholderResolver$PlaceholderSegment.getValue(DefaultPropertyPlaceholderResolver.java:381)
      at app//io.micronaut.context.env.DefaultPropertyPlaceholderResolver.resolveRequiredPlaceholdersObject(DefaultPropertyPlaceholderResolver.java:116)
      at app//io.micronaut.context.env.PropertySourcePropertyResolver.resolvePlaceHoldersIfNecessary(PropertySourcePropertyResolver.java:791)
      at app//io.micronaut.context.env.PropertySourcePropertyResolver.resolveSubMap(PropertySourcePropertyResolver.java:561)
      at app//io.micronaut.context.env.PropertySourcePropertyResolver.getProperties(PropertySourcePropertyResolver.java:293)
      at app//io.micronaut.core.value.PropertyResolver.getProperties(PropertyResolver.java:113)
      at app//io.micronaut.configuration.kafka.config.KafkaDefaultConfiguration.resolveDefaultConfiguration(KafkaDefaultConfiguration.java:82)
      at app//io.micronaut.configuration.kafka.config.KafkaDefaultConfiguration.<init>(KafkaDefaultConfiguration.java:54)
      at app//io.micronaut.configuration.kafka.config.$KafkaDefaultConfiguration$Definition.instantiate(Unknown Source)
      at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2311)
      ... 129 more

Steps To Reproduce

1. Trigger a pipeline in GitHub Actions:

name: Build/Test

on: push

concurrency:
    group: ${{ github.ref }}
    cancel-in-progress: true

jobs:
    build-test:
        runs-on: ubuntu-20.04
        env:
            SERVICE_REPO: 'my-service'
            DOCKERFILE_PATH: 'docker/Dockerfile'
        steps:
            - name: Setup Java
              uses: actions/setup-java@v4
              with:
                  java-version: 17
                  distribution: adopt
            - name: Start Test Resources
              uses: gradle/actions/setup-gradle@v3
              with:
                  arguments: startTestResourcesService
            - name: Assemble Test
              uses: gradle/actions/setup-gradle@v3
              with:
                  arguments: assemble test --no-daemon --fail-fast

2. Monitor the Assemble Test step for delays or timeouts related to Docker image pulling required for testing.

Environment Information

No response

Example Application

No response

Version

4.4.3

[22dasi1bwi]

I'm having the same problem (on my local machine), running on Micronaut 4.3.7. Tests with other images (like Postgres for example) are working fine though.

Interesting enough: my CI on Github is running just fine.