Cucumber task will time out when iterating over a large list of items

[hammzj]

Current behavior

I have a test which validates a website that displays a configuration page of feature flags for a given organization. These feature flags are displayed as a list of divs that include a title, description, and value. In one example, there are 90 items I need to validate in a single step, using the backend contents and checking that they're accurate on the webpage.

Whenever I try to validate these through cypress open, the testrunner freezes for a slight bit, will complete the validation, but fail due to an error with a Cypress task not being completed in 60 seconds.

Testrunner output

Test error

Desired behavior

Add ability to set the task timeout of the task either using a Cypress configuration option or environment variable.

Secondly, allow debugging to be enabled for the Cypress Cucumber internal tasks.

Test code to reproduce

Note: this test case can cause problems with Chrome and crash the browser occasionally! As well, the length of the list of items may need to changed based on your computer's internal memory. I'm able to recreate this without crashing on a MacOS with Ventura 13.6 and 16GB of RAM at 70 iterations, but Chrome crashes above that.

I have a test project with a branch here that can be forked: https://github.com/hammzj/cypress-testing/tree/dev/cypress-issue-task-timeout

Versions

• Cypress version: 12.12.0
• Preprocessor version: 18.0.2 (also on 18.0.6)
• Node version: 18.16.0

Checklist

• I've read the FAQ.
• I've read instructions for logging issues.
• I'm not using [email protected] (package name has changed and it is no longer the most recent version, see #689).

[hammzj]

I do see that adding a small cy.wait in the for each block seems to avoid the testrunner freezing as frequently and causing the task to timeout, so that is definitely a way to avoid this issue without code changes. However, this shows that Cypress has issues with many sequential commands that do not have a gap between them.

[badeball]

The task seen failing isn't run unless messages or pretty reporting is turned on, which I can't see that it is in your repository. Can you make the necessary changes and confirm that you see the error using the code you're providing me?

[hammzj]

It had been happening earlier, but I'll see about making those changes. This might be intermittent due to it relying on hardware architecture as well.

However, based on my proposed solution, is it possible to allow configuring the timeout of Cucumber tasks from an environment variable instead of it being hard set at 60 seconds?

[badeball]

You can configure task timeouts using taskTimeout. However, I'm not going to make any changes to the library until it is evident that it actually is a task timeout happening as opposed to Cypress misreporting where time is spent (this has come up before).

[hammzj]

Is taskTimeout configurable via a tag pattern, like in the test-configuration.md? I could utilize that, though I am finding a short 100 ms wait is helping mitigate these problems.

I am still working on a more consistent reproducible example as well.

[badeball]

Is taskTimeout configurable via a tag pattern, like in the test-configuration.md? I could utilize that, though I am finding a short 100 ms wait is helping mitigate these problems.

No, I don’t think so. The tag pattern is merely a bridge to test configuration as described here: https://docs.cypress.io/guides/references/configuration#Test-Configuration

This library doesn’t contain any knowledge of valid tags and will simply propagate “anything”. If Cypress doesn’t recognise a property, I suppose it will silently ignored it, albeit unrecognised tags aren’t tested so I’m not entirely sure.

[hammzj]

Right, I've understood that but not so much as what the preprocessor offers in terms of abstracting tags to the Cypress configuration.

In this case, would it be possible that a new tag is added for @taskTimeout(X) in lines with how @defaultCommandTimeout(X) and others operate? That would work best in fixing this issue until cypress-io/cypress#25326 is resolved. I am going to make a note there as well on the slowdown.

[badeball]

In this case, would it be possible that a new tag is added for @taskTimeout(X) in lines with how @defaultCommandTimeout(X) and others operate?

No. As I tried to say, this library doesn’t concern itself with what tags you write, it merely translates them. You can write your above-mentioned tag but it’s just not going to do anything, because it’s not a recognised test-specific property by Cypress.

[hammzj]

It is possible to set the taskTimeout with a tag 😄 :

Feature

Testrunner

The property for taskTimeout is included in Cypress.TestConfigOverrides which is returned within the function tagToCypressOptions. Only thing that will need updated is the markdown doc for "test-configuration".

[badeball]

Only thing that will need updated is the markdown doc for "test-configuration".

No, That page is only intended to be a mirror of Cypress' documentation. If it's wrong, then Cypress' documentation is wrong. If you have found that there's additional, undocumented configuration options available, then you need to direct your attention appropriately.

[hammzj]

I would have been happy to make that change in the markdown document as much as I'm happy to address it with Cypress. There's always room for clarity, however, and there's definitely more encouraging ways to address developers raising attention to issues without negative reinforcement.

[hammzj]

cypress-io/cypress-documentation#5532

[hammzj]

Documentation has been updated on Cypress' side with the PR listed above!

Edit: I meant to add that I could open a PR here to update the docs with that tag since the documentation has been updated on Cypress' side. Not to be persistent, but it is a good callout in case others need to toggle that option per scenario. Let me know if that works!

[badeball]

I managed to reproduce the issue eventually. Furthermore, I managed to reproduce it without using the preprocessor, as seen here. In other words, it has nothing to with the preprocessor. The error also appears to disappear with Cypress v13.0.0. Considering this resolved, thus closing.