Assertion failures or unexpected errors in both the before() and after() block of a nested suite will cause the NEXT suite's before() blocks to be skipped

[grayson-poon]

Current behavior

Assertion failures or unexpected errors in both the before() and after() block will cause the NEXT suite's before() blocks to be skipped.

When doing a test run with Cypress v12.16.0, I had bad request failures (response code level of 400) in both the before() and after() blocks of a nested context() block (Context block #1). This caused the tests in Context block #1 to be to skipped (which was expected). However, the before() blocks of the next context() block (Context block #2) were skipped, causing the tests in that suite to also fail (this was not expected).

I have created a matching test structure to the tests where I found this issue and was able to replicate the scenario I'm describing above. See my examples of the test structure and console output.

I have also attached a log file of the debug output during a test run of only this file - cypress-issue-debug-output.log. In the logs, there is the following error message / stack trace:

Because this error occurred during a `after all` hook we are skipping the remaining tests in the current suite: `Context block #1`
    at Context.eval (cypress/e2e/api/families/test.cy.js:25:43)

which implies that only the blocks in Context block #1 should be affected by the errors.

Please let me know if this is expected behavior for Cypress, or if there is another active issue that is being tracked, as I was not able to find one.

Thanks!

Edit: Note that I initially encountered this error on Cypress v12.16.0, then upgraded my Cypress to the latest (v13.15.0) and saw the same issue.

Desired behavior

Cypress should treat nested suites as distinct entities. The failures of one suite shouldn't have any effect on another test suite's execution.

More specifically in this scenario, the before() blocks of Context block #2 should execute and not be skipped.

Test code to reproduce

In a new test file, copy and paste the following Cypress test code. Notice the actual console output below the code chunk.

describe("My Tests", () => {
    context("Context block #1", () => {
        before("Context block #1 - before block #1", () => {
            console.log("#1");

            cy.request("https://catfact.ninja/fact").then((response) => {
                expect(response.status).to.eq(400);
            });
        });

        before("Context block #1 - before block #2", () => {
            console.log("#2");

            expect(true).to.be.true;
        });

        it("Context block #1 - it block #1", () => {
            console.log("#3");

            expect(true).to.be.true;
        });

        it("Context block #1 - it block #2", () => {
            console.log("#4");

            expect(true).to.be.true;
        });

        after("Context block #1 - after block #1", () => {
            console.log("#5");

            cy.request("https://catfact.ninja/fact").then((response) => {
                // having a failed assertion or an unexpected error in both the before and after block of one suite will cause the next suite's before blocks to NOT run
                expect(response.status).to.eq(400);

                // try using this passing assertion, and notice the next suite's before blocks DO run
                expect(response.status).to.eq(200);
            });
        });
    });

    context("Context block #2", () => {
        let testString = "initial string";

        before("Context block #2 - before block #1", () => {
            console.log("#6");

            cy.request("https://catfact.ninja/fact").then((response) => {
                console.log(response);
                testString = "this string is assigned if the before block executes, but it does not";
            });
        });

        before("Context block #2 - before block #2", () => {
            console.log("#7");

            cy.request("https://catfact.ninja/fact").then((response) => {
                console.log(response);
                testString = "this string is assigned if the before block executes, but it does not";
            });

            testString = "this string is assigned if the before block executes, but it does not";
        });

        it("Context block #2 - it block #1", () => {
            console.log("#8");
            console.log("The value of initialString inside the it() blocks of Context block #2 is: " + testString);

            cy.contains(testString).should("exist");
        });
    });
});

Console output:

#1
#5
#8
The value of initialString inside the it() blocks of Context block #2 is: initial string

Cypress Version

13.15.0

Node version

18.15.0

Operating System

Windows 11

Debug Logs

Debug logs are attached as a .log file in the Current Behavior section.

Other

No response

[jennifer-shehane]

Yah, this is not expected imo. Simplified reproducible example:

context("Context block #1", () => {
		before("Context block #1 - before block #1", () => {
				console.log("#1") // PRINTS
				expect(true).to.be.false
		})

		before("Context block #1 - before block #2", () => {
				console.log("#2")  // DOESN'T PRINT
		})

		it("Context block #1 - it block #1", () => {
				console.log("#3") // DOESN'T PRINT
		})

		it("Context block #1 - it block #2", () => {
				console.log("#4") // DOESN'T PRINT
		})

		after("Context block #1 - after block #1", () => {
				console.log("#5") // PRINTS
				expect(true).to.be.false
		})
})

context("Context block #2", () => {
		before("Context block #2 - before block #1", () => {
				console.log("#6") // DOESN'T PRINT
		})

		before("Context block #2 - before block #2", () => {
				console.log("#7") // DOESN'T PRINT
		})

		it("Context block #2 - it block #1", () => {
				console.log("#8") // PRINTS
				expect(true).to.be.true
		})
})