Flaky tests

[Aerijo]

Description

Over at least the last few days a lot of unrelated PRs to Atom have been failing CI due to the following test failures on the macOS test job. I have not been able to reproduce the failures locally after several tries (macOS 10.15.6). This issue is intended to document any findings.

CI runs

• https://github.visualstudio.com/Atom/_build/results?buildId=81464&view=logs&j=6de74252-975b-522d-7a3d-227ca79ca6c1&t=c47046dc-f6da-5780-3e71-47228bf0ade3
• https://github.visualstudio.com/Atom/_build/results?buildId=81455&view=logs&j=6de74252-975b-522d-7a3d-227ca79ca6c1&t=c47046dc-f6da-5780-3e71-47228bf0ade3
• https://github.visualstudio.com/Atom/_build/results?buildId=81347&view=logs&j=6de74252-975b-522d-7a3d-227ca79ca6c1&t=c47046dc-f6da-5780-3e71-47228bf0ade3
• https://github.visualstudio.com/Atom/_build/results?buildId=81284&view=logs&j=6de74252-975b-522d-7a3d-227ca79ca6c1&t=c47046dc-f6da-5780-3e71-47228bf0ade3
• https://github.visualstudio.com/Atom/_build/results?buildId=81294&view=logs&j=6de74252-975b-522d-7a3d-227ca79ca6c1&t=c47046dc-f6da-5780-3e71-47228bf0ade3&l=509
• https://github.visualstudio.com/Atom/_build/results?buildId=81308&view=logs&j=6de74252-975b-522d-7a3d-227ca79ca6c1&t=c47046dc-f6da-5780-3e71-47228bf0ade3

Unique failures

ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:307:71)

ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)

ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the path when when core:move-to-bottom is triggered and last item is collapsed
      Expected '<li class="list-nested-item"><div class="list-item path-row" data-file-path="/Users/runner/work/1/s/node_modules/find-and-replace/spec/fixtures/project/sample.coffee"><span data-name="sample.coffee" class="icon-file-text icon"></span><span class="path-name bright">sample.coffee</span><span class="path-match-number">(5 matches)</span></div></li>' to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)

ProjectFindView (ripgrep=true)
  panel focus
    it ProjectFindView (ripgrep=true) panel focus focuses the find editor when the panel gets focus
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1544:55)

ProjectFindView (ripgrep=true)
  panel focus
    it ProjectFindView (ripgrep=true) panel focus moves focus between editors with find-and-replace:focus-next
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1555:51)

ProjectFindView (ripgrep=true)
  panel focus
    it ProjectFindView (ripgrep=true) panel focus moves focus between editors with find-and-replace:focus-next
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1564:53)

ProjectFindView (ripgrep=true)
  finding
    when the find string contains an escaped char
      when regex search is disabled
        it ProjectFindView (ripgrep=true) finding when the find string contains an escaped char when regex search is disabled finds the escape char
          Expected object with length 0 to have length 1
            at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:316:84)
            at Generator.next (<anonymous>)
            at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:3:273)

Raw

ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:307:71)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)
    it ResultsView core:move-to-top and core:move-to-bottom selects the path when when core:move-to-bottom is triggered and last item is collapsed
      Expected '<li class="list-nested-item"><div class="list-item path-row" data-file-path="/Users/runner/work/1/s/node_modules/find-and-replace/spec/fixtures/project/sample.coffee"><span data-name="sample.coffee" class="icon-file-text icon"></span><span class="path-name bright">sample.coffee</span><span class="path-match-number">(5 matches)</span></div></li>' to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)

ProjectFindView (ripgrep=true)
  panel focus
    it ProjectFindView (ripgrep=true) panel focus focuses the find editor when the panel gets focus
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1544:55)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
    it ProjectFindView (ripgrep=true) panel focus moves focus between editors with find-and-replace:focus-next
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1552:53)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1555:51)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1564:53)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)

ProjectFindView (ripgrep=true)
  finding
    when the find string contains an escaped char
      when regex search is disabled
        it ProjectFindView (ripgrep=true) finding when the find string contains an escaped char when regex search is disabled finds the escape char
          Expected object with length 0 to have length 1
            at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:316:84)
            at Generator.next (<anonymous>)
            at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:3:273)
  panel focus
    it ProjectFindView (ripgrep=true) panel focus moves focus between editors with find-and-replace:focus-next
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1552:53)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1555:51)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1564:53)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)

ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:307:71)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)
    it ResultsView core:move-to-top and core:move-to-bottom selects the path when when core:move-to-bottom is triggered and last item is collapsed
      Expected '<li class="list-nested-item"><div class="list-item path-row" data-file-path="/Users/runner/work/1/s/node_modules/find-and-replace/spec/fixtures/project/sample.coffee"><span data-name="sample.coffee" class="icon-file-text icon"></span><span class="path-name bright">sample.coffee</span><span class="path-match-number">(5 matches)</span></div></li>' to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)

ProjectFindView (ripgrep=true)
  panel focus
    it ProjectFindView (ripgrep=true) panel focus focuses the find editor when the panel gets focus
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1544:55)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
    it ProjectFindView (ripgrep=true) panel focus moves focus between editors with find-and-replace:focus-next
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1552:53)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1555:51)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
      Expected element '[object HTMLElement]' or its descendants to have focus.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/project-find-view-spec.js:1564:53)
        at jasmine.Spec.args.<computed> (/Users/runner/work/1/s/spec/jasmine-test-runner.coffee:89:27)
ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:307:71)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)
    it ResultsView core:move-to-top and core:move-to-bottom selects the path when when core:move-to-bottom is triggered and last item is collapsed
      Expected '<li class="list-nested-item"><div class="list-item path-row" data-file-path="/Users/runner/work/1/s/node_modules/find-and-replace/spec/fixtures/project/sample.coffee"><span data-name="sample.coffee" class="icon-file-text icon"></span><span class="path-name bright">sample.coffee</span><span class="path-match-number">(5 matches)</span></div></li>' to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)

ResultsView
  core:move-to-top and core:move-to-bottom
    it ResultsView core:move-to-top and core:move-to-bottom selects the first/last item when core:move-to-top/move-to-bottom is triggered
      Expected undefined to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:307:71)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)
    it ResultsView core:move-to-top and core:move-to-bottom selects the path when when core:move-to-bottom is triggered and last item is collapsed
      Expected '<li class="list-nested-item"><div class="list-item path-row" data-file-path="/Users/runner/work/1/s/node_modules/find-and-replace/spec/fixtures/project/sample.coffee"><span data-name="sample.coffee" class="icon-file-text icon"></span><span class="path-name bright">sample.coffee</span><span class="path-match-number">(5 matches)</span></div></li>' to have class 'selected'.
        at jasmine.Spec.<anonymous> (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:320:101)
        at Generator.next (<anonymous>)
        at step (/Users/runner/work/1/s/node_modules/find-and-replace/spec/results-view-spec.js:1:428)