Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github actions forces stdin through a fifo #10959

Open
4 of 15 tasks
tymscar opened this issue Nov 13, 2024 · 7 comments
Open
4 of 15 tasks

Github actions forces stdin through a fifo #10959

tymscar opened this issue Nov 13, 2024 · 7 comments

Comments

@tymscar
Copy link

tymscar commented Nov 13, 2024

Description

When using GitHub actions, there is an ever-present fifo on stdin for the programs running there, and sometimes, that makes those programs misbehave because their heuristic tells them to use the empty fifo instead of the files in the current working directory.

Platforms affected

  • Azure DevOps
  • GitHub Actions - Standard Runners
  • GitHub Actions - Larger Runners

Runner images affected

  • Ubuntu 20.04
  • Ubuntu 22.04
  • Ubuntu 24.04
  • macOS 12
  • macOS 13
  • macOS 13 Arm64
  • macOS 14
  • macOS 14 Arm64
  • macOS 15
  • macOS 15 Arm64
  • Windows Server 2019
  • Windows Server 2022

Image version and build link

I have recreated the simplest example of the bug happening, and you can see it in action here: https://github.com/tymscar/github-actions-test/actions/runs/11797546880/job/32861766821

The main line that is relevant to the discussion is on that page, part of the debug output of rg:
rg: DEBUG|grep_cli|crates/cli/src/lib.rs:209: for heuristic stdin detection on Unix, found that is_file=false, is_fifo=true and is_socket=false, and thus concluded that is_stdin_readable=true

This has been reported to ripgrep too, and it was suggested that it's a GHA problem, after which I tried to replicate it as lean as possible.

Is it regression?

no

Expected behavior

When running a binary such as rg "hello" in GitHub Actions, I expect it to search for the string in the local files.

Actual behavior

In actuality, there is an ever-present empty fifo that tricks the heuristic of ripgrep (and other binaries) into thinking it should look at stdin instead of the default behavior.

Repro steps

Run any binary that has a heuristic about when to use stdin vs local files, such as ripgrep, and if it's not told explicitly to search for local files, it will default to the empty fifo. The best example is ripgrep, because when run with debug like rg --debug "hello" it will print that it found an open fifo on stdin from GitHub Actions. Locally on another machine, or in a vm, there is no present fifo, just in GHA.

@vidyasagarnimmagaddi
Copy link
Contributor

Hi @tymscar

We have observed the issue.

The command rg --debug "Hello" failed as ripgrep don't have a file or input stream to search. Kindly trying using

      - name: Run ripgrep without specifying a file (will fail, so debug)
        run: rg --debug "Hello" hello.txt

Thanks.

@tymscar
Copy link
Author

tymscar commented Nov 13, 2024

Yes, that fixes the ripgrep example, but not the overall issue. Why is there an open fifo?
In some binaries, you can't force it not to take the fifo into account.

@vidyasagarnimmagaddi
Copy link
Contributor

Hi @tymscar , Will into in it and provide our inputs, thanks

@tymscar
Copy link
Author

tymscar commented Nov 13, 2024

Thank you!

@vidyasagarnimmagaddi
Copy link
Contributor

Hi @tymscar

name:  ripgrep in gha

on: push
jobs:
  testing-ripgrep:
    runs-on: ubuntu-latest
    steps:
      - name: Redirect stdin to /dev/null globally to prevent FIFO issues
        run: |
          exec 0</dev/null  # Redirect stdin globally
      - name: Install latest ripgrep in gha
        run: |
          wget https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-x86_64-unknown-linux-musl.tar.gz
          tar -xvf ripgrep-14.1.1-x86_64-unknown-linux-musl.tar.gz
          sudo mv ripgrep-14.1.1-x86_64-unknown-linux-musl/rg /usr/local/bin
          rg --version
      - name: Run ripgrep on stdinput
        run: echo "Hello, world!" | rg "Hello"

      - name: Create a file with a message
        run: echo "Hello, world!" > hello.txt

      - name: Run ripgrep on a file
        run: rg "Hello" hello.txt  

      - name: Run ripgrep with no file, explicitly searching current directory
        run: rg --debug "Hello" .  # Explicitly search the current directory to prevent stdin read

      - name: Run ripgrep without specifying a file (debug)
        run: rg --debug "Hello" .  # Adding a directory search prevents stdin read   

I used the above logic and it worked for me
ripgrep has built-in heuristics taking input source depending on how it's called . By explicitly specifying the search path "." , by heuristics and ensuring the tool behaves as expected, directly searching files in the current directory.

@tymscar
Copy link
Author

tymscar commented Nov 13, 2024

Indeed, @vidyasagarnimmagaddi, I am aware of this. You had previously mentioned this in your previous comment, but addressing only the symptom does not resolve the underlying issue.

The reason the heuristics fail is because there is a persistently open FIFO on GitHub Actions that should not exist, as far as my understanding goes. The reason for its presence remains unclear, and this is the critical aspect that requires further investigation.

@vidyasagarnimmagaddi
Copy link
Contributor

Indeed, @vidyasagarnimmagaddi, I am aware of this. You had previously mentioned this in your previous comment, but addressing only the symptom does not resolve the underlying issue.

The reason the heuristics fail is because there is a persistently open FIFO on GitHub Actions that should not exist, as far as my understanding goes. The reason for its presence remains unclear, and this is the critical aspect that requires further investigation.

Hi @tymscar , Will redo my analysis and revert you back .Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants