pyscan seems to depend on pip

[jugmac00]

Describe the bug
tool does not work when pip is not on your path

To Reproduce

• use e.g. Ubuntu with no pip installed
• run pyscan

❯ pyscan 
pyscan v0.1.5 | by Aswin S (github.com/aswinnnn)
Failed to execute 'pip list' command. pyscan caches the dependencies from pip with versions to be faster and it could not run 'pip list'. You can turn this off via just using --cache-off [note: theres a chance pyscan might still fallback to using pip]

Note:
Not having a global pip is indeed a very good thing on e.g Ubuntu, as then you completely avoid installing packages into the global site-packages, which could break your Ubuntu installation.

And when you ask how I install packages... When creating a venv, the tool virtualenv also installs pip and setuptools into that environment.

[aswinnnn]

if you read the error, you can see that using the --cache-off flag will turn off the caching at the beginning so im not sure i understand your problem?

The fallback also should not pose a problem as it resorts to pypi.org when a version couldn't be gotten from pip. However I see the point in making pyscan aware of whether pip exists or not beforehand.

[jugmac00]

so im not sure i understand your problem?

pyscan does not do anything. When I put pip on the path, it at least lists the dependencies...

compare

with pip on path (ignore the error for now - reported in #13)

~/Projects/hypermodern via 🐍 v3.8.10 (xxx) 
❯ pyscan
pyscan v0.1.5 | by Aswin S (github.com/aswinnnn)
Found 3 dependencies
pytest : A version could not be detected through source or pip, so retrieving latest version from pypi.org instead.
Failed to make a request to pypi.org:
HTTP status client error (404 Not Found) for url (https://pypi.org/pypi/httpx[http2]%3E=0.23.1/json)
pypi.org error: HTTP status client error (404 Not Found) for url (https://pypi.org/pypi/httpx[http2]%3E=0.23.1/json)

without pip on path

❯ pyscan
pyscan v0.1.5 | by Aswin S (github.com/aswinnnn)
Failed to execute 'pip list' command. pyscan caches the dependencies from pip with versions to be faster and it could not run 'pip list'. You can turn this off via just using --cache-off [note: theres a chance pyscan might still fallback to using pip]

with `--cache-off

❯ pyscan --cache-off
pyscan v0.1.5 | by Aswin S (github.com/aswinnnn)
Found 3 dependencies
Failed to execute 'pip list' command. pyscan caches the dependencies from pip with versions to be faster and it could not run 'pip list'. You can turn this off via just using --cache-off [note: theres a chance pyscan might still fallback to using pip]

Maybe I fail to understand the output and/or the --cache-off function.

I currently do not have access to a single Python project where pyscan would work 100% and outputs something like on the screenshot from your readme.

Could you link to a working project? So I can make sure I understand which issues are from the Python projects, and which from my system setup.

[aswinnnn]

That is extremely weird and something that has never come up in my test cases. Have you tried it with a requirements.txt ? Specifying the versions are optional

I'm AFK at the moment but the reqs i tested against, one had no versions and other one had versions, both didnt fail.

The pypi.org error happens to due to the way the URL is string formatted, somehow the version gets injected into it instead of solely the package name, very weird.

[jugmac00]

Indeed, requirements.txt + putting pip on path works.

/tmp/test-r 
❯ echo requests > requirements.txt

❯ pyscan 
pyscan v0.1.5 | by Aswin S (github.com/aswinnnn)
Using requirements.txt/constraints.txt as source...
Found 1 dependencies
requests : A version could not be detected through source or pip, so retrieving latest version from pypi.org instead.
|-| requests [2.31.0] -> No vulnerabilities found.
Finished scanning all found dependencies.

[aswinnnn]

Now that we know this works, it just got even more weirder. Could you show me the content of what pyscan was trying to read from earlier where the 404 errors happened? Im sure something is going wrong somewhere along the parsing process.

[jugmac00]

The issue which resulted in the 404 is reported in #13 - there is also a link to the pyproject.toml.

[aswinnnn]

theres no parsing done at extract_imports_pyproject() at extractor.rs, i must have forgotten it. Looks like we found the root of the problem