Can see system site-packages in rez-env?

[AustinScola]

Hi, I'm new to rez and I'm trying to use rez-env similar to how I would a use a virtual environment.

One thing that is unexpected to me is that when I am in the rez-env I can still see system site-packages when I run python3 -m pip list?

For example, if I have mypy installed globally on my computer with python3 -m pip install mypy, and I run rez-env, then running python3 -m pip list | grep mypy shows hits?

Is this expected? Is there a way to isolate the the rez-env? I tried both --no-implicit and the --no-local arguments but neither of those seemed to do what I want?

[ColinKennedy]

One quick thing to check - I believe Rez sources your user settings (e.g. in bash, ~/.profile) within each rez-env call. So if, for example, you export PYTHONPATH=/lib64/python3.6/path/to/site-packages:$PYTHONPATH, it'll include those. I think pasing rez-env --norc disables that. Try running rez-env with --norc to see if the path goes away, then you'll know for sure it's coming from your user.

[AustinScola]

--norc doesn't seem to fix it either. But, I did notice that when I use that and run which python it doesn't seem to be using rez:

$ rez-env --norc --quiet
> bash-5.1$ echo "${PATH}"
/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.:/opt/rez/bin/rez
> bash-5.1$ which python3
/usr/local/bin/python3

[ColinKennedy]

It's mentioned in other replies but I'd assumed your full command would be rez-env python --norc, since you're asking for python. Of course, you'll probably need to rez-bind python if you haven't already so there's a package for it.

That said, @davidlatwe and @instinct-vfx is more knowledgeable about these things than I could be.

[instinct-vfx]

Well you do not have python in your request so it will use the system python. You need to create a python package and then use rez-env python to add it to the rez shell.

[AustinScola]

Okay, getting closer, but pip still shows a bunch of packages

$ rez-env --norc --quiet python
> bash-5.1$ which python
/Users/austin.s/.rez/packages/local/python/3.9.10/platform-osx/arch-x86_64/os-osx-12.3/bin/python
> bash-5.1$ python -m pip list # still lists a bunch of packages?

[davidlatwe]

Agreed with @instinct-vfx , and you may also try rez-python -m pip list, that rez-python is the Python in rez's venv (if it's production installed.)

[davidlatwe]

Hmmm, no. rez-python is part of rez command tools, and all rez's command tool exclude PYTHONPATH due to the "production install".

[AustinScola]

Don't really know what you mean by "production install"?

Here is what I was expecting to be able to do:

$ rez-env --norc --quiet python mypy-0.931
> bash-5.1$ rez-python
>>> import mypy

Is there a way to achieve something like this?

[instinct-vfx]

Yes. You need to bind or create a python package. And you can use rez-pip to create rez packages from pip packages. e.g. rez-pip mypy. Then you can do just that rez goes to great length to ensure that execution environments are separated (the parent environment of your machine seperated from rez's own execution environment and that environment from the environment rez creates for you. If it would not you would end up with roadblocks all day. For example i might be resolving python-2.7 for a DCC still stuck on py2 with a Rez deployment installed with python 3.6 on a system that has python 3.10 installed. If you resolve that python 2.7 environment it obviously needs to be isolated from your system python an rez's own python.

[davidlatwe]

If you did follow the README/Wiki instruction when installing rez, that's "production install". And now knowing what you're after, rez-python isn't what you want indeed.

What you need is a rez package which provide a python interpreter for you, so the command rez-bind python that mentioned by @ColinKennedy should get the job done.

[AustinScola]

So I did follow the wiki for installation. And I ran rez-bind --quickstart as part of that. So how can I create a venv where I can get a python REPL that has access to just the requirements I want. Or better yet, what I really want to do is run something like rez-python -m my_module inside the venv? (I also already ran rez-pip for mypy.)

[nerdvegas]

Just to clear things up: * the rez installation itself (where the cli tools like rez-env are) are installed into a venv. If you run rez-python, that drops you into an interpreter _within the rez installation_. This is all internal stuff and almost certainly not what you want to be doing - rez-python is for mucking about with rez internals. * rez-env resolves rez packages. If you 'rez-env python mypy' then both python and mypy need to be available as rez packages (for eg, they may have been created via rez-pip, or rez-bind, or a native rez package) * rez-env places you into a configured subshell, or "context". This is not a venv (but you could consider it analogous to one). * rez-bind is not ideal and we plan on deprecating it. In any case, its purpose is to create a rez package for you that interfaces with software already on your system. * I'm not sure why your rez-bind'ed python can see all your system-installed py packages, but whatever reason that is, it will be down to standard python mechanisms. I don't see the issue myself, but I have built python from source. There's probably something either with the rez-bind'ed python package, or your system, that's causing those packages to be visible. In any case, once you're in the rez-env, rez isn't doing anything special (you're just in some kind of configured shell), so you should be able to debug python from there and determine why you can see all those packages. That will then inform us as to whether the rez-bind module needs to be changed, or there's something in your system causing this. Hth A

…

On Sat, Mar 26, 2022 at 5:49 AM Austin Scola ***@***.***> wrote: So I did follow the wiki for installation. And I ran rez-bind --quickstart as part of that. So how can I create a venv where I can get a python REPL that has access to just the requirements I want. Or better yet, what I really want to do is run something like rez-python -m my_module inside the venv? — Reply to this email directly, view it on GitHub <#1260 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMOUSSZLJS2TIRCMZSCZEDVBYDC7ANCNFSM5RUTKC6A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[AustinScola]

Hey @nerdvegas thank you for the explanation. That really helps clear up.

When I run the rez-binded python from the rez-env, and print sys.path I see '/usr/local/lib/python3.9/site-packages' as the last item in the list. For now, I think I'll just modify sys.path to remove that and then I shouldn't be able to see site packages anymore.

[nerdvegas]

I'm happy if you have a workaround, let me know if you run into any issues doing that. I'm loath to put much effort into rez-bind given our desire to deprecate it. Cheers A

…

On Tue, Mar 29, 2022 at 12:58 AM Austin Scola ***@***.***> wrote: Hey @nerdvegas <https://github.com/nerdvegas> thank you for the explanation. That really helps clear up. When I run the rez-binded python from the rez-env, and print sys.path I see '/usr/local/lib/python3.9/site-packages' as the last item in the list. For now, I think I'll just modify sys.path to remove that and then I shouldn't be able to see site packages anymore. — Reply to this email directly, view it on GitHub <#1260 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMOUSRRONSOQ4PN6P4CBL3VCG3GVANCNFSM5RUTKC6A> . You are receiving this because you were mentioned.Message ID: ***@***.***>