Add plugin system and local wheels support #91

JeanChristopheMorinPerso · 2024-02-03T23:06:22Z

Fixes #64.

Add a new generic plugins system and builtin plugins. The glugin system is based on pluggy and has 5 hooks: prePipResolve, postPipResolve, groupPackages, cleanup and metadata.

The 2 builtin plugins are the PySide6 and shiboken6 plugins for correctly installing PySide6>=6.3. The plugins take care of merging all the PySide6 related packages (PySide6, PySide6-Addons and PySide6-Essentials) into one rez package.

$ rez-pip2 PySide6 --prefix /tmp/asd --python 3.11 -- --find-links /tmp/wheels --no-index

$ rez-env PySide6 --paths ~/rez_packages:/tmp/asd -- python -c 'import PySide6,shiboken6; print(PySide6); print(shiboken6)'
<module 'PySide6' from '/tmp/asd/PySide6/6.6.1/9645a50b415bcaad6bcde791aeb0859a43c56501/python/PySide6/__init__.py'>
<module 'shiboken6' from '/tmp/asd/shiboken6/6.6.1/9645a50b415bcaad6bcde791aeb0859a43c56501/python/shiboken6/__init__.py'>

🎉

This only works on Linux and macOS. On Windows, it works with Python 3.7 but not 3.8 and above. PySide6 on Windows makes use of os.add_dll_directory...

Additionally, add support for installing local wheels. Considering that PySide is a big package, I needed that functionality to speed up the development of the plugin system.

TODOs:

Add option to disable plugins/hooks
Test if builtin plugins can be overridden
Test on Windows
~~Write tests~~
~~Cleanup the code~~
~~Write docs (and automatically generate docs for builtin plugins).~~

src/rez_pip/rez.py

src/rez_pip/plugins/__init__.py

JeanChristopheMorinPerso · 2024-02-03T23:15:01Z

src/rez_pip/rez.py

@@ -113,8 +136,8 @@ def make_root(variant: rez.packages.Variant, path: str) -> None:
        pkg.pip = {
            "name": dist.name,
            "version": dist.version,
-            "is_pure_python": metadata["is_pure_python"],
-            "wheel_url": wheelURL,
+            "is_pure_python": isPure,


This annoys me. I don't think we really need this...

What is annoying about it?

It's kind of useless and getting the value of "is pure" is annoying. Do you think I should keep it?

src/rez_pip/plugins/PySide6.py

src/rez_pip/cli.py

codecov · 2024-02-04T01:49:50Z

Codecov Report

Attention: Patch coverage is 95.35714% with 13 lines in your changes missing coverage. Please review.

Project coverage is 90.95%. Comparing base (71ba391) to head (90e8b99).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/rez_pip/pip.py	88.57%	3 Missing and 1 partial ⚠️
src/rez_pip/plugins/__init__.py	94.11%	2 Missing and 1 partial ⚠️
src/rez_pip/cli.py	93.75%	1 Missing and 1 partial ⚠️
src/rez_pip/compat.py	75.00%	1 Missing and 1 partial ⚠️
src/rez_pip/rez.py	93.93%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #91      +/-   ##
==========================================
+ Coverage   81.14%   90.95%   +9.81%     
==========================================
  Files           8       12       +4     
  Lines         716      918     +202     
  Branches      133      174      +41     
==========================================
+ Hits          581      835     +254     
+ Misses        121       59      -62     
- Partials       14       24      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

BryceGattis

Left a couple questions/comments.

BryceGattis · 2024-04-30T12:46:18Z

docs/source/conf.py

    "rez": ("https://rez.readthedocs.io/en/stable/", None),
 }

 # Force usage of :external:
-intersphinx_disabled_reftypes = ["*"]
+# intersphinx_disabled_reftypes = ["*"]


Is this something we are wanting to keep commented out or remove?

I don't remember the exact reason I did this, but I remember that it was causing issues. I'll probably have to investigate this more to remember why I changed that.

docs/source/plugins.rst

src/rez_pip/pip.py

src/rez_pip/plugins/PySide6.py

BryceGattis · 2024-04-30T13:18:44Z

src/rez_pip/plugins/PySide6.py

+    packages: typing.List[str],
+) -> None:
+    pyside6Seen = False
+    variantsSeens = []


Should this just be variantsSeen?

BryceGattis · 2024-04-30T13:23:29Z

src/rez_pip/rez.py

@@ -113,8 +136,8 @@ def make_root(variant: rez.packages.Variant, path: str) -> None:
        pkg.pip = {
            "name": dist.name,
            "version": dist.version,
-            "is_pure_python": metadata["is_pure_python"],
-            "wheel_url": wheelURL,
+            "is_pure_python": isPure,


What is annoying about it?

Signed-off-by: Jean-Christophe Morin <[email protected]>

…e arguments are immutable The code is also a little bit cleaner IMO Signed-off-by: Jean-Christophe Morin <[email protected]>

Signed-off-by: Jean-Christophe Morin <[email protected]>

…force immutability at runtime. Signed-off-by: Jean-Christophe Morin <[email protected]>

Signed-off-by: Jean-Christophe Morin <[email protected]>

… positional arguments instead of kwargs Signed-off-by: Jean-Christophe Morin <[email protected]>

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso mentioned this pull request Feb 3, 2024

Fail to install PySide6 #64

Open

JeanChristopheMorinPerso commented Feb 3, 2024

View reviewed changes

JeanChristopheMorinPerso force-pushed the plugins branch from 5163f94 to c7edc67 Compare February 4, 2024 00:36

Repository owner deleted a comment from sonarqubecloud bot Mar 6, 2024

JeanChristopheMorinPerso force-pushed the plugins branch 3 times, most recently from 178c63c to f9b6577 Compare April 28, 2024 23:38

JeanChristopheMorinPerso changed the title ~~Initial commit adding a plugin system and a PySide6 plugin~~ Plugin system Apr 28, 2024

JeanChristopheMorinPerso force-pushed the plugins branch from f9b6577 to e57ce92 Compare April 28, 2024 23:58

BryceGattis reviewed Apr 30, 2024

View reviewed changes

JeanChristopheMorinPerso mentioned this pull request Nov 17, 2024

rez-pip with custom package.py AcademySoftwareFoundation/rez#1871

Open

JeanChristopheMorinPerso changed the title ~~Plugin system~~ Add plugin system Jan 1, 2025

JeanChristopheMorinPerso force-pushed the plugins branch from cd5f530 to 27b6a5c Compare January 1, 2025 17:34

JeanChristopheMorinPerso added 16 commits January 1, 2025 13:33

Initial commit adding a plugin system and a PySide6 plugin

209e30f

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix bug with requirements of merged packages

b503662

Signed-off-by: Jean-Christophe Morin <[email protected]>

Simplify registration of builtin plugins

7006ab9

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add pluggy to project deps

5a7ff04

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix tests

d854773

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix remaining issues (I think)

f41fb6d

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix download tests and add a test for skipped downloads

2915723

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix typing

d184853

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix typing

5bd4d54

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add basic test for plugins module

f93a87d

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix typing again

a8fa6d0

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add type hints and new dataclasses to express plugin authors that som…

91c7aca

…e arguments are immutable The code is also a little bit cleaner IMO Signed-off-by: Jean-Christophe Morin <[email protected]>

Improve plugins docs and autogenerate signatures

788fb4f

Signed-off-by: Jean-Christophe Morin <[email protected]>

Explicitly use tuples in hookspecs to improve typing even more and en…

13667aa

…force immutability at runtime. Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix some mypy errors

b798d8d

Signed-off-by: Jean-Christophe Morin <[email protected]>

More typing fixes and fix regression introduced by calling hooks with…

8e53cf3

… positional arguments instead of kwargs Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso added 6 commits January 1, 2025 13:33

More docs improvements for plugins

00f9ba9

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add more docstrings

5f6aca9

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix tests

c620857

Signed-off-by: Jean-Christophe Morin <[email protected]>

More typing fixes

a453bcf

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix all mypy errors

c37fe73

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add missing __eq__ method in PackageGroup for tests

d39e1d7

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from 9fd066c to d39e1d7 Compare January 1, 2025 18:34

More tests and docs

9609a36

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from 0d113a3 to 9609a36 Compare January 1, 2025 21:14

fix warning in docs

22c2f6d

JeanChristopheMorinPerso added this to the 0.4.0b1 milestone Jan 1, 2025

JeanChristopheMorinPerso mentioned this pull request Jan 1, 2025

Improve usability of rez_pip as python library. #89

Closed

3 tasks

JeanChristopheMorinPerso added 5 commits January 1, 2025 21:26

Fix handling of local wheels that would lead to duplicated packages

1e32ddc

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix PackageGroup.dists sharing a list. Default lists strike again!

8b0e22f

Signed-off-by: Jean-Christophe Morin <[email protected]>

autouse setupPluginManager fixtures

d883658

Signed-off-by: Jean-Christophe Morin <[email protected]>

Reset rez caches between tests

63418a4

Signed-off-by: Jean-Christophe Morin <[email protected]>

Add first integration test

2f0c2de

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from 511a798 to 2f0c2de Compare January 2, 2025 02:34

Add missing .coveragerc

ff56b6a

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso changed the title ~~Add plugin system~~ Add plugin system and local wheels support Jan 2, 2025

Try to fix integration tests on Windows

ea54157

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from c8b5aae to ea54157 Compare January 2, 2025 02:56

JeanChristopheMorinPerso added 2 commits January 1, 2025 22:02

Remove hardcoded path separator in rez_pip.install.isWheelPure

6fe3e41

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix integration tests again

32dff3b

Signed-off-by: Jean-Christophe Morin <[email protected]>

JeanChristopheMorinPerso force-pushed the plugins branch from 58e81fd to 32dff3b Compare January 2, 2025 03:20

JeanChristopheMorinPerso added 2 commits January 5, 2025 14:56

Some more docs

a3df115

Signed-off-by: Jean-Christophe Morin <[email protected]>

Fix order of hooks in docs

90e8b99

Signed-off-by: Jean-Christophe Morin <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add plugin system and local wheels support #91

Add plugin system and local wheels support #91

JeanChristopheMorinPerso commented Feb 3, 2024 •

edited

Loading

JeanChristopheMorinPerso Feb 3, 2024

BryceGattis Apr 30, 2024

JeanChristopheMorinPerso May 1, 2024

codecov bot commented Feb 4, 2024 •

edited

Loading

BryceGattis left a comment

BryceGattis Apr 30, 2024

JeanChristopheMorinPerso May 1, 2024

BryceGattis Apr 30, 2024

BryceGattis Apr 30, 2024

Add plugin system and local wheels support #91

Are you sure you want to change the base?

Add plugin system and local wheels support #91

Conversation

JeanChristopheMorinPerso commented Feb 3, 2024 • edited Loading

JeanChristopheMorinPerso Feb 3, 2024

Choose a reason for hiding this comment

BryceGattis Apr 30, 2024

Choose a reason for hiding this comment

JeanChristopheMorinPerso May 1, 2024

Choose a reason for hiding this comment

codecov bot commented Feb 4, 2024 • edited Loading

Codecov Report

BryceGattis left a comment

Choose a reason for hiding this comment

BryceGattis Apr 30, 2024

Choose a reason for hiding this comment

JeanChristopheMorinPerso May 1, 2024

Choose a reason for hiding this comment

BryceGattis Apr 30, 2024

Choose a reason for hiding this comment

BryceGattis Apr 30, 2024

Choose a reason for hiding this comment

JeanChristopheMorinPerso commented Feb 3, 2024 •

edited

Loading

codecov bot commented Feb 4, 2024 •

edited

Loading