Diverse things thread

[madeddy]

Hi!

I noticed we have some small leftovers from the move to py3 and miscellaneous. I guess, even if they don't eat any bread, you want probably to address them at some point. I just write it down so it's not forgotten.

• The un.rpyc readme are still advising about py2 usage
• Same for make
• unrpyc.py#L195 😁
• unrpyc-compile.py#L62 has a strange redefinition of traceback and the import for it is unused (Or i got it really wrong...)
• The are also some unused imports in most modules and unused vars even (can be fixed at some later point i would say)

[CensoredUsername]

Yeah, I should probably fix those un.rpyc ones at one point (if anyone else ever wants to compile that. Tbh un.rpyc mostly exists to stop people from sending me issues that can be summarized as "how do I command prompt", I'm pretty sure you're familiar with those ;)

unrpyc-compile.py#L62 has a strange redefinition of traceback and the import for it is unused (Or i got it really wrong...)

That one is a doozy. It might be a bit of a remnant from ren'py versions of days past. It might not be necessary any more I'd even half forgotten about it. But for a summary of what lead to that.

When bootstrapping un.rpyc inside of ren'py, we have to perform a bit of a dance around the python import system. For that, it's useful to know what the import system actually does. It works in essentially the following steps:

• first, construct the full name of the module that we're importing. This is based on the name that was given to be imported, and the current module/package we're in (which it can get from the package attribute).
• If that name is present in sys.modules: return the object you just found
• If it isn't, it will iterate through the loaders in sys.meta_path. The first of which normally looks for stuff in sys.path and if it finds something matching that name, it'll compile and execute it.

Now ren'py inserts its own loader that handles importing of ren'py things. Which is very neat except we don't want to actually import ren'py things. So we remove that loader. Unfortunately that loader is/was also responsible for importing some python modules, IIRC like traceback and codecs. So after we remove that, we cannot import those properly any more.

But by importing those before we remove the loader, we can ensure that their contents are in sys.modules, and from there we can load them. It is likely a remnant of me discovering that workaround. But usually when I test un.rpyc it doesn't produce errors, so I should test if it's still ok, otherwise people might get some annoying issue reports.

[madeddy]

I'm pretty sure you're familiar with those ;)

That was a good one.
Good to know, this annoying totally unneeded injector(Did i mention totally means absolutely...) exists purely, because some people search always for something to click on. And any mention of a CLI yields a response along the line "CLI? Can i eat it, is it tasty?". BAH!

Your "roundtrip" explanation about traceback was really interesting, so thanks for it. I'm just not sure i understood it fully, so i will reread this a few times. LOL.

That was fun.

[madeddy]

Just so... For the pathlib pull i had some alternates instead of the nested triple for loop. e.g. i had this on in store:

worklist = [elem for entry in args.file for globitem in glob_or_complain(entry)
            for elem in traverse(globitem)]

If you ever feel like making new pythonists cry, i think that's probably a way. I encountered something like this "thing" ~10 years ago and pulled my hair.

Alternate 2 is the reverse:

globlist = []
for entry in args.file:
    globlist.extend([item for item in glob_or_complain(entry)])

worklist = []
for entry in globlist:
    worklist.extend([ele for ele in traverse(entry)])

Interestingly as full comprehensions they would need a extra step, as they produce then list of lists(something i'm glad i got rid of) and would need flattening with .chain() or similar.

[madeddy]

In the pathlib switch i mucked up some code in glob_and_complain().
Ok, it doesn't happen in like 99.9% of use cases but i'm still annoyed by myself, because i tested this with over 100 path variations. Still, i did not try put a "in this way" invalid path in there.

olli@tty2*blue $ python unrpyc.py /home/olli/.xlib/RPG/_test/OurBrightDays-0.1.3-pc/gam
File not found: /home/olli/.xlib/RPG/_test/OurBrightDays-0.1.3-pc/gam
Traceback (most recent call last):
File "/home/olli/Code/Git/unrpyc/unrpyc.py", line 334, in <module>
main()
File "/home/olli/Code/Git/unrpyc/unrpyc.py", line 284, in main
for globitem in glob_or_complain(entry):
TypeError: 'NoneType' object is not iterable

Well. return comes from a ternary now and retval is only returned if True. The empty list from glob is (obviously) replaced by None and this bangs.

On top of this i noticed, strict in Path().resolve(strict=True) is useless against a empty list and otherwise the output of glob is anyway clean.

If you don't do it before, i will fix this by return to the old test code in one of the next pulls i have in store. I mean this isn't badly needed in think.

[CensoredUsername]

It happens. That's why there's a dev branch ;) Just pull request a fix when you have time.

[madeddy]

If its ok with you i have also another pull coming in time. With some cleanup of unused imports etc. and a lot of PEP8 which should improve readability a lot in some cases and also code quality.

Could you when its opportune verify whats with the unused var in util.py#L446? Can this be removed or should there be a needed test which was never implemented?

[CensoredUsername]

Could you when its opportune verify whats with the unused var in util.py#L446? Can this be removed or should there be a needed test which was never implemented?

I think it's just a leftover.

[madeddy]

I am trying to understand the "renpy" fake package and it would be nice if you could shine a bit light on this.
So. We declare it in renpycompat.py(in dev branch) with use of magic.py. Sometimes we use from .renpycompat import renpy and in other modules just import renpy and this works also... Does magic make this global available or what else is happening?

[CensoredUsername]

Does magic make this global available or what else is happening?

The purpose of magic.fake_package is to make the python runtime environment pretend like there is an actual package at that name. So after that call, you can just import it. How it works is simply by inserting an entry into sys.modules, and as I've explained before, when you import something, the first thing the runtime does is check if there's already a matching object in sys.modules and if there is, just return that.

So you can do silly things like:

>>> sys.modules["foo"] = "bar"
>>> import foo
>>> foo
"bar"

Sometimes we use from .renpycompat import renpy

That one was added by you ;). .renpycompat imports renpy after faking the package, and that line just re-imports whatever is at renpycompat.renpy. Module imports are intrinsically public in python after all. I didn't comment about it because it just makes it a bit clearer where the hell that renpy module is coming from.

It's similar to doing something like:

from pickle import sys

Which works because internally pickle imports sys, and not because sys is a submodule of pickle.

[madeddy]

Thanks for explaining!

That one was added by you ;)

I know! But i misunderstood this nicely. 😊
I thought this "fake" package "renpy" is different from all the other modules, somehow lives where its created by magic and is only from renpycompat(at least in 'dev' now) available. So, we could in every file do from renpycompat import renpy or just import renpy and there is no difference?

[CensoredUsername]

So, we could in every file do from renpycompat import renpy or just import renpy and there is no difference?

Yup.

[madeddy]

Alright. Just for info.
I managed to get the issues with the TL feature done.

A bit more complicated and more code changes as expected. At least i understand now why J.McBarn did go with two runs. Basically its still so, but i loop it after the first in the second without exit. So its as said in the "ish" just one argparse option.

I waited with PRs about this for the other code merges, so i can rebase locally. This will then close #202, partly #211 and possibly also #183 with a second PR, as i overhauled the readme a bit for this.

[madeddy]

Makes it sense to use the TL feature with option init_offset or sl_custom_names? And more important, would it work? (I cannot test this. No files.)
Because if not i will simply cut them from use with translate.

[CensoredUsername]

They only make sense during the second step of the TL feature right now.

[madeddy]

Ok, thanks.

[madeddy]

On a side note about the spawn/fork start methods, don't know if you read it already: "Fork" is as standard on Linux on his way out with Python 3.14 and to be replaced by spawn.

Maybe we should set it already to spawn on LX so we know already about future issues and have it on all OS on the same start method.

[CensoredUsername]

"Fork" is as standard on Linux on his way out with Python 3.14 and to be replaced by spawn.

Makes sense. Stuff's been moving away from fork as while it seems simple, it's gotten more and more edge cases over the years.

I see no reason to move to spawn already though, it works fine and the code that deals with it is only a tiny part of unrpyc.

Anyway, with the tl work completed I'll go squash and merge that branch and cut a new release. Stuff should be nice and stable again. Only work remaining might be to port most of this stuff to legacy as well.

[madeddy]

I see no reason to move to spawn already though, it works fine

I only brought it up because I've read more than once that fork in MP is supposed to be broken. No idea how to understand this claim.

merge that branch and cut a new release

Sounds good.
I have just two little additions of cosmetic nature. If you're interested in them at all. I played just with "badges" a bit for better/fast info and with the argparse help output/order.

See if you're interested to add it. #227

[CensoredUsername]

Seems good. I'm a bit low on time right now so it might take a few days to get it in.

[madeddy]

Ah, no stress.

Summa summarum: I think, with everything since January unrpyc is pretty well set for now. After this, i see just some small things for the foreseeable future and a few remaining leftovers in the tracking issues and for legacy. Don't know if i can help with some of this.

Only work remaining might be to port most of this stuff to legacy as well.

What of all this do you want to backport?

ps: Don't forget to change the __version__ string before release.

Greets

[CensoredUsername]

Well that was a bit of a wait, but doing it right now.

[madeddy]

Very nice.

I lost a bit track of what we did last, but i think we where basically done for the foreseeable future. Most of the issues left are minor or seldom used code things and older stuff, or not?

Greets