[stdlib][Draft] Introduce heapq module

[bgreni]

Going down the list of "stuff we need because Python has it", we have the heapq module.

There exists a pure python implementation in the Python standard library, so for now we can follow it basically exactly, then write some benchmarks and tweak if from there if possible?

[gabrieldemarmiesse]

Maybe we can wait a bit before introducing new data structures? We are constantly in the middle of rewrites (new hash, new type of copy, new to_format method instead of __str__). Let's wait until the dust is settled down and we feel good about our existing structs before adding new collections that will require those rewrites and maybe new rewrites that we have not anticipated yet.

I'm all for good python compatibility, but with heapq we would be biting more than we can chew at the moment.

[bgreni]

Maybe we can wait a bit before introducing new data structures?

That's fair, but in this case the heapq module doesn't actually define any new collections, but rather a bunch of free functions to treat a List as a heap and only uses the most basic methods on List so I feel it should be largely unaffected by any of these rewrites?

[gabrieldemarmiesse]

I'll let the maintainers chim in but in my opinion, we do not have enough bandwidth at the moment. Furthermore some rewrites may affect free functions (like references being constantly changing).

[guidorice]

"stuff we need because Python has it"

heapq would be a nice enhancement because it's useful for AI related stuff, like Hierarchical Navigable Small World (vector similarity search). Also noting that python has a C implementation, so benchmarking against the mojo impl would be interesting- and important:
https://github.com/python/cpython/blob/033510e11dff742d9626b9fd895925ac77f566f1/Modules/_heapqmodule.c
https://github.com/python/cpython/blob/033510e11dff742d9626b9fd895925ac77f566f1/Lib/heapq.py#L581

[JoeLoser]

Any interest in rebasing this branch since the stdlib has come quite a ways since this was first introduced? We're happy to review once that is done.

[bgreni]

@JoeLoser I'm curious given the new attitude around loosening our commitment to being a "superset" if we might rather refactor this into it's own distinct data structure instead of a collection of functions operating on lists like in Python?

[JoeLoser]

@JoeLoser I'm curious given the new attitude around loosening our commitment to being a "superset" if we might rather refactor this into it's own distinct data structure instead of a collection of functions operating on lists like in Python?

I'm glad that you asked! Personally, yes, I'd go that direction. Do you have any interest in writing up a small API design sketch proposal and we can discuss and go that route before going too far? Feel free to consider inspiration from https://doc.rust-lang.org/std/collections/struct.BinaryHeap.html and friends.

[bgreni]

I'm glad that you asked! Personally, yes, I'd go that direction. Do you have any interest in writing up a small API design sketch proposal and we can discuss and go that route before going too far? Feel free to consider inspiration from https://doc.rust-lang.org/std/collections/struct.BinaryHeap.html and friends.

Great I'd love to!

[bgreni]

Closing this now that the proposal for a dedicated BinaryHeap struct is in progress #3776