Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use of qsize leads to NotImplementedError on macOS #14

Closed
harej opened this issue Oct 5, 2017 · 2 comments
Closed

Use of qsize leads to NotImplementedError on macOS #14

harej opened this issue Oct 5, 2017 · 2 comments

Comments

@harej
Copy link

harej commented Oct 5, 2017

Norepinephrine:Citationbase jh$ mwrefs extract enwiki-20170901-pages-meta-history10.xml-p2969251p3015717.bz2
page_id	page_title	rev_id	rev_timestamp	reference
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/bin/mwrefs", line 11, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mwcli/router.py", line 56, in main
    module.main(sys.argv[2:])
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mwrefs/utilities/extract.py", line 35, in main
    run(dump_files)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mwrefs/utilities/extract.py", line 53, in run
    for vals in xml_dump.map(dump_files, process_dump):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mw/xml_dump/map.py", line 59, in map
    threads = max(1, min(int(threads), pathsq.qsize()))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/queues.py", line 117, in qsize
    return self._maxsize - self._sem._semlock._get_value()
NotImplementedError
Norepinephrine:Citationbase jh$

According to The Manual, qsize() in the multiprocessing module:

may raise NotImplementedError on Unix platforms like Mac OS X where sem_getvalue() is not implemented.

Source: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue.qsize

@halfak
Copy link
Member

halfak commented Oct 5, 2017

This is an issue with mwxml

@halfak
Copy link
Member

halfak commented Oct 5, 2017

Ha! Actually we can fix this by switching to the new mwxml library. I filed #15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants