Page MenuHomePhabricator
Create Task
Maniphest T313493

Add support for async session to python-mwapi
Closed, ResolvedPublic
Actions

Description

The MW API lacks support for asynchronous frameworks. We would like to add support for asynchronous functionality to mwapi using the asynchronous HTTP client/server aiohttp.

For revscoring models, with the implementation of the MWAPI HTTP cache to the api extractor (https://github.com/wikimedia/revscoring/pull/522), we can fetch data from the MediaWiki API asynchronously and populate the MWAPICache object, to prevent calling the blocking mwapi Session.

For non-revscoring models like outlink-topic model, we can simply change the blocking mwapi Session to the non-blocking mwapi AsyncSession to fetch data asynchronously.

PR: https://github.com/mediawiki-utilities/python-mwapi/pull/46

Related Objects
Search...

Event Timeline

achou created this task.Jul 21 2022, 12:16 PM
achou added a parent task: T311043: Use non-blocking HTTP calls to get outlinks for Outlinks topic model .
achou moved this task from Parked to In Progress on the Machine-Learning-Team (Active Tasks) board.Jul 21 2022, 12:20 PM
elukey added a comment.Jul 22 2022, 4:33 PM

Aiko's patch has been merged!

New version of mwapi released by Aaron: https://pypi.org/project/mwapi/0.6.0/ :)

achou added a comment.Jul 25 2022, 7:59 AM

The current version has an issue in Python 3.7 :(

asyncio.exceptions.TimeoutError will raise AttributeError: module 'asyncio' has no attribute 'exceptions' since Timeout is in class concurrent.futures._base.TimeoutError in Python 3.7, but in class asyncio.exceptions.TimeoutError in Python 3.8 onward.

The fix is to change it to asyncio.TimeoutError that is acceptable for either version. Here is the patch: https://github.com/AikoChou/python-mwapi/commit/90d2f5d40a9aaaf9822b8e59540b30c85965621a

elukey added a comment.Jul 25 2022, 8:53 AM

@achou let's create a pull request when you are ready, I'll ask Aaron to review and cut 6.1 :)

achou added a comment.Jul 26 2022, 6:07 AM

@elukey I created a pull request for the fix https://github.com/mediawiki-utilities/python-mwapi/pull/47, thank you!!

elukey added a comment.Jul 26 2022, 4:21 PM

https://pypi.org/project/mwapi/0.6.1/ :)

calbon closed this task as Resolved.Aug 2 2022, 2:48 PM
calbon moved this task from In Progress to Completed on the Machine-Learning-Team (Active Tasks) board.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits