Page MenuHomePhabricator

API code 'readonly' not handled on Wikidata
Closed, ResolvedPublic

Description

This API response looks like it should cause a 'back off and retry':

{'servedby': 'mw1017', 'error': {'info': 'The wiki is currently in read-only mode', 'help': 'See https://test.wikidata.org/w/api.php for API usage', 'readonlyreason': 'The database has been automatically locked while the slave database servers catch up to the master', 'code': 'readonly'}, 'warnings': {'watch': {'*': 'The title parameter has been deprecated.'}}}

https://travis-ci.org/wikimedia/pywikibot-core/jobs/63795749#L918

Note that this is not the internal_api_error_ReadOnlyError error, which already does start a 'back off and retry' reaction.

Event Timeline

jayvdb raised the priority of this task from to Needs Triage.
jayvdb updated the task description. (Show Details)
jayvdb added a project: Pywikibot.
jayvdb subscribed.
Restricted Application added subscribers: Aklapper, Unknown Object (MLST). · View Herald TranscriptMay 24 2015, 5:00 AM

I ran into this twice tonight while running on Wikidata:

2015-07-01 22:32:31             api.py, 1649 in             submit: VERBOSE  Pausing due to database lag: Waiting for 10.64.16.15: 6 seconds lagged
2015-07-01 22:32:31        throttle.py,  249 in               wait: INFO     Sleeping for 5.0 seconds, 2015-07-01 22:32:31
2015-07-01 22:32:36        throttle.py,  140 in  checkMultiplicity: VERBOSE  Found 1 wikidata:wikidata processes running, including this one.
2015-07-01 22:32:36             api.py, 1649 in             submit: VERBOSE  Pausing due to database lag: Waiting for 10.64.16.144: 9 seconds lagged
2015-07-01 22:32:36        throttle.py,  249 in               wait: INFO     Sleeping for 5.0 seconds, 2015-07-01 22:32:36
2015-07-01 22:32:42             api.py, 1730 in             submit: VERBOSE  API Error: query=
{'action': [u'wbsetreference'],
 u'assert': [u'user'],
 u'baserevid': [225657903],
 u'bot': [1],
 u'format': [u'json'],
 u'maxlag': ['5'],
 u'snaks': [u'{"P813": [{"datavalue": {"type": "time", "value": {"after": 0, "precision": 11, "time": "+00000002015-07-01T00:00:00Z", "timezone": 0, "calendarmodel": "http://www.wikidata.org/entity/Q1985727", "before": 0}}, "property": "P813", "snaktype": "value"}], "P854": [{"datavalue": {"type": "string", "value": "http://www.mfa.org/collections/object/reading-on-the-rocks-grand-manan-33709"}, "property": "P854", "snaktype": "value"}]}'],
 'statement': [u'Q20635272$AF33EB11-5184-4212-9CE4-88BC1B5823CC'],
 u'token': [u'5345asdfasdf44878+\\']}
2015-07-01 22:32:42             api.py, 1732 in             submit: VERBOSE             response=
{u'servedby': u'mw1122', u'error': {u'info': u'The wiki is currently in read-only mode', u'readonlyreason': u'The database has been automatically locked while the slave database servers catch up to the master', u'code': u'readonly', u'help': u'See https://www.wikidata.org/w/api.php for API usage'}}

  File "C:\pywikibot\coredev\pywikibot\data\api.py", line 1734, in submit
    raise APIError(**result['error'])
pywikibot.data.api.APIError: readonly: The wiki is currently in read-only mode
<class 'pywikibot.data.api.APIError'>
CRITICAL: Closing network session.

Would indeed be much nicer if the bot would do the graceful back off and recover logic.

jayvdb triaged this task as High priority.Jul 1 2015, 11:35 PM
jayvdb added a project: Pywikibot-network.

Only seen on Wikidata so far.

It looks like the api provides no information about how long the bot should wait. is there some other way to determine how long to wait?

[21:51:25] <multichill> In https://phabricator.wikimedia.org/T100123 the last question. You might be able to answer that
[21:51:31] anomie looks
[21:51:35] <multichill> How long should we back off when we hit read-only?
[21:55:18] <anomie> multichill: For the particular situation there, "The database has been automatically locked while the slave database servers catch up to the master", you're basically waiting for maxlag to go back down. When using the actual maxlag parameter, the recommendation is 5 seconds.
[21:56:14] <multichill> So just the normal 5, 10, 20, 40 etc routine

Interestingly this is now not Wikidata isn't it.

On Wikidata to database slips into read-only mode several times a day (see T123867) and every time bots crash. Fairly annoying.

While looking at T144023 I wondered if this should be fairly easy to fix. The api gives a clean error:

WARNING: API error readonly: The wiki is currently in read-only mode pywikibot.data.api.APIError: readonly: The wiki is currently in read-only mode [readonlyreason:Waiting for 7 lagged database(s); help:See https://www.wikidata.org/w/api.php for API usage]
           if code == "readonly":
                   pywikibot.log(
u"Pausing due to database in read-only: " + info)
                   self.site.throttle.lag(self.throttle)
continue

Would this work? Any idea how to test this?

Will probably be solved with T154011.

I doubt it. Normal editing and Wikidata editing use different code and api modules. Normal editing will raise pywikibot.OtherPageSaveError, Wikidata editing will raise pywikibot.data.api.APIError

Multichill renamed this task from API code 'readonly' not handled to API code 'readonly' not handled on Wikidata.Apr 22 2017, 9:46 AM
Multichill added a project: Pywikibot-Wikidata.
Multichill claimed this task.

Change in T154011 got merged. Should fix this one too.