Working on countrycode "by" in language "be-x-old" WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['250'] WARNING: Waiting 5 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['125'] WARNING: Waiting 10 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['62'] WARNING: Waiting 20 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['31'] WARNING: Waiting 40 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['15'] WARNING: Waiting 80 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['7'] WARNING: Waiting 120 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['3'] WARNING: Waiting 120 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['1'] WARNING: Waiting 120 seconds before retrying. WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['0'] WARNING: Http response status 301 WARNING: Non-JSON response received from server wikipedia:be-x-old; the server may be down. Set geilimit = ['0'] Unknown error occurred when processing country by in lang be-x-old Maximum retries attempted without success. Working on countrycode "aq" in language "en" Unknown error occurred when processing country aq in lang en (2006, 'MySQL server has gone away') Working on countrycode "it-88" in language "ca" Unknown error occurred when processing country it-88 in lang ca (2006, 'MySQL server has gone away') Working on countrycode "de-nrw-k" in language "de" Unknown error occurred when processing country de-nrw-k in lang de (2006, 'MySQL server has gone away') …
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Replace be-x-old by be-tarask throughout | labs/tools/heritage | master | +86 -86 | |
| Commenting out be-x-old configuraiton | labs/tools/heritage | master | +81 -81 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | JeanFred | T131714 Hold a Wiki Loves Monuments coding sprint at Wikimedia Hackathon 2016 | |||
| Resolved | Lokal_Profil | T112415 Rename wikipedia:be-x-old to be-tarask in the Monuments Database | |||
| Resolved | Lokal_Profil | T134728 matchWikiprojectLink assumes language codes contain word characters when they may also contain dash |
Event Timeline
Comment Actions
This is quite bad because no harvesting but a few countries before be-x-old in the config gets done. I think I am just going to comment out this config and figure it out later.
Comment Actions
Oh, okay, this is due to T11823. I can update monuments_config.py. Do we also need to rename the database table ?
Comment Actions
Change 241255 had a related patch set uploaded (by Jean-Frédéric):
Commenting out be-x-old configuraiton
Comment Actions
Change 281220 had a related patch set uploaded (by Lokal Profil):
Replace be-x-old by be-tarask throughout
Comment Actions
I've also changed user-config to be-tarask. Once the ongoing monuments run is done we could try deploying
Comment Actions
But I guess the by_be-tarask table msut be crated manually before it is ingested into monuments_all at the next run?
Comment Actions
I don't think this one worked. Or did we never deploy it (as in recreating the table)?
Comment Actions
Ok. So the table exists and the config looks right.
I'll try running a country specific database update manually for this one once https://gerrit.wikimedia.org/r/#/c/287202/ has been deployed and this is possible again. Hopefully any error message thrown will tell me more.
Comment Actions
Ok. SO the problem is that we are running a pywikibot install from before the rename. Updating to a later one.
Keep an eye on this link to see if it worked, post harvest.
edit: List harvest worked, should get merged into monuments_all tomorrow and then be accessible on the above link.
Comment Actions
The table now gets populated but matchWikiprojectLink() fails since - is not a word character and so not caught by the \w+ regexp. (T134728)
There might also be an issue with source field still being to long but that is hard to verify until the above is fixed.
Comment Actions
Looks like the source field still isn't long enough. At this point we can either increase the size further or give up on storing the urlencoded text (I could suggest a patch for pywikibot) and instead do urlencoding on the fly where necessary. Maybe a different task though.