@Bmueller Thank you for the proposal! It looks great to me. +1 to split into two groups. Having a significant amount of time like you proposed for the last step with everyone seems very useful to me because the two groups conclusions might overlap and/or conflict significantly.

I agree with Amir that this is a very important topic.
For example, Wikisource workflows rely a lot on templates that are slightly different in each wikis, making very hard hard to roll out change in all wikis. It leads to an increasing technological gap between big and small wikis.
Global templates/modules might decrease the technical gap that exists between wikis.

@Tpt I can't remember if we ever added the magicword for overwriting and/or removing it on a specific page.

@kaldari Done. The Community Tech team is already a maintainer of the tool.

It maybe happens because the update of the page_props DB table is delayed until after the refresh of the Index: page.

@Reedy Thank you!

@Aklapper Indeed. Thanks!

@Lydia_Pintscher Improving JSON-LD structure like I proposed requires a strong refactoring of Purtle. I would not commit to do it anytime soon and I believe no one else would. So, I don't think it's a good idea to block JSON-LD deployment for that.

Extension:Replace_Text only works on pages content with content model "wikitext". ProofreadPage Page: pages are using an other content model ("proofread-page"). Extension:Replace_Text might be extended to support ProofreadPage content models to.

QueryPage was supposed to use the returned values of QueryPage::getOrderFields() to sort the results instead of value. it seems that recent code dropped partially this feature. Adding back the full support of QueryPage::getOrderFields() or, at least, strongly deprecate it seems the proper way to go.

It is maybe related to T184867. Has the first page of the PDF a smaller size orresolution than the other ones?

Option B seems the most usable to me and the most consistent. Prefixes like "c-wd" has the disadvantage of still having "wikidata" in it, and it's imho quite confusing.

Related guidelines: http://kb.daisy.org/publishing/docs/html/index.html

Thank you!

Wsexport is now using /data/project/wsexport/tool/temp as a temp directory. There are some files still leaked there, maybe because of an error while generating them.

An error that seems to be related:

@Samwalton9 This OAI-PMH feed was targetting the Index: pages of Wikisource, so instead of having "Hamlet" you might have "The Complete Works of William Shakespeare, volume 1, 1919". It's probably not what you are interested in. @Samwilson tool sounds more suitable to the Wikipedia Library use case.

@bd808 Indeed, wsexport has stopped submitting calibre conversion to the grid and does it inside of the webservice worker now. So this task is obsolete

HotCats does not support proofread-index content model. It should have special support for it to work properly. There is currently https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ProofreadPage/+/493793 under review that adds a "category" field to index pages and that could be worked upon to implement proper support in HotCats.

Sorry for the misunderstanding. Search in the page: namespace is not enabled by default on Wikisource and I believe only a few people use it, mostly contributors to do curation tasks. So, it's definitely possible that people missed a slow decrease of the index coverage. Most of Page: pages are very rarely edited because there are very few reasons to change a proofread page, opposite to a Wikipedia article.

Does that seem reasonable?

Thank you for your help!

It has indeed been fixed by https://gerrit.wikimedia.org/r/499801

It's maybe related to https://phabricator.wikimedia.org/T219514

Possible plan for the migration:

The package is now deployed on ToolForge.

The package is now deployed on ToolForge.