May 29 2022
Indeed, this bug is related to c1f80efe05349cc39cb82dc9347b0a7699b11dfd (T304303): the live preview feature relies on the JS API allowing to fetch the current content of the editing area. To fix T183950 we dropped the hack inside of ProofreadPage that provides not only the page body but also the proofreading level and the page header/footer as page content. So, after this change, only the page body is fed to the live preview feature, hence the bug. A possible solution might be to add back the "complete page content" hack inside of ProofreadPage but some signficant work would be required to make features like find/replace work with this hack.
May 26 2022
I have written an implementation draft: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikisource/+/793867
May 9 2022
Mar 27 2022
Feb 14 2022
Jan 31 2022
Jan 13 2022
rel=prefetch has been added targetting the previous and next pages to speed up bulk proofreading according to T230689
Jan 7 2022
@Xover Sadly there is not hope for it to be pushed today except if someone is managing to follow the emergency deployment process: https://wikitech.wikimedia.org/wiki/Deployments/Emergencies
High priority: breaks a part of the basic Wikisource proofreading workflow.
Jan 6 2022
@Inductiveload Thank you!
I wrote a change that should fix the bug https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ProofreadPage/+/751946/
Jan 4 2022
The bug happend again on the new disk with the "exhibitorsherald2*" files. I removed them to put the tool back on. We should maybe prevent files >1GB to be downloaded and processed.
Dec 8 2021
Dec 1 2021
@Xover Thank you so much for having found this issue. It was under my nose all this time.
Nov 24 2021
@Mahir256 The change I wrote is in fact buggy. It's looking for a field named "toc" and not a field which data field is "toc". I'm writing a fix right now. Sorry for that.
Nov 23 2021
@Soda do you have a HiDPI screen? If it's the case, I believe modern browsers will download and use the bigger image that is provided by the srcset attribute before the initialization of Openseadragon. Then Openseadragon is initialized and use the regular size image. It's what happens with my 4K screen configured with 2x scaling.
Nov 21 2021
Sorry, I made a confusion between two different numerical systems: the ICU "beng" numerals are defined as the 10 digits "০১২৩৪৫৬৭৮৯" and, so, are not using base 16 as the currency numerals requested by this task. The base 10 numerical system is properly implemented and available now, the base 16 numerals are not yet availlable and not provided by ICU.
Nov 19 2021
@BenAtOlive If I understand correctly, you are asking if Oxigraph can be used to build a distributed SPARQL database with reader nodes and writer nodes? Its definitely possible to reuse Oxigraph components for that. For example I am currently moving the SPARQL parser out of Oxigraph and I plan to add general SPARQL query optimizations to it. However, building a query evaluator against a distributed storage where data are not in the same machine as the one executing (or at least scheduling) the query is a quite different task as building a query executor against a single node database. Access latencies are easily an order of magnitude higher with a remote store and it changes quite a lot of tradeoffs. So, yes for making the part of Oxigraph that are easy to reuse reusable but sadly I am not sure that naively spliting the query evaluator out of Oxigraph would provide a good query evaluator on top of a distributed storage.
Nov 18 2021
The change fixing this issue is now merged.
Nov 16 2021
Nov 15 2021
It's a great idea! I just believe P2860 (cites work) should be removed from the list. It's definitely not a classifying property.
Oct 3 2021
It is now fixed. There was some broken job data in the file system because of the full disk problem. I have opened T292365 to prevent it to happen again.
It's because the disk is currently full. I have removed a few huge books from the job queue and rebooted the server. However it does not seems to be enough to bring the website back.
Oct 2 2021
Sep 17 2021
@Ladsgroup Thank you! So what would you think about MCR for storing the "main" data and some change tags as "secondary" data for queryability and recent changes filtering? This way we would get the best of both worlds.
Sep 15 2021
Sep 14 2021
Sep 13 2021
@Inductiveload Great question! We might start now. Maybe with oldwikisource?
Sep 8 2021
Sep 2 2021
Aug 31 2021
@Thadguidry Great! Thank you!
@Thadguidry Hi! It's a great idea to improve the roadmap. Thank you for pushing me to do that! I have updated the GitHub issues and milestones and added a link to them in the README:
Each milestone should now have a short description and a set of linked issues. Most of the issues have been connected to a milestones.
I am a bit afraid of adding a ROADMAP file to the repository because it is yet another source of truth to maintain.
To also keep a single source of truth I made the choice to not expand the milestone description too much and keep the detailed discussions inside of issues. Here is [one about the storage backend and Sled]( https://github.com/oxigraph/oxigraph
/milestones?direction=desc&sort=completeness&state=open). However, the milestone+connected issues descriptions should allow to make sense of the roadmap. Feel free to ping me if you believe it's not clear enough.
Aug 26 2021
Thank you @So9q for opening this task!
Aug 25 2021
I'm working on a fix. Should be ready soon. Sorry for the troubles.
Aug 12 2021
Fixed and deployed.
Aug 7 2021
Sorry, I should not have closed this issue. The change that should fix this problem is not merged yet: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ProofreadPage/+/704564
So, it makes sense that the problem does not seem solved on Wikisource.
Aug 5 2021
How widespread is the impact of this bug remaining unfixed? If it's a nitch use and we can fix it with a backport in the short term then I wouldn't roll back. On the other hand, if this is causing some fallout elsewhere or triggering major production breakage then definitely roll back.
Aug 4 2021
Yes. To make it work properly "data": "toc" should be added to the relevant field in Mediawiki:Proofreadpage_index_data_config.
Jul 31 2021
@DannyS712 Thank you! I have understood why e.g. https://phabricator.wikimedia.org/diffusion/EPRP/browse/master/includes/Pagination/Pagination.php$25 is not flagged by these sniffs. It's because abstract methods are not covered by the lint: https://github.com/wikimedia/mediawiki-tools-codesniffer/blob/5f813f727ee47d5c90981b44e82f0c5a0263e933/MediaWiki/Sniffs/Commenting/FunctionCommentSniff.php#L198
Jul 19 2021
Thank you @Ankry, @Xover @Inductiveload! So I guess rel="preload" seems the way to go for the next page image. What about using rel="prefetch" for the previous one? This way, it gets a good chance to be loaded too but with less priority. What do you think about it?
Jul 17 2021
@Ankry Thank you! So, I guess that the main goal is to make sure that the image thumbnail is already generated when a Page: page is displayed.
If I remind correctly preload is to ask the browser to aggressively fetch a content for the current page rendering and prefetch is to ask the browser to load the content for future use in the navigation.
Jul 13 2021
Jun 30 2021
But aiui the granularity would be per-project and not per-index. How would a project start experimenting with this or avoid the need for a big bang migration? Use two toc fields in the index, one for the traditional way and one for a "Wikidata-based toc"? That would necessitate PRP differentiate between "present" and "non-empty" for the index data field tagged with "data": "toc", I think. That sounds a little more complicated, but is perhaps not too much so?
Jun 29 2021
Jun 8 2021
@Tpt Is there a technical reason for this?
May 22 2021
Done and deployed
Sorry for stepping in. A maybe relevant task when designing a new zooming system: T43614
Mar 27 2021
After thinking a little bit (and asking my wife who does not contribute to Wikisource), I have a feeling using an indicator is maybe not highlighting enough the proofreading level information: it's the first thing people are often looking for if they go to a page: page. But I am not an UX design expert so maybe an indicator is good enough.
Thank you for pushing this. I am a bit afraid it might create a backlash with some contributors feeling that the proofreading level is not highlighted enough. It might be nice to start a discussion on the biggest Wikisource communities. A first step is maybe to move the quality level from the <pages> tag and in the Index namespaces to an indicator.
This is presumably very nearly T239033, since if the extension can add a "Source" link for a mainspace page user of <pages/>, it can do it in any other namespace?
@Inductiveload Yes, I guess it should be also done in onSkinTemplateNavigation. The tricky part is that the "source" link is not known from the current page title but from the parser output, and onSkinTemplateNavigation do not have access to the parser output. It's why I have not done the migration to server side yet. There is maybe a good solution, I have not taken much time to look for an alternative yet.
Mar 26 2021
Mar 25 2021
@Tpt, is that fix complete or is there more that needs to be considered.
@Inductiveload https://gerrit.wikimedia.org/r/672760 generated "null pointer exceptions" during Page: pages rendering if the page was not already connected to an index (trace in T278379). Antoine and I reverted the change to fix this error. It was the simplest way we found to fix this error quickly on Wikisource, sorry. I have started to write a change that should fix this problem: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ProofreadPage/+/674691/1/includes/Page/PageContent.php
Feel free to integrate it into your change and push it again to gerrit.
Sorry again the quick revert.
Mar 19 2021
@Tpt: hmm, but how will you then get the prp-pagequality-N qualityN classes applied? Or do that manually?
It should be automatically as part by the LinkRenderer: the classes are added by a linker hook.
@Inductiveload I believe we could switch <pagelist> rendering to outputting HTML. $parser->getLinkRenderer()->makeLink allows to easily reuse MediaWiki linking system.
Mar 18 2021
@Inductiveload Yes, ProofreadPageInit.php is a good place for that.
Mar 16 2021
Not sure if that's a use case that is needed?
@Inductiveload I believe it should be better to use templatestyle because it does a lot of nice and useful CSS transformation and sanitization.
Mar 9 2021
Mar 8 2021
Mar 4 2021
Feb 19 2021
Jan 22 2021
Why is MOBI described as being for Calibre? I would've thought EPUB would be closer to Calibre's "default" format.
Jan 21 2021
Jan 20 2021
@Yash9265 wrote a change that removes navigation links when transcluding Special:IndexPages. I believe this problem is now solved.
I believe we want to display the export links only in namespaces where ProofreadPage allows the transclusion to happen and displays its navigation links.
For now it's only the main namespace but some Wikisource requested to be able to use it in other namespace (T53980).
Possible way to fix both problems at the same time is maybe create a per-wiki config inside of ProofreadPage and reuse it for the Wsexport link.
Jan 15 2021
We don't show the links on https://wikisource.org. I don't know if this matters. I don't know if users use that site.
Jan 13 2021
The change adding "thai" numerals support is now deployed on Wikisource.
Sorry, I confused this task with an other one.
My 2 cents: I like the idea of using Calibre ePub to ePub to do the file split in case of too big files. The current implementation in Wsexport is very bad. An other option would be to fix it inside of Wsexport by having a look of what Calibre is actually doing internally.
Jan 10 2021
So, the question is: Can the ProofreadPage extension be smarter about where the pagenum template is inserted to avoid putting it in "dead" table space?