@Tpt This is almost certainly not the same issue as T204020 (which is a dup of T219376, where some possible approaches to fix it are suggested). 20402 is basically the result of a naïve text layer extraction algorithm in MediaWiki-DjVu that fails badly when faced with unexpected data in a DjVu file, combined with the fact that the DjVuLibre tools will happily accept invalid input that it will choke on when asked to output it again. MediaWiki-DjVu can absolutely be modified such that it will fail gracefully in these cases. But, crucially, such DjVu files can be reliably identified (djvutxt --detail=page file.djvu | egrep "^failed"), and the example from this task does not exhibit that problem.

This is the same issue as T219376 which contains some possible approaches to fix this.

It looks like the mentioned javascript error in the console is gone now (as of 1.35.0-wmf.10 today; I can't recall if I checked on wmf.8 due to the recent hiccup in the deployment calendar), but the text layer is still not loading in the example page. Absent any console messages I have no idea how to debug this further. Are there server side logs that can be checked? A test system where it can be traced? Should I open a new task for that, and if so, with what tags or projects so that it has a chance to hit the right teams' radar?

@Aklapper See link in task description: https://en.wikisource.org/w/index.php?title=Page:Atalanta_-_Vol._2.djvu/18&action=edit&redlink=1

Incidentally, the correct place to solve this is in ProofreadPage because it has knowledge of what the next page in the sequence is. However, I suspect it would need facilities from core to do this intelligently: standard prefetching requires injecting a <link rel="prefetch" href="…" /> in the page, preferably before the browser sees it. ProofreadPage could of course "cheat" the way this gadget does (good version 1.0 / proof of concept / minimum viable product goal?), but that'd be kinda complicated, inelegant, overly specific, and prone to breaking over time.

Possibly related: T236469

This is a common operation on the Wikisources: what on a Wikipedia would be "a page" is on Wikisource "this collection of pages in multiple namespaces". Thus having a list of pages, possibly into the thousands (one per page of a book, for example), is a very common scenario. Based on recent experience on enWS it is even more common than mass deleting all contribs from a particular user (most of our recent spammers have constrained themselves to a single page).

Is it expected behaviour that mw.loader.using('mediawiki.RegExp', function() {…}) will fail silently?

Hmm. While DjVu and PDF (and, I think, TIFF) has explicit text layers; any kind of image can in principle contain text and could benefit from a structured way to store the OCR as actual text. We have oodles of images-of-text in JPEG, PNG, etc. formats in addition to the "book" formats (DjVu, PDF). Even book scans are not-infrequently uploaded as a couple hundred JPEGs gathered in a category (if we're lucky).

Ok, a couple of things here…

As the single community member (and admin) who expressed reservations there, I can also confirm that the discussion was a valid community consensus process; that the thread's closing was appropriate for enWS norms; that the closing comment accurately reflects the community's intent; and that this request is within the scope of that consensus. If needed we can get @Hesperian or @Mpaa (our bureaucrats) to confirm this here or on-wiki.

PS. Also: 13 years since this request was filed y'all! Probability of getting addressed tends to be inversely proportional to age of the request, so this is now the third least likely request to be addressed that I follow.

In T8808#1666317, @Billinghurst wrote:

Are we looking for a fourth option of "brand new name"?

Hmm. Perhaps this issue could also be solved by remex being intelligent about where this particular span belongs? At least inside a <tr>, or outside a <tr> but inside a <tbody>, the correct placement of it is inside the immediately preceding <td>. If remex knows the difference between ordinary page content and the stuff inside the ProofreadPage-specified interface file it should be feasible to do this at that layer with the existing infrastructure there.

In T228594#5478755, @Aklapper wrote:

[…] What happened or has changed that this task suddenly became "Unbreak Now!" priority, weeks after it was created?

Just an idle thought…

Now it's inserting extra whitespace before the close brackets of the citation, and it's still inserting duplicate links in |url= and |pages=. See https://en.wikipedia.org/w/index.php?title=Hamlet&diff=909523776&oldid=909485888&diffmode=source

Hmm. I'm not sure what's going on in Ineuw's browser to cause that ve.init undefined error, but I pretty sure the general OCR gadget issue that several people on enWS have reported is related to this console message (that I'm surprised Ineuw isn't seeing, but it may be being hidden by the ve.init issue):

[Error] Error: Syntax error, unrecognized expression: An error occurred during ocr processing: /tmp/52004_20706/page_0011.tif
	error (load.php:20:880)
	select (load.php:37:233)
	find (load.php:41:184)
	init (load.php:142:753)
	jQuery (load.php:2:505)
	hocr_callback (Script Element 1:594:825)
	fire (load.php:45:980)
	fireWith (load.php:47:174)
	done (load.php:126:628)
	(anonymous function) (load.php:130)

Which is an unhandled error emitted by this line (it looks like) of phetools.

Supporting move to "All wikis in a given language" is probably a pretty tall order since this was made as a one to one tool (and how would you merge multiple divergent edit histories?). But if we could get, say, FileExporter on Commons and FileImporter on English Wikisource, it would make it a lot easier when we have to repatriate books that are about to be deleted on Commons. That should be at least within the realm of a reasonable short term goal; and with that in place it may be possible to create user scripts or other external tools that can semi-automate the "All wikis in a given language" bit.

I cannot reproduce this either.

Some thoughts from a dabbler in user scripts on WMF wikis...

This looks to me like MediaWiki-DjVu is choking on the new file and thus never regenerating the thumbnails / page images. Not obvious that ProofreadPage is involved in this at all (except that this is where it becomes most visible). I downloaded the actual .DjVu from Commons and confirmed that the file indeed contains the fixed pages.

@eugrus The page numbers are generated by a community-maintained script, loaded from https://en.wikisource.org/wiki/MediaWiki:Common.js, that lives in https://en.wikisource.org/wiki/MediaWiki:PageNumbers.js. You will need various bits of supporting code from Common.js (in addition to loading the PageNumbers.js script itself) in order to get it to work.

From a technical perspective, the only clear advantage to the status quo would seem to be the OCR engine (ABBYY) that IA has access to but we (currently) do not.

All three text fields resize ok vertically here (they used to resize also horizontally, but I believe that was changed deliberately a while back). Their default size appear to be two lines.

Quick update from enWS: 1.33/wmf.12 is now live there and a quick check suggests it fixes the disappearing header and footer fields. IOW this probably confirms this was caused by T209939.

Quick update from enWS: 1.33/wmf.12 is now live there and a quick check suggests it fixes the disappearing horizontal rule. IOW this probably confirms this was caused by T209939.

Quick update from enWS: 1.33/wmf.12 is now live there and a quick check suggests it fixes the disappearing horizontal rule and the tiny/uneditable header and footer fields. No structured testing done, but it looks very promising.

Just to note for future reference, there have now been reports of this happening also in the browsers PaleMoon and WaterFox, who both seem to be Mozilla/Firefox forks, as well as Internet Explorer (unspecified version). My own tests suggest the "Horizontal layout" option has no effect either way on this (turned it off, no change), except in so far as it probably enables the "Show header and footer fields" option just above it (not sure what the default is for that). And if the header and footer fields aren't displayed at all you will of course not be able to reproduce any problems with them.

@ShakespeareFan00 The deployment that (aiui) rolls back this change isn't yet live on enWS. See most recent Tech News and the thread "Issue with the FI Template display" on the Scriptorium. Based on the schedule given in Tech News, the deployment should hit enWS at some point today (or possibly tomorrow, depending on how they schedule the rollout).

@stjn I did not say that <style> outside <head> is wrong (it isn't: <style> is allowed anywhere "flow content" is allowed).

Hmm. This looks like a pretty naive implementation. I had expected TemplateStyles to parse out any <templatestyles> elements and inserting them in the <head> element (or through ResourceLoader or something): expanding them inline is going to create loads of problems, of which this is a perfect example.

@Aklapper It seems very likely. Chromium is a fork of WebKit (the Safari engine) which exhibits the same problem (cf. the comments in the other task), and the timing (at least roughly) of this problem appearing coincides with the deployment of the changes there.

@EsmeShepherd The developers are aware of the problem you are experiencing, and the plan is (aiui) to fix it (by rolling back the changes that caused it) on 7 or 8 January. At that time, or shortly after, the header and footer fields should return to the size and behaviour they had previously.

@Tpt @TheDJ @Samwilson Looking quickly at the patch here, this is not something I imagine it's possible to do using only CSS changes. Flex box is a completely new layout model and is almost certain to require code (generated markup) changes and interact badly with other styling and scripts (including the markup produced by templates or hard coded in pages). As an example, imagine what changes would be necessary to let users switch back and forth between the two as skins. My gut feeling is that this would have to be a part of a "ProofreadPage UI 2.0" level effort in practice.

@Ankry The disappearing header and footer is more likely than not an issue only in Safari, or possibly on Retina-resolution devices (1pt = 4px, aka @2x, or higher), based on the reports I've seen (and it's probably some wonkyness related to font size: manually removing the font-size: 13px set on the text field using the web inspector makes the text field large enough to use).

@ShakespeareFan00: Either. Edit filters, aiui, can be managed more easily; but, based on my limited understanding, either should be able to solve this problem without actually creating an entirely new log.

Could not this (and would be better as) a javascript Gadget? It's just chaining together two existing operations so they can be triggered by a single button click. It would still need to be implemented, of course, but a Gadget sounds far less resource-heavy than changing the MW editing interface.

Could the log aspect perhaps be done using an edit filter? That wouldn't help retroactively (I don't think), but for all new edits they would be searchable.

Sigh. Yes, it was the content blocker. :facepalm: I should have double checked that first of all.

Observed on the last couple of release versions of Safari on macOS 10.13.4 and .5, and mobile Safari (iOS) on the last major release (11.3.1). Using the 1Blocker content blocker, but with *.wikipedia.org, *.wikimedia.org, and *.wikisource.org whitelisted (just to be sure: nothing on WMF sites should really be blocked to begin with).

The CS1-based templates—which are actually in use across a lot of different language wikipedias—quite correctly provides distinct parameters for the title of the book and the title of a chapter within that book; much as it provides distinct parameters for an encyclopedia and a particular entry in that encyclopedia. Based on the description here, Zotero also provides distinct parameters for these, it just does so in a needlessly complicated way (changing the semantics of the title attribute based on type, when you need two "title" type fields anyway, is just asking for needless complications in implementation). Or put another way: both CS1 and Zotero appear to be capable of expressing the concepts in question here.

In T188798#4048566, @Tgr wrote:

Adding a new configuration setting which is used in JS and form submit checks is more effort but that's the decent way to fix this IMO.

@TBolliger That sounds like a very good first approach; but I would urge you to keep in mind the possibility, longer term, to make this configurable per project. The discussion on enwp is crystallising a lot of factors applicable to that local project, but needs will differ across projects and making it configurable will allow local consensus to determine the appropriate limit for that specific project's needs. When or whether that is implemented, just having that as part of the requirements will shape the implementation of the short term solution: for example in terms of truncate/disclose functionality for revision history views to deal with revisions imported from a project with larger local limits.

In T6714#4020625, @Nemo_bis wrote:

It would not be a great idea to vary per project. At a minimum, wikis would then interfere with each other when some revision is exported and imported to another wiki with lower limit.

As I read various stuff related to this, including @brion's / Community Tech's RFC page, it seems the underlying database and backend change to 1000 bytes has happened previously, and that the March event was actually just removing a user interface level limitation to the status quo length (250 bytes or whatever it was). If that is so, then there is existing front end code to limit what editors are allowed to input in this field that can be deployed with a limit in line with community wishes regardless of what the backend technically allows. And in front end code it should be much easier to make this limit based on characters and not bytes, to ease the burden on the non-Latin communities.

I think the approach suggested by @Krinkle above may be overkill.

In T153431#4015388, @Samwalton9 wrote:

As for recent activity, that's a valid concern, but we're aiming for the requirements to be checked at the point of access, such that if you're away for a month and make 10 edits the first day you're back, you'd get access again straight away, not the following month. Does that sound more reasonable?

Hmm. I can easily be away from the project for a month from time to time. It'd be kinda annoying to be cut off from access to partner sources when I got back. Or picture the scenario where I edit every other month: I'm away for a month, get cut off, am back to editing for a month without access, and then get access for the next month when I'm not editing.

Note that there are nuances to this. For instance, when editing an existing citation using VE, any existing (in wikimarkup) empty parameters that are present should not be removed. There may be other such gotchas that I'm not thinking of right now. Perhaps the most surgical (least risk) approach would be to "remove" (not insert) empty parameters if and only if they were added to the editing interface by VE due to being "suggested" in the TemplateData.

1. Edits are now made by the user running the tool and not IABot, and users are responsible for any edits they make regardless of whether they were manual or automated.
2. Wikipedia doesn't mandate a single reference style, so short citations with full references in a separate section are by definition equally a part of the concept "references" as full references inside <ref> tags.

@Cyberpower678
Is that (must be inside <ref> tags) a requirement? Is it documented? It seems really non-intuitive that URLs inside the various cite templates (all the CS1/CS2 templates take a |url= parameter) are not checked. That would make IABot non-functional for any article that uses short citations. And what about plain extlinks?

In T183087#3954723, @Rammanojpotla wrote:

Removing " or #wikiPreview (live preview root) " in the documentation would solve this right?

Well, the code I looked at would be…

Ok, getting sufficiently annoyed with the status quo, and inspired by the Wikisource gadget, I hacked up a user script for enwiki to illustrate the point.

Note that this is likely to affect every single account registered before SUL (so 2007/2008), and those that have stuck around the projects for 10+ years may be "high value targets" for TWL (I'm guessing; I may be wrong). It is also likely to affect every tool that needs the age of the account, and not just the Library Card Platform (to wit: it hit the IABot Management Interface, cf. my comment above; and the "Popups" gadget on enwiki exhibits similar symptoms).

One possible cause for this, cf. T164974, is that the user account is old and has no registration date. Might be worth checking as trigger anyway.

In T163098#3190792, @Isarra wrote:

Don't discount what's possible due to politics - the politics across the projects vary considerably, and Wikisource actually has this already: https://en.wikisource.org/wiki/Wikisource:Scriptorium/Archives/2015-07#New_Gadget. That may be a gadget, but there's definitely demand for this being properly implemented in the skins themselves.

In T163098#3190753, @Isarra wrote:

• View/edit/etc would likely make more sense as tabs (cactions menu). CentralNotice is an another example of a special page with extra tabs on, so there is precedent, and this would be a use that would be very clear to users.