In T362502#9720490, @Uzume wrote:

If the issue goes away with a purge there isn't really an issue with the page anyway. The issue is with a stale page cache. If you manage to update a page with such an error in order add processing to catch and categorize such an error, then the error would already have gone away because you necessarily had to update the cached page by editing the page either directly or through one of its transclusions.

Also the issue seems to not be in the Proofread "Index" page but rather in the "File" page which are mostly on Commons.

Even though a dictionary with, 100,000 entries with 300 bytes per file_url/filename would use 30MB of memory. Hardly an issue if you want to upload 100,000 files at once.

My bad

Ok

The post_processor function is great and I use it now in my code. I see that this is way more flexible when uploading multiple files. In my case I work with one book file at a time, so no need to do that, and in my mind it looked cleaner to just grab something instead of defining functions and such.
But I insist, isn't even more cleaner to just be able to grab the filename using the file_url?

I think it's a little convoluted but after a while I managed to get it working for my specific purpose.

In T360837#9676200, @Xqt wrote:

Both UploadRobot.process_filename() and UploadRobot.upload() already gives the filename back to the calling process (which is UploadRobot.run() by default. I've no glue for what the last_filename attribute is good for. Could you pleas explain a bit more.

Can confirm:

1. that the problem is present both at Commons and every local wiki
2. ?action=purge on a local description page (e.g. Spanish Wikisource Archivo:Filename.pdf) solves the problem locally, but only after '?action=purge'ing the Commons description page.

In T284504#7140995, @Tpt wrote:

@Tpt Is there a technical reason for this?

The <pages> tag implementation fetches the Index: page content from the database. Using it in the actual index that is used by the <pages> tag will lead to the use of the currently saved version of the index page on e.g. preview. So, to avoid strange bugs for the Wikisource contributors, enabling <pages> in the index pages requires a bit more than just removing this validation check.

&action=purge seems to solve this issue. Can we search for other PDF files that have this issue?

I guess this is related, but I had never experienced this bug before, and yesterday I uploaded a lot of books, and at least 5 had this issue

Loss of mouse scrolling is a major problem right now, specially for big books with small fonts, where lots of zooming in/out and scrolling is needed.

I ended up creating a custom JS script on my side that added two links. The page+2 is specially useful

I support this, and in I've seen an implementation of this in other digital libraries

Also, it doesn't "remember" your choice. When you finally find a good one, then the next day it will be hard to remember which one was it ;)

I did some test using this script

In T358635#9584134, @Xqt wrote:

In T358635#9583971, @Ninovolador wrote:

In T358635#9583360, @Xqt wrote:

@Ninovolador: you mean that the content is loaded anyway even that flag is set to False?

It doesn't work at all. It does not work as a genetator. No prompt or console output, just hangs

No, it works, for example:

>>> import pywikibot
>>> from pywikibot.proofreadpage import IndexPage
>>> site = pywikibot.Site('wikisource:en')
>>> ip = IndexPage(site, 'index:Popular Science Monthly Volume 1.djvu')
>>> gen = ip.page_gen(content=False)
>>> next(gen)
ProofreadPage('Page:Popular Science Monthly Volume 1.djvu/2')

Maybe it needs a lot of time to load all the pages. But this is due to the sorting funtion; propose to remove that in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1007342

In T358635#9583360, @Xqt wrote:

@Ninovolador: you mean that the content is loaded anyway even that flag is set to False?

It doesn't work at all. It does not work as a genetator. No prompt or console output, just hangs

I have the exact same problem. I just needed an easy way to loop through PDF files in a Category and it cannot be done because of this bug... Why wasn't the patch implemented?

In T352524#9374563, @Samwilson wrote:

Maybe someone at Wikimedia OCR project can tell us how they manage to get the full resolution image from every page

The Wikisource extension uses the same image that's already in the Page namespace page (i.e. from ProofreadPage). The width of this image can be customized per Index page, but is usually somewhere around 1000 pixels.

In T301985#7720422, @Samwilson wrote:

Good idea!

What particular features are you using in the advanced mode? Because it might be a nicer workflow to actually make it unnecessary to open the advanced mode and to make those features available directly in the page editor on-wiki. For example, T294903 is about adding the crop tool to the main image viewer, and T279405 is for better language-selection.

I've now tried to change the tabindex myself, in my User/Common.js.

In T271958#8170354, @Tgr wrote:

Not being able to save unrecognized CSS is the entire point of TemplateStyles.

At least add the possibility of saving the page. Currently it blocks you for even doing that (other "invalid" code passes just as well)

I agree. The "smaller" projects deserve more recognition

Mainly crop tool, so it would be covered by the T294903 patch when (if) it will be implemented.

Total support. It's not a rare task, but it's very time-consuming if you haven't the right tools (in this case, a bot account with admin privileges and a proper script) .

For some reason, the bug isn't present here: https://es.wikisource.org/wiki/Usuario:Ignacio_Rodr%C3%ADguez/Testpage2

This should be simple enough?

@Billinghurst Finally I did it with Ref/Note and it works fine. I guess we can call this very obscure bug out of scope.

UPDATE:
I found what it seems to be te issue.

This is very common. Most of the time we work with tabular data or whole works that have a two column layout, all the actual OCR tools fail

@Inductiveload it seems resolved. Thanks!

@Urbanecm Can you do the same check with the wikis I mentioned in the original post? eswikisource, enwikisource, eswikibooks for example.
The bug isn't currently affecting Meta...

Any chance with this change working on cross-page references?

This is a very serious bug, and a well-intentioned user can cause (reversible) damage to the pages, and frustration.
No one is going to even triage it?

Most of the recent failures are due this very same bug. See the log for any upload failure with a link for DJVU download.

Also, long dead tasks (e.g. CRITICAL errors) should die on themselves, for allowing to retry later on

An example is "Author" namespace (ns:102) on enWS. We could use the same namespace, you changed Autor to ns:104

PLEASE REVERT! It was changed namespace Index to Author
Namespace 104 was Index. We needed new namespace, not delete an existent one!!!

@XZise
Another use could be for semi-automatic bot tasks, especially for complex regular expressions that take time to process. So, you can do the process in two times. First automatically (accept all changes but don't send them online), over the night for example. Second, only work with pages that need changes and accept each one manually.

I edited the task description to be more informative.