In T226922#6804308, @MGA73 wrote:

I have not testet it but if it is only possible to move a page if there is also a file involved then this might work.

If you want to move File:A.jpg ---> File:B.jpg

1. Undelete File:B.jpg (if it is deleted and if not just skip to 2)
2. Move File:B.jpg ---> File:A.jpg
3. Delete File:A.jpg
4. Undelete File:A.jpg
5. Move File:A.jpg ---> File:B.jpg
6. Delete File:B.jpg (if it is supposed to be deleted)

In T257066#6655978, @Ycrusoe wrote:

This is not so simple. In Wikisource we convert music scores from images to make them editable and proofread the music scores. So users needs to be able to edit them some way. How are they expected to edit the resulting images if their role is just converting images into an editable form?

So the editable format is lilypond?

Yes. And the image should not replace it. The lilypond code should be still available in some way in the newest page version.
So, if it is updated, the image may also be updated.

The second link is an ogg file. Maybe you could move to midi or musicxml as editable format?

Anyway, my proposed solution was just meant as a quick and dirty hack so Wikipedia end users for the time being can at least see the scores that are currently in lilypond. These scores are usually small examples anyway, not full pieces, but they are important for end users trying to learn from Wikipedia. Once the lilypond security issue is solved, it could be reverted back to <src> (the src could actually be kept as a comment or deactivated somehow). Some scores in Wikipedia are already in image format, so I guess it wouldn't be that bad (for end users).

Making a dirty hack is better than doing nothing. If lilypond cannot be executed on WMF servers, it may be executed on an external server, in specially prepared secured, chrooted environment, and the image uploaded/updated by a bot.

In T257066#6654054, @Ycrusoe wrote:

Hi,

I'm one among a presumably significant group of people around the world trying to learn music during isolation, and I've come across this issue in Wikipedia. I wonder what is the status, since there doesn't seem to be a lot of updates lately.

From my limited technical knowledge about this, I am thinking that if the LilyPond security is very difficult to overcome, anyone with access to all the Wikipedia source (understood as the source anyone can read in the edit page of each article) could run lilypond on an offline machine with no sensitive info and generate an image for each <score> tag, and another simple script could then substitute which <score> tag with an appropriate <img> tag. A special attribute could be added so all these cases can easily be reverted to something else if the <score> issue is solved in the future.

In T257066#6598861, @Beeswaxcandle wrote:

In T257066#6576540, @FordPrefect42 wrote:

Since a couple of days, saving any article that calls the score extension leads to the score becoming invisible (without having the score itself changed). So the workaround of reading pre-rendered files from the cache seems no longer to be working. It is really annoying that the problem is still not solved after such a long time of waiting. There are currently 1,241 pages calling score on en:wikipedia, 748 on de:wikipedia which are endangered of losing the proper display of the contained scores.

And 2,161 on en:Wikisource. And 1,254 pages marked as waiting for a score to be added. In addition, we've got several books that are stalled while waiting for this update.

In T228594#6622158, @Xover wrote:

@Ankry @Koavf frWS (and several other projects) cross-load the script at mul:MediaWiki:OCR.js. Could you apply this diff from enWS (which uses a local copy)? That should, hopefully, fix the problem for most language projects that use this tool.

Just another user hit by this problem:
https://ticket.wikimedia.org/otrs/index.pl?Action=AgentTicketZoom&TicketNumber=2020090410005808

In T257625#6309763, @Aklapper wrote:

@Ankry: You can still reach all links, all links are working. This inconvenience clearly does not qualify as "Unbreak now".

Broken on all wikis, no workaround, multiple complants from users on IRC. Maybe the change that introduced the bug should be reverted if no chance for quick fix deployment?

In T128173#6214317, @Tpt wrote:

I believe that the approach would be:
For each language:

1. If there is a local inter-language link to this language in the wikitext of the page, use it and stop the lookup.
2. If there is a sitelink to this language in the Wikidata item of the page, use it and stop the lookup.
3. If there is a sitelink to this language in a Wikidata item connected to the Wikidata item of the page using edition of (P629), use it and stop the lookup.
4. If there is a sitelink to this language in a Wikidata item connected to the Wikidata item of the page using edition (P747), use it and stop the lookup.
5. If there is a sitelink to this language in a Wikidata item connected to the Wikidata work item using edition (P747), the work item being connected to the Wikidata item of the page using edition of (P629), use it.

This way, I think we cover all the use cases. 1, 2 and now 3 are already implemented.

In T128173#6213769, @Tpt wrote:

Since yesterday, if a Wikisource page is connected to a Wikidata item that states that it is an edition of the work using P629, the sitelinks of the work item are displayed on the page in the "In other languages" sidebar.
The next step is to look for the other editions on Wikidata using P747.

Just a recent example OTRS ticket related to this problem:
https://ticket.wikimedia.org/otrs/index.pl?Action=AgentTicketZoom&TicketNumber=2020041510003496

In T227590#6042974, @Ammarpad wrote:

In T227590#6042811, @Ankry wrote:

• some edits were errorneously assigned to wrong files (likely due to some bug)

In that case, the bug should be reported so that the root cause can be found and fixed to prevent more assignment of erroneous edit in the future. Masking the problem by manual workaround is not a good solution in my view.

In T227590#6042974, @Ammarpad wrote:

In T227590#6042811, @Ankry wrote:

• a file with deleted revisions was moved; and the revisions needed to be undeleted and merged with the file history after the move (they remained under the old filename)

That looks like a separate bug to me. When a file is moved, the software should move it with its complete revisions deleted or not deleted. No revision should left behind for human admin to manually later 1. undelete, 2. merge, and 3. re-delete again.
I cannot verify this bug myself though. If you can confirm that's the case, it should be reported as a distinct bug.

In T227590#6042735, @Ammarpad wrote:

What's the use case of moving such pages in File namespace? The pages should be deleted as they should not be created in the first place [1]. And indeed your two examples above have been deleted, only the pl.wikisource one remains and you created it for the purpose of this report.

[1]. The real bug here, I think, is to disallow creating pages in the File namespace completely. Commons and Enwiki (and probably other wikis) already have policies to speedily delete such pages. It should be enforced by the software.

I'd like to note that adding a newline between any pages will break other things. Eg. when an image is added on a separate page inside a paragraph:

• page1: start of paragraph text
• page2: image (as a <div>)
• page3: end of paragraph text

If a newline is inserted between pages, parser will end the paragraph before the image, inserting an extra </p> there and the actual paragraph becomes split. This is not an expected behaviour.

In T242422#5792913, @Reedy wrote:

T242425: Wikidata shows different image (file) sizes, than Commons reports for the current version seems similar/related

In T242422#5792892, @Xover wrote:

This problem coincides with deployment of MediaWiki 1.35/wmf.14 today, but I find nothing obviously connected in the changelog. A few changes mention touching calls to int(), but not in any context that would obviously be connected to this issue.

In T241824#5773583, @Billinghurst wrote:

Which user_right is required?

In T218626#5729595, @DonTrung wrote:

I don't get why the Wikimedia Foundation doesn't just implement this by default, what would we truly lose if partial blocks were implemented? Admins aren’t forced to use them when they don't want to, “giving the community a choice” just means that they (we) could prevent something that's a net benefit to it.

In fact, policies are written around the technical capabilities and limitations of the MediaWiki software, not vice versa. There is literally not a single reason why a piece of software that doesn't limit anything possible today but only expand it should not just be enabled by default, if partial blocks had the potential to damage the workflow of the community I could understand that it would require prior discussion, but this is unnecessary bureaucracy.

The current behaviour tends to cause problems and mislead users, see
https://wikisource.org/w/index.php?title=Special:Permalink/765318#Wikipedia_link_from_the_main_page_of_the_Wikisource

In T215558#5621892, @Hsarrazin wrote:

On frwikisource, we repaired dozens of files with missing pages... no purge of pages, books or files on commons works...

and the problem is only apparent for users who opened the pages before the repairing... because it is not caused by page or file cache, but by the browser cache (Firefox, IE, Chrome, Safari all give that symptom)

in each case, it was easily solved by purging the cache of the web browser (i.e. deleting the stored images of the pages).

In T218155#5508932, @jhsoby wrote:

In T218155#5508368, @jayantanth wrote:

Thanks @Ankry for your comments. Presently there are 28,307 Page: and 40+ associate Index files. So are you advice to do this again start from scratch ?

I can do this all configuration if I have interface admin access.

Ankry is only talking about the index pages, not the page pages (hehe). However, there shouldn't be any problem importing the index pages. We did that for the Neapolitan Wikisource, and it worked just like it should. As long as the parameter names used in [[MediaWiki:Proofreadpage index template]] stay the same, the index pages will work when imported. If you want to we can translate the parameter names, but there is basically no advantage to doing so, since those will never normally be read or used by humans. Doing so would also require changing the index page code while importing, and I think it's an unnecessary complication.

In T218155#5506935, @jayantanth wrote:

Could you please anyone import all the pages (ns:0)/Index files/Pages?

This does not seem to be deployed to "all wikisource". sourceswiki is a wikisource project and partial blocks are not available there.

@Peteforsyth It seems that there was a misplaced tie in the Lilypond code. After this fix:
https://en.wikisource.org/w/index.php?title=Page%3AChopin_Nocturnes_Op_9_Kistner_First_Edition_1833.djvu%2F6&type=revision&diff=9580515&oldid=7253044
it works fine.

In T64717#3224069, @Liuxinyu970226 wrote:

per krinkle

@Ebe123 @Krinkle I do not think the https://gerrit.wikimedia.org/r/495882 is correct:

1. it works only with multipage scores (well, on an inlined score image the extra spacing is not expected, but if the midi player is added the spacing is still needed)
2. it adds extra space not only before the player, but also between pages (well, in most cases it does not hurt, but see below)
3. the space is in em (current context font-size dependant). And this can sometimes give unexpcted results.

I would suggest the extra space to be in px, added only after the last page and only if the player is present.
Maybe just put the player into an extra div with non-zero margin-top?

In T212881#5341855, @Reedy wrote:

I note new wikis were never created in a timely fashion anyway...

Yes, exactly.
Just notifying wiki users about this need.

Pages need to be purged in order to display new namespace name in page title.

This is probably a side effect of T227590.

I think, this interaction between ProofreadPage and WikiEditor toolbar cannot be fixed inside ProofreadPage. The search and replace tool incorrectly assumes that all raw content of the wikicode is provided to user

In T225204#5311736, @Aklapper wrote:

I don't know what "How about first and last line?" means.

Please see T225204#5245842 and provide a clear and complete list of steps to reproduce, the expected outcome, and the actual outcome.

As suggested by @Urbanecm I also tried Special:MergePages with a strange result:

Source page:      File:T173070Ryde Tesco pedestrian walkway.JPG
Destination page: File:Ryde Tesco pedestrian walkway.JPG

In T226922#5310950, @Urbanecm wrote:

You can do this manually, by merging the wikitext by hand and then deleting the prefixed version.

@Peteforsyth, I quite often encounter this problem while uploading a new version of a multi-page file with thousands of thumbnails (generally files with >500 pages). See eg T206190 or T214759 .
I know 3 workarounds to deal with this problem in Wikisource (when this already happen):

1. change scan with in index after upload a new version (if they are 1px wider/narrower new thumbnails must be generated
2. use js to replace thumbnails for specific pages with thumbnails with other names; see example in pl.ws MadiaWiki:Commons.js :

https://pl.wikisource.org/w/index.php?title=MediaWiki:Common.js&diff=prev&oldid=2188790
(useful for few already existing pages)

3. upload the file locally

From my experience, the outdated thumbnails seem to disapear after few weeks to half a year.

@Ladsgroup Are thre any tickets describing what are the script problems that this task should depend on?

High resolution thumbnails from the file, like:
https://upload.wikimedia.org/wikipedia/commons/thumb/f/f0/%D0%9F%D1%83%D1%88%D0%BA%D0%B8%D0%BD._%D0%95%D0%B2%D0%B3%D0%B5%D0%BD%D0%B8%D0%B9_%D0%9E%D0%BD%D0%B5%D0%B3%D0%B8%D0%BD_(1837).pdf/page7-1834px-%D0%9F%D1%83%D1%88%D0%BA%D0%B8%D0%BD._%D0%95%D0%B2%D0%B3%D0%B5%D0%BD%D0%B8%D0%B9_%D0%9E%D0%BD%D0%B5%D0%B3%D0%B8%D0%BD_(1837).pdf.jpg
look poor and exhibit artifacts likely resulting from scalling-up jpg image with lossy compression

OK, after some delay the file works.

$ qpdf --check Mueller_letter_to_Barr_2019-03-27.pdf
checking Mueller_letter_to_Barr_2019-03-27.pdf
PDF Version: 1.3
File is not encrypted
File is not linearized
No syntax or stream encoding errors found; the file may still contain
errors that qpdf cannot detect

Now, qpdf does not detect errors, but thumbnails are still not generated.

Is this really fixed? I still get strange Content-Type for the thumbnail from description (however it is not [text/html] already:

$ wget -S https://upload.wikimedia.org/wikipedia/commons/thumb/5/57/Status_iucn3.1_LC_cs.svg/200px-Status_iucn3.1_LC_cs.svg.png 2>&1 | grep Content-Type
  Content-Type: application/x-www-form-urlencoded

Full headers:

HTTP/1.1 200 OK
Date: Wed, 15 May 2019 07:04:15 GMT
Content-Type: application/x-www-form-urlencoded
Content-Length: 8100
Connection: keep-alive
Etag: b485920910bc973c3fad353a9b809944
Server: ATS/8.0.3
X-Object-Meta-Sha1Base36: s73fklaf49dfygd9oug7c9kbjy5wu6h
Last-Modified: Mon, 03 Apr 2017 12:50:01 GMT
X-Timestamp: 1491223800.69668
X-Trans-Id: txc11c43e2ea804120a6f6f-005cd50dde
X-Varnish: 265855185 200635104, 101450013, 151146220 113949253
Via: 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)
Age: 8574
X-Cache: cp3038 hit, cp3038 hit/77
X-Cache-Status: hit-front
Server-Timing: cache;desc="hit-front"
Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
X-Analytics: https=1;nocookies=1
X-Client-IP: 2a00:6d47:10:b95:dad:beef:baba:dead
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache, X-Varnish
Timing-Allow-Origin: *
Accept-Ranges: bytes

I have recently implemented in plwikisource a LUA based solution of this problem.
part1:
https://pl.wikisource.org/wiki/Strona:PL_Wyspia%C5%84ski_-_Warszawianka.djvu/55
part2:
https://pl.wikisource.org/wiki/Strona:PL_Wyspia%C5%84ski_-_Warszawianka.djvu/56
merged:
https://pl.wikisource.org/wiki/Warszawianka_-_melodia
Feel free to use this concept elsewhere;
However, this is still a "work in progress" so undocumented, etc.

It seems that this problem is fixed by https://gerrit.wikimedia.org/r/499801

It seems that this fix fixed also T219371
So they were related...

In T218535#5041247, @Beeswaxcandle wrote:

I would rather deal with my five scores on enWS myself. Even if a bot amends them, I'll still have to go back into the pages to do other tasks. I would be surprised if there were any other enWS scores affected as I'm the principal editor of such and I would have been contacted by anyone else seeing the problem.

In T218535#5040598, @Ebe123 wrote:

@Ankry: It would be helpful, thanks! I didn't know about this quarry-ing :) Would be good to group by page, but also make sure that content in the score tag was modified in the timeframe, and include all wikis)

In T218535#5040209, @Ebe123 wrote:

Impossible to have a full list of broken scores, but they can be fixed by adding a comment to the scores, which will force regeneration.

I think, this still needs some maintenance to regenerate broken score images, created while the wrong configuration was active.

the problematic version restored after revi's delete and history merged on-wiki.
Is there still any phabricator-related issue here?

This is probably related to the problem described in T214759.

Well, I think I have found some correlations.

1. The 503 error is likely accidental here; it appered once or twice, I cannot reproduce it.
2. The problem concerns many (maybe all?) djvu files with a lot of thumbnails (likely much more than 1000).

Maybe also related to T206190 as there are thousands of outdated thumbnails related to this file.

Please note: PDF rendering is critical for Wikisources where users dgitize books. Providing an OCR layer instead of real page images is misleading contributors who may interpret OCR errors as print errors. Please advice about urgent fix.