Page MenuHomePhabricator

rest api incorrectly places references
Closed, ResolvedPublicBUG REPORT

Description

The rest_v1 api used in Wikisource places references at the end of each transcluded <pages ... /> section instaad of in the <references/> tag location. It breaks WS-Export which relies on this api.

Examples:
https://pl.wikisource.org/api/rest_v1/page/html/Epub_Test
vs.
https://pl.wikisource.org/wiki/Epub_Test
(references in the middle of text instead of after the "Przypisy" wiki section mark)

https://fr.wikisource.org/api/rest_v1/page/html/La_Vie_de_saint_Alexis
vs.
https://fr.wikisource.org/wiki/La_Vie_de_saint_Alexis
(the reference in the middle of text instead of at the end of the page)

Unfortunately, I cannot say when the problem was introduced :( Probably in the last few weeks or months.

Event Timeline

Aklapper renamed this task from rest api incorectly places references to rest api incorrectly places references.Oct 4 2022, 6:47 AM

If there is no chance to fix the problem quickly in API, maybe a workaround in WS-Export may be introduced?

At the moment, all Wikisource-based e-books which contain references seem to be broken.

I did some tests, the problem does not occur when we transclude content using the standard way:
https://pl.wikisource.org/api/rest_v1/page/html/Rest_api_test_01 (OK)
vs
https://pl.wikisource.org/wiki/Rest_api_test_01 (see: https://pl.wikisource.org/w/index.php?title=Rest_api_test_01&action=edit)

However, transclusion using the tag (used on ws) <pages ... /> the problem occurs:
https://pl.wikisource.org/api/rest_v1/page/html/Epub_Test (NOK!)
vs
https://pl.wikisource.org/wiki/Epub_Test (see: https://pl.wikisource.org/w/index.php?title=Epub_Test&action=edit)

or

https://pl.wikisource.org/api/rest_v1/page/html/Mohikanowie%20paryscy%2Fca%C5%82o%C5%9B%C4%87 (NOK)
vs
https://pl.wikisource.org/wiki/Mohikanowie_paryscy/ca%C5%82o%C5%9B%C4%87

This indicates, in my opinion, a problem related to the handling of the Prooofread extension by rest_api or the parser - someone familiar with the Proofread extension should also look at this.

I kindly ask for any help - currently, 100% of e-books with footnotes generated by wikisources (pl, en, fr ...) are incorrect.

Tpt subscribed.

After some investigation, it seems this bug has been introduced by the migration to Parsoid. It's likely that Parsoid fully parses wikitext from custom MW tags like <pages>, including references, before including them in Parsoid output. Adding native support of <pages> to Parsoid might fix this problem (or not, not sure...).

Yes, any extension that deals with wikitext will need to implement functionality for Parsoid. See https://www.mediawiki.org/wiki/Parsoid/Extension_API#Parsoid_API_for_extensions ... We probably need a phab task for Pages that is a child task of T258838

Yes, any extension that deals with wikitext will need to implement functionality for Parsoid. See https://www.mediawiki.org/wiki/Parsoid/Extension_API#Parsoid_API_for_extensions ... We probably need a phab task for Pages that is a child task of T258838

We definitely need such task to resolve the problem before Parsoid becomes the main content parser in Wikisources. Otherwise many pages will be silently broken like many ebooks have been broken for few months, now. Who can/should do this?

cscott added subscribers: Jgiannelos, cscott.

Tagging @Jgiannelos on this who is working on the Parsoid port of ProofreadPage. This might Just Work once his port is complete, but if additional work is needed we can re-triage it.

(This bug was flagged by a community member after the Parsoid Read Views talk at Wikimania 2025.)

MSantos triaged this task as Medium priority.Fri, Nov 21, 10:27 AM

After enabling the flag to allow parsoid support on proofread page it looks like the issues are fixed. References are rendered at the end of the article.