Wikisource Export: Underscores show instead of spaces in some titles
Closed, ResolvedPublic3 Estimated Story PointsBUG REPORT
Actions

Assigned To

Authored By

	dom_walden
	Dec 17 2020, 10:33 AM

Description

As a Wikisource user, I want to see spaces correctly used between words in the title rather than underscores, so people's first impression of the ebook export is not to see an error.

Background: On the title page of an exported ebook, underscores are shown instead of spaces (see screenshot below).

Books to reproduce problem:

Example 1:
- Test Version: https://wsexport-test.wmflabs.org/?lang=en&page=Book_of_Jasher%2FChapter_30&format=epub-3&fonts=
- Prod Version: https://wsexport.wmflabs.org/?lang=en&page=Book_of_Jasher%2FChapter_30&format=epub-3&fonts=
Example 2:
- Test Version: https://wsexport-test.wmflabs.org/?lang=en&page=Dictionary+of+the+Swatow+dialect%2Fnng&format=epub-3&fonts=
- Prod Version: https://wsexport.wmflabs.org/?lang=en&page=Dictionary+of+the+Swatow+dialect%2Fnng&format=epub-3&fonts=

Acceptance Criteria:

Restore previous behavior to display spaces between the words in the title of ebook exports rather than underscores

Screenshots:
wsexport-test:

wsexport production:

Details

	Subject	Repo	Branch	Lines +/-
	Bump wikimedia/parsoid to 0.13.0-a23	mediawiki/vendor	master	+4 K -3 K
	Use prefixed text for content of links up the path	mediawiki/services/parsoid	master	+1 -1

Customize query in gerrit

Related Objects

Mentioned In: T276649: PHP Notice: Undefined property: stdClass::$title
T264788: Wikisource: migrate API to Parsoid's API
T270495: Wikisource Export: HTML entities like "&" rendered as "&" in ebook titles
Mentioned Here: T270495: Wikisource Export: HTML entities like "&" rendered as "&" in ebook titles

Event Timeline

dom_walden created this task.Dec 17 2020, 10:33 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 17 2020, 10:33 AM

Also ampersands are &, e.g. https://en.wikisource.org/wiki/The_life,_trial,_and_execution,_of_Wm._Booth

In T270373#6698717, @Inductiveload wrote:

Also ampersands are &, e.g. https://en.wikisource.org/wiki/The_life,_trial,_and_execution,_of_Wm._Booth

Good catch.

I think this might be an older bug, so I have raised it separately here T270495.

ifried moved this task from New & TBD Tickets to Needs Discussion on the Community-Tech board.Dec 18 2020, 7:37 PM

dom_walden mentioned this in T264788: Wikisource: migrate API to Parsoid's API.Jan 4 2021, 10:20 AM

ifried updated the task description. (Show Details)Jan 12 2021, 10:58 PM

ifried updated the task description. (Show Details)

ARamirez_WMF set the point value for this task to 3.Jan 12 2021, 11:56 PM

ARamirez_WMF moved this task from Needs Discussion to Up Next on the Community-Tech board.

ARamirez_WMF moved this task from Up Next to Kanban-2020-21-Q3 on the Community-Tech board.

ARamirez_WMF edited projects, added Community-Tech (Kanban-2020-21-Q3); removed Community-Tech.

This seems to be a difference in how Parsoid handles links of the form [[../]]. With the MediaWiki parser (source) we get:

<a href="/wiki/Book_of_Jasher" title="Book of Jasher">Book of Jasher</a>

but with Parsoid (source), it's:

<a rel="mw:WikiLink" href="./Book_of_Jasher" title="Book of Jasher">Book_of_Jasher</a>

Note the underscores in the text part.

We could fix this in our tool, but I do wonder if it's intentional in Parsoid.

Samwilson claimed this task.Jan 14 2021, 3:23 AM

Samwilson moved this task from Ready 🎬 to In Development 💻 on the Community-Tech (Kanban-2020-21-Q3) board.

@ssastry do you have any insight into the above?

I'll raise this in our next weekly meeting and one of us will follow up.

What's the wikitext source? For [[Chapter 30]] Parsoid should definitely be outputting <a rel="mw:WikiLink" href="./Book_of_Jasher" title="Book of Jasher">Book of Jasher</a> and be totally compatible with the core parser output:

$ echo "[[Chapter 30]]" | php bin/parse.php  --normalize

<p><a href="Chapter_30" title="Chapter 30">Chapter 30</a></p>

The title output should be consistent between the two parsers as well. The problem seems to be in the export tool, not anything in Parsoid?

In T270373#6758772, @cscott wrote:

What's the wikitext source?

It's listed above. This is reproducible with,

echo "[[../]]" | php bin/parse.php --pageName "Book of Jasher/Chapter 30" --domain "en.wikisource.org"
<p data-parsoid='{"dsr":[0,7,0,0]}'><a rel="mw:WikiLink" href="./Book_of_Jasher" title="Book of Jasher" data-parsoid='{"stx":"simple","a":{"href":"./Book_of_Jasher"},"sa":{"href":"../"},"dsr":[0,7,2,2]}'>Book_of_Jasher</a></p>

Change 657201 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Use prefixed text for content of links up the path

https://gerrit.wikimedia.org/r/657201

gerritbot added a project: Patch-For-Review.Jan 19 2021, 10:34 PM

ARamirez_WMF removed Samwilson as the assignee of this task.Jan 19 2021, 11:18 PM

ARamirez_WMF moved this task from Kanban-2020-21-Q3 to Older: Tracking Work by Others on the Community-Tech board.

ARamirez_WMF edited projects, added Community-Tech; removed Community-Tech (Kanban-2020-21-Q3).

Change 657201 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Use prefixed text for content of links up the path

https://gerrit.wikimedia.org/r/657201

Arlolra closed this task as Resolved.Jan 20 2021, 7:38 PM

Arlolra claimed this task.

Maintenance_bot removed a project: Patch-For-Review.Jan 20 2021, 8:10 PM

Change 658462 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a23

https://gerrit.wikimedia.org/r/658462

gerritbot added a project: Patch-For-Review.Jan 25 2021, 11:48 PM

Change 658462 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a23

https://gerrit.wikimedia.org/r/658462

Maintenance_bot removed a project: Patch-For-Review.Jan 26 2021, 1:10 AM

Umherirrender mentioned this in T276649: PHP Notice: Undefined property: stdClass::$title.Mar 6 2021, 3:04 PM