Page MenuHomePhabricator
Create Task
Maniphest T256392

Wikisource Export: Migrate WSExport Gadget to Wikisource Extension
Closed, ResolvedPublic5 Estimated Story PointsNov 4 2020
Actions

Description

As a Wikisource user, I want the WSExport gadget to be migrated to the Wikisource extension, so that I can access links to download books (via WSExport) on all wikis with the extension enabled.

Background: In the process of consulting with Wikisource users on the project talk page, we have learned that some users don't have the ability to click download links (such as "Download as MOBI") via the side panel and other places. This is the case on Czech Wikisource, for example. This is because they do not have the WSExport gadget enabled on their wiki. Rather than enabling it on each wiki, it would be more efficient to migrate the gadget over to the Wikisource extension. That way, all wikis that have the extension enabled can have access to the gadget. One important note about this ticket is that it overlaps with another wish (Migrate Wikisource specific edit tools from gadgets to Wikisource extension). However, it may make sense to include this as part of the ebook export improvement project, as many users cannot easily download books in their wikis at this time.

Acceptance Criteria:

Pages to be updated

Gadgets removed
These gadgets have been removed, and only (optionally) need to be deleted by a sysadmin or local sysop:

stockText gadget
These apparently may not need any updates:

Mobile.js
These add a link to export to the mobile interface. The extension does not do this right now, so we have left these as-is:

Details

Due Date
Nov 4 2020, 5:00 AM
SubjectRepoBranchLines +/-
Don't show export links on pages that don't existmediawiki/extensions/Wikisourcemaster+2 -1
Add sidebar links to Wikisource Export toolmediawiki/extensions/Wikisourcemaster+223 -9
Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 25 2020, 4:44 PM
ifried renamed this task from Wikisource: migrate wsexport gadget to wikisource extension to Wikisource: migrate wsexport gadget to wikisource extension [placeholder].Jun 25 2020, 4:44 PM
ifried updated the task description. (Show Details)Jun 25 2020, 7:52 PM
ifried updated the task description. (Show Details)Jun 25 2020, 8:32 PM
ifried updated the task description. (Show Details)Jun 25 2020, 8:37 PM
ifried updated the task description. (Show Details)
ifried renamed this task from Wikisource: migrate wsexport gadget to wikisource extension [placeholder] to Wikisource: Migrate WSExport Gadget to Wikisource Extension.Jun 25 2020, 8:56 PM
JAnD subscribed.Jun 30 2020, 7:24 AM
ifried renamed this task from Wikisource: Migrate WSExport Gadget to Wikisource Extension to Wikisource Export: Migrate WSExport Gadget to Wikisource Extension.Oct 15 2020, 10:18 PM
ARamirez_WMF set the point value for this task to 5.Oct 15 2020, 11:48 PM
Samwilson mentioned this in T264122: Wikisource Ebooks: investigate if we can prevent automated downloads (to improve reliability) [8H].Oct 19 2020, 5:14 AM
ifried moved this task from Needs Discussion to Up Next (June 3-21) on the Community-Tech board.Oct 22 2020, 4:00 PM
ifried moved this task from Up Next (June 3-21) to Kanban-2020-21-Q2 on the Community-Tech board.Oct 22 2020, 6:13 PM
ifried edited projects, added Community-Tech (Kanban-2020-21-Q2); removed Community-Tech.
ARamirez_WMF changed the subtype of this task from "Task" to "Deadline".Oct 22 2020, 7:40 PM
ARamirez_WMF set Due Date to Nov 4 2020, 5:00 AM.
Samwilson claimed this task.Nov 5 2020, 3:22 AM
Samwilson moved this task from Ready 🎬 to In Development 💻 on the Community-Tech (Kanban-2020-21-Q2) board.
Samwilson added a comment.Nov 6 2020, 3:23 AM

Different Wikisources show different formats in the sidebar links. @ifried Should we make it configurable to match the status quo, or just have a set list? Also, which namespaces should the links be shown for? I was wondering if we should show it on all $wgContentNamespaces. I haven't actually checked them all, but it looks like lots of the 114s below are just because they copied the gadget code from enwikisource.

WikisourceNamespacesFormats
bn0, 114epub, mobi, rtf, txt, pdf-a5, choose
en0, 114epub-3, mobi, choose
fr0epub, mobi
gu0, 114epub-3, mobi, choose
hy0, 114epub
kn0, 114epub-3, mobi, choose
nl0, 114epub
pa0, 114epub-3, mobi, choose
pt0, 108epub
sa0, 114epub-3, mobi, choose
ta0, 114epub-3, mobi, choose
th0, 114epub-3, mobi, choose
tr0, 114epub-3, mobi, choose
uk0, 114epub-3, mobi, choose
vi0, 114epub-3, mobi, choose
zh0, 114epub-3, mobi, choose
gerritbot added a comment.Nov 13 2020, 5:05 AM

Change 640867 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/extensions/Wikisource@master] Add sidebar links to Wikisource Export tool

https://gerrit.wikimedia.org/r/640867

gerritbot added a project: Patch-For-Review.Nov 13 2020, 5:05 AM
Samwilson mentioned this in T255790: Wikisource: Replace ElectronPDF with WSExport PDF support.Nov 16 2020, 9:54 AM
Aklapper added a parent task: T255790: Wikisource: Replace ElectronPDF with WSExport PDF support.Nov 16 2020, 9:59 AM
Samwilson added a comment.Nov 17 2020, 5:12 AM

@ifried another question: should the extension prevent duplicate links from appearing? Or is it okay to have duplicates display for the short time between the new extension version being deployed and the old gadgets being turned off?

I'd thought to start with that we could look to see if the gadget is enabled, and if it is then not add the new links. But the links are not always added by a gadget (see full list below of every JS page that mentions 'wsexport'), and so we won't be able to prevent them all. I think it's probably okay to just not add them if either the WSexport or ePubDownloadLink gadget is enabled; sound okay?

Samwilson moved this task from In Development 💻 to Review/Feedback 💬 on the Community-Tech (Kanban-2020-21-Q2) board.Nov 17 2020, 5:31 AM
ifried added a comment.EditedNov 18 2020, 12:01 AM

We discussed this in our standup today, and we agreed upon the following:

  • We should post on relevant scriptoriums our plan, which is:
    • To remove cases of double links in sidebar
    • Display links to 3 file types ('Download as PDF', 'Download as EPUB,' 'Download as MOBI')
    • Perhaps change link name from "Choose format" to "Other formats" or "More formats"
  • We should see if people respond with general approval or if they have concerns
Samwilson added a comment.Nov 18 2020, 8:35 AM

The other text to finalize is that for the tooltips. At the moment it's:

LinkTooltip
Download EPUBDownload an EPUB version of this work
Download MobiDownload a Mobi version of this work
Download PDFDownload a PDF version of this work
Other formatOpen a form to choose other formats and fonts etc.

Is 'Mobi' capitalized like that? Or should it be MOBI? https://en.wikipedia.org/wiki/Mobipocket

ifried added a comment.Dec 21 2020, 6:31 PM

@Samwilson Thanks for bringing this up and apologies for the delay in response! While it seems like either Mobi or MOBI is acceptable, based on some quick research online, I think it probably makes sense to use MOBI (all caps). The other formats are capitalized in Wikisource, so it's better UX to have the same format. Also, I have found that some other online tools that deal with exporting or converting ebook file types use "MOBI" in all caps, so this would be a familiar user experience to many users.

Samwilson mentioned this in T270627: Add Archive: namespace to Ukrainian Wikisource.Dec 22 2020, 2:27 AM
Samwilson added a comment.Dec 22 2020, 2:29 AM

Thanks @ifried I've updated the above patch.

Once it's merged and deployed we can go around and disable the gadgets in MediaWiki:Gadgets-definition on all the above wikis.

gerritbot added a comment.Jan 8 2021, 9:16 PM

Change 640867 merged by jenkins-bot:
[mediawiki/extensions/Wikisource@master] Add sidebar links to Wikisource Export tool

https://gerrit.wikimedia.org/r/640867

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.26; 2021-01-12).Jan 8 2021, 10:00 PM
Maintenance_bot removed a project: Patch-For-Review.Jan 8 2021, 10:10 PM
HMonroy moved this task from Review/Feedback 💬 to QA 🐛 on the Community-Tech (Kanban-2020-21-Q2) board.Jan 11 2021, 5:37 PM
ARamirez_WMF edited projects, added Community-Tech (Kanban-2020-21-Q3); removed Community-Tech (Kanban-2020-21-Q2).Jan 12 2021, 8:24 PM
ARamirez_WMF moved this task from Ready 🎬 to QA 🐛 on the Community-Tech (Kanban-2020-21-Q3) board.Jan 12 2021, 8:30 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 3:59 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:03 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:07 PM
MusikAnimal updated the task description. (Show Details)
Tpt updated the task description. (Show Details)Jan 13 2021, 4:11 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:12 PM
Tpt updated the task description. (Show Details)Jan 13 2021, 4:15 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:17 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:19 PM
MusikAnimal updated the task description. (Show Details)
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:24 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:26 PM
MusikAnimal updated the task description. (Show Details)
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:37 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:44 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:47 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:51 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 4:56 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 5:03 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 5:18 PM
MusikAnimal updated the task description. (Show Details)Jan 13 2021, 5:21 PM
Samwilson added a comment.Jan 13 2021, 11:32 PM

This is ready for QA now.

Bodhisattwa updated the task description. (Show Details)Jan 14 2021, 7:10 AM
dom_walden moved this task from QA 🐛 to Product sign-off 🤘 on the Community-Tech (Kanban-2020-21-Q3) board.Jan 15 2021, 4:09 PM
dom_walden subscribed.

The Wikisource extension replaces the various WSExport download links in the sidebar, which were previously implemented via a gadget. E.g.

en_sidebar_link.png (920×746 px, 68 KB)

I have been able to use the sidebar links to export an epub on each of the wikisources in the "Pages to be updated" section above.

As far as I can tell we have removed the old gadget from each of the wikisources as well. If the old gadget is still in use on a site, the Wikisource extension will not add its own links.

We don't show the links on https://wikisource.org. I don't know if this matters. I don't know if users use that site.

I have briefly tested it on each skin. The links do not appear on Minerva, but they did not before on the gadget either.

The links do not appear on the mobile version of the site either, but many have custom gadgets/JS which add export links. We haven't touched these. E.g.

mobile_epub_link.png (586×1 px, 40 KB)

Some wikisources have banner which include wsexport links (the stockText gadget). We haven't touched this. E.g.

fr_stocktext_banner.png (589×1 px, 76 KB)

Currently, the sidebar links will only work for books in the main namespace. This will be fixed in T272038.

There are a few bugs I have raised:

Test environment Wikisource – (0c0e97a) 07:36, 11 January 2021.

Tpt subscribed.Jan 15 2021, 4:36 PM

We don't show the links on https://wikisource.org. I don't know if this matters. I don't know if users use that site.

I believe we should. It's the incubator for Wikisources and there was a wexport gadget their. I believe the Wikisource extension should be installed there too. I have opened T272163 about it.

Zdzislaw subscribed.EditedJan 16 2021, 1:43 PM

@Samwilson First, many many thanks for the development of wsexport!
I would like to make two points. Have you considered adding a target = "_blank" attribute to the "Other formats" link so as not to "lose" the Wikisource page visitor? And ... do you consider selecting (configuring) a set of formats (links) for a given wiki?

One more thing - I think there should be no links to ws-export on non-existing pages, e.g. The Travels of Scarmentado.

gerritbot added a comment.Jan 18 2021, 5:17 AM

Change 656649 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/extensions/Wikisource@master] Don't show export links on pages that don't exist

https://gerrit.wikimedia.org/r/656649

gerritbot added a project: Patch-For-Review.Jan 18 2021, 5:17 AM
Samwilson mentioned this in T272253: WS Export: open 'choose formats' link in new tab.Jan 18 2021, 5:19 AM
Samwilson mentioned this in T272254: WS Export: allow wikis to specify which export links are shown.Jan 18 2021, 5:30 AM

Thanks for the feedback!

I would like to make two points. Have you considered adding a target = "_blank" attribute to the "Other formats" link so as not to "lose" the Wikisource page visitor?

I've raised this: T272253: WS Export: open 'choose formats' link in new tab

And ... do you consider selecting (configuring) a set of formats (links) for a given wiki?

I've raised this: T272254: WS Export: allow wikis to specify which export links are shown

One more thing - I think there should be no links to ws-export on non-existing pages, e.g. The Travels of Scarmentado.

Good idea. The above patch does this and should be deployed soon.

gerritbot added a comment.Jan 18 2021, 7:44 AM

Change 656649 merged by jenkins-bot:
[mediawiki/extensions/Wikisource@master] Don't show export links on pages that don't exist

https://gerrit.wikimedia.org/r/656649

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.27; 2021-01-19); removed MW-1.36-notes (1.36.0-wmf.26; 2021-01-12).Jan 18 2021, 8:00 AM
Maintenance_bot removed a project: Patch-For-Review.Jan 18 2021, 8:10 AM
ifried closed this task as Resolved.Jan 26 2021, 11:21 PM
ifried moved this task from Product sign-off 🤘 to Done 🏁 on the Community-Tech (Kanban-2020-21-Q3) board.

Thank you, everyone, who wrote tickets to address the various questions and concerns that came out of this work, which we will individually analyze and prioritize as a team. As this work is now on production, I'm marking it as Done.

Bebiezaza updated the task description. (Show Details)Jun 4 2022, 9:24 AM
Restricted Application added subscribers: Ericliu1912, Stang. · View Herald TranscriptJun 4 2022, 9:24 AM
Stang unsubscribed.Jun 4 2022, 7:11 PM
tstarling subscribed.Feb 22 2024, 12:07 AM
		// Do not add the export links if the user has a gadget that does the same.
		// @TODO Remove this after all these gadgets have been removed.
$ for w in $(expanddblist wikisource); do echo $w; echo 'print_r(array_intersect(\MediaWiki\Extension\Gadgets\GadgetRepo::singleton()->getGadgetIds(), [ '\''ePubDownloadLink'\'', '\''WSexport'\'' ]));' |  mwscript eval.php --wiki=$w; done
...
mrwikisource
Array
(
    [20] => WSexport
)
...
tewikisource
Array
(
    [34] => WSexport
)
...

Pages to edit:

Restricted Application edited projects, added Community-Tech; removed Community-Tech (Kanban-2020-21-Q3). · View Herald TranscriptFeb 22 2024, 12:07 AM
Bodhisattwa updated the task description. (Show Details)Feb 22 2024, 4:39 AM
Bodhisattwa updated the task description. (Show Details)Feb 22 2024, 5:07 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL