Page MenuHomePhabricator

Languages are bundled unnecessarily in HTML
Closed, ResolvedPublic

Description

Interwikis for [[testwiki:San Francisco]] take almost 13K even though this feature is not heavily used(?). Ideally, we should load them with JS only when user needs them while people without JS will see just a link to a special page with interwikis for a given page.


Version: unspecified
Severity: enhancement

Details

Reference
bz40678

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 12:54 AM
bzimport set Reference to bz40678.
bzimport added a subscriber: Unknown Object (MLST).

So if I understand correctly:

  • We add the text "This article is available in <a>X languages</a>" where the <a> tag links to a page listing all the different language versions.
    • What should this link to? If a Special page is it is an issue if this doesn't cache?
  • In javascript when clicked an ajax request is triggered that requests languages
    • what happens whilst the request is in process - do we see a loader?
      • is a delay acceptable?
  • If an article has no alternative languages do we show the text or not?

Note that in beta mode, we have another way to look at this - if languages are treated as a section, then we could retrieve them on initial page load and only add to the DOM when the section is toggled open / button is clicked.

So for this to work we can use javascript but need a fallback for browsers not running javascript for whatever reason.

Ideally we'd need a page that lists all the languages for a given article
e.g.
San Francisco/Languages
or Special:AvailableLanguages/San Francisco

The javascript code already exists in beta.

In beta/alpha the language overlay this is particularly a problem as it currently also requests via the api language versions of the page.

Ideally we should replace the html for the language section in beta with a link 'View available languages'.

When clicked without javascript/slow connection the 'Read in another language button' should open
Special:PageLanguages?title=San Francisco which simply lists all the available languages that the given article in title parameter is in

When javascript kicks in it opens the overlay via data from the api.

The additional javascript call now doesn't apply. Marking as enhancement as although solving this will reduce the payload for all alpha, beta and stable users it is not urgent

Generation of language code is heavily tied to the parser. I fear that it will be difficult to create a special page that does this without some upstream changes.

An alternative way of doing this might be to introduce a new action value and not think of this a special page - action=languages that allows us to share the parser code more easily. I still think a special page would be a better solution though.

This sounds like it will be useful for zero as well as they can more easily overwrite urls..

@Jon, there is already prop=langlinks...

Example: https://en.wikipedia.org/w/api.php?action=query&prop=langlinks&format=json&lllimit=500&titles=Apple

It seems like a solution might be to create Special:Languages/<pagename> that just pretty prints the contents of that API query, unless I misunderstand you.

bingle-admin wrote:

Prioritization and scheduling of this bug is tracked on Mingle card https://wikimedia.mingle.thoughtworks.com/projects/mobile/cards/1573

@Theopolisme exactly - the solution would strip out the language links currently printed at the bottom of the page and replace them with
<a href="wiki/Special:Languages/Article Title" class="button">read in another language</a> (JavaScript would hijack this to render the overlay)

The reader experience would be no different to how it is currently, however the page size would dramatically decrease.

Any work towards realising this would be much appreciated (for example the creation of the special page)

Hello, I'm from GCI, I'm going to try this task out.
I know very little web-related stuff, but I figured I want to challenge myself and learn some new things (that's the point of GCI, right?).

If you know very little web related stuff this might be a very tricky one to start with.. although it will certainly challenge you!

@Jon, a few things.

I've created Special:Languages/<pagename> which returns both the languages and the language variants available for any given page.

As far as the JavaScript overlay, what I've written right now just loads Special:Languages for the given page and then parses it to get the language links. This is more efficient and allows us to remove all of the getPageLanguages JavaScript stuff... who isn't for removing code? ;)

But now for a question. Right now I've implemented all of this stuff directly on the stable version...but reflecting on what I've done, would it make more sense to do it as a beta feature initially (given that a lot of code is being touched)? I don't know what the procedures are exactly (and how you make that determination), so please let me know how you'd like me to proceed.

Once you let me know I'll move the code around as necessary and get the initial patch uploaded.

Thanks!

Great stuff Theo!

Definitely as a beta feature first. If you push your changes I can help guide you with how to slot it all together.

We particularly have to be careful here as it changes the resulting HTML for all users which will have cache implications!

Hi!

Still having some trouble getting my interwiki/langlinks all set up. I've added rows to the interwiki table for each language and added some interwiki links to a test page. When viewing the test page, the interwiki links all display correctly in the sidebar.

However, when I go to /api.php?action=query&prop=langlinks&llurl=true&lllimit=max&titles=Test4 the language links don't appear. Same with /api.php?action=query&prop=iwlinks&titles=Test4

Is there some magical final step to get the langlinks api to work? (I've already run the update.php script, to no avail...)

Thanks :)

I'm a bit confused - why not implement this purely in javascript to call the api? Why do we need a Special:Languages page?

as a fallback

  1. to be good web citizens
  2. to support users without JavaScript (yes these do exist)
  3. to provide access to this information when bit servers are down
  4. to support users who run browsers that we do not support in ResourceLoader

Sorry, makes perfect sense - we use the same strategy in zero with hijacking and using an api call. Must have not been fully awake :)

@Yuri, any insights on the langlinks API? Right now I've written the code but I can't test it :p

(In reply to comment #20)

@Yuri, any insights on the langlinks API? Right now I've written the code
but I
can't test it :p

Strange - API gets its info from langlinks table directly, so there shouldn't be any magic going on. Use the $wgDebugToolbar=true to see what SQL query is actually made against that table, and see if that data is actually there.

So it looks like the langlinks table isn't propagated, but the interwiki table is.

The API query for langlinks:

SELECT ll_from,ll_lang,ll_title FROM langlinks WHERE ll_from = '27' ORDER BY ll_lang LIMIT 5001

This smells strongly of some maintenance script that needs to be run.

  • goes to sift through /maintenance *

UPDATE: It was! Through trial and error I ran clearInterwikiCache.php and refreshLinks.php and then the api worked properly. Yay :)

Change 103739 had a related patch set uploaded by Theopolisme:
Add language (interwiki) links on demand

https://gerrit.wikimedia.org/r/103739

Change 103849 had a related patch set uploaded by Theopolisme:
Add Special:Languages

https://gerrit.wikimedia.org/r/103849

Change 103849 merged by jenkins-bot:
Add Special:MobileLanguages

https://gerrit.wikimedia.org/r/103849

Change 103739 merged by jenkins-bot:
Use Special:MobileLanguages in beta/alpha

https://gerrit.wikimedia.org/r/103739

Thanks to Theo this is now in beta which is super exciting :D

Two problems to solve before pushing to stable

  1. variants do not show - see bug 58988
  2. The language button always shows now - this is a question for design - what should happen if the language button is clicked and no languages are available? (ccing Jared)

Zero note: we replace each langlink with a magical "special:zero?redirect=xxx" where the xxx is the langlink. Later, the zero dynamically catches all such redirects and shows an js overlay. If this info is now coming from the mobile special page and a langlinks api, we need to rethink the interception strategy.

Are you sure the language button always shows? In my tests, it only appears when the page has langlinks...

Could you now simply disable the JavaScript in Zero and rewrite the Special:MobileLanguages URL?

In alpha yes: all searches result in changing the page context and loading the page via JavaScript - in which case the language button will be created regardless. Not a big deal but a slight change in behaviour that we need to think through. Worth noting that now it is only when you click on the language icon that you know if a page has languages or not (previously this was determined beforehand)

Now languages are lazily loaded. This is great!

There's only one odd thing with existing behaviour:

If you edit a page with no languages after saving you will you see a button which opens an empty language list. We don't know how many languages are available until that button is pressed...

In this situation we should either print a line instead saying "No languages available." or show the existing language in that list. Jared thoughts?

Once we wrap that up after testing this in the wild for a bit we should push the changes to stable!

I guess we should keep things simple and not render the language button when languages are not available (as it currently does)
Opened a bug and added it as a dependency.

Now the dependency is cleared this code can be pushed to stable. We should ensure it has been in production for at least a week on English Wikipedia and there have been new performance issues but if someone wants to work on that patch they can feel free!

https://gerrit.wikimedia.org/r/#/c/112619/ was deployed to cover the Wikipedia Zero case - the user will just be sent to the Special: page and the dynamic loading will be disabled. This still has the overall perf advantage spread across the corpus of articles.

Change 115978 had a related patch set uploaded by Jdlrobson:
Card 703: Promote lazy languages to stable

https://gerrit.wikimedia.org/r/115978

Yeyyy! Thanks to everyone specifically Theo for getting this done! Big bug killed :D

Change 115978 merged by jenkins-bot:
Card 703: Promote lazy languages to stable

https://gerrit.wikimedia.org/r/115978