Page MenuHomePhabricator

Show SVGs in page language if available
Closed, ResolvedPublic13 Story Points

Description

To help communities use translated SVGs created by SVG Translate Tool, Parser should by default use page language for thumbnails. Currently, the default language is used unless the user explicitly specified a language, e.g. [[File:Foo.svg|lang=sq]].

The full discussion of how this could look is at T202181: [Spike 4 hours] Investigate the work involved in defaulting SVGs to show wiki language if available, the technical plan is the following:

  • Create a way for Parser::makeImage to let the media handler to know page's content language, as controlled by a feature flag.
  • Because varying all SVGs for all languages would generate a lot of distinct thumbnail URLs and thus create a huge load on caches and scalers, SvgHandler should be aware of languages actually present in the image and not create image URLs varied by languages needlessly.
    • Caveat: this requires loading image metadata, need to make sure performance will not suffer.

Potential production impact: while currently most SVG thumbnails are in form of https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/Neural_crest.svg/450px-Neural_crest.svg.png i.e. using default language, this change will result in more language-specific thumbs like https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/Neural_crest.svg/langru-450px-Neural_crest.svg.png.

  • Currently, ~2.6% of files on Commons are SVGs.
  • Current number of translated ones of them is negligible, but this project aims to increase this number.
  • The plan above requires that language-specific URLs will be generated only when a translation in the requested language is present.

Event Timeline

MaxSem created this task.Sep 20 2018, 9:54 PM
Restricted Application added a project: Operations. · View Herald TranscriptSep 20 2018, 9:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Niharika renamed this task from Show SVGs in in wiki language if available to Show SVGs in wiki language if available.Sep 21 2018, 1:02 PM
Niharika moved this task from Untriaged to To be estimated/discussed on the Community-Tech board.

@MaxSem @Mooeypoo If this ticket is a lot of work (sounds like it), it might be better to split it in smaller chunks.

MaxSem updated the task description. (Show Details)Sep 22 2018, 2:26 AM
MaxSem renamed this task from Show SVGs in wiki language if available to Show SVGs in page language if available.Sep 25 2018, 10:43 PM
MaxSem updated the task description. (Show Details)
cscott added a subscriber: cscott.Sep 26 2018, 3:46 PM

Note that we're using a non-standard extension to SVG in order to translate SVGs. Some earlier discussion here: T5593: [Epic] SVG client side rendering, T60920: lang support for SVG images using SystemLanguageAttribute ill-defined and not properly supported in browsers.

That said, that earlier discussion was about difficulties with client-side rendering; this task appears to be a server-side solution, and that should be fine. We can make our own rules when we control the renderers. ;)

Parsoid will probably have to learn how to pass the lang flag to the thumbnailer at some point, but you can do this in core first and then we'll follow the same API on our side eventually. Maybe open a new phab task for "parsoid should pass the lang flag this way" once you've figured out exactly what "this way" is.

ema moved this task from Triage to Watching on the Traffic board.Oct 1 2018, 9:15 AM
Niharika triaged this task as Normal priority.Oct 1 2018, 9:56 PM
fgiunchedi added a subscriber: fgiunchedi.

Adding Thumbor too since I'm sure it'll be affected as well. re: swift space concerns I don't think it'll be a problem unless the rasterized SVGs take up a lot of space, which I don't think it is the case. Thanks for the heads up!

Niharika set the point value for this task to 13.Oct 2 2018, 11:51 PM
Niharika moved this task from To be estimated/discussed to Estimated on the Community-Tech board.
MaxSem claimed this task.Oct 3 2018, 7:55 PM
MaxSem moved this task from Ready to In Development on the Community-Tech-Sprint board.

Change 464437 had a related patch set uploaded (by MaxSem; owner: MaxSem):
[mediawiki/core@master] Remove ugly function existence check

https://gerrit.wikimedia.org/r/464437

Change 464936 had a related patch set uploaded (by MaxSem; owner: MaxSem):
[mediawiki/core@master] WIP: display SVGs in page language

https://gerrit.wikimedia.org/r/464936

Change 464437 merged by jenkins-bot:
[mediawiki/core@master] Remove ugly function existence check

https://gerrit.wikimedia.org/r/464437

Samwilson added a subscriber: Samwilson.

I think this just needs rebasing.

Will you merge it then? :P

I was waiting for others to weigh in. They haven't. I've +2'd it. :)

Change 464936 merged by jenkins-bot:
[mediawiki/core@master] Display SVGs in target language

https://gerrit.wikimedia.org/r/464936

Is this out on beta cluster or does it have a config switch we need to change?

It's behind a feature flag. I'll flip it on beta today, we'll need a ticket for prod rollout. We'll probably need to wait until after the Great Undertaking because the rollout will require some undistracted performance monitoring.

Change 470921 had a related patch set uploaded (by MaxSem; owner: MaxSem):
[operations/mediawiki-config@master] beta: $wgMediaInTargetLanguage = true

https://gerrit.wikimedia.org/r/470921

Change 470921 merged by jenkins-bot:
[operations/mediawiki-config@master] beta: $wgMediaInTargetLanguage = true

https://gerrit.wikimedia.org/r/470921

Seems to work pretty well so far. See this versus this versus this.

@Mooeypoo @MaxSem How are we planning to measure the increase in production load this feature causes, if any?

Glrx added a subscriber: Glrx.Nov 1 2018, 7:54 PM
Dvorapa added a subscriber: Dvorapa.Nov 1 2018, 8:30 PM

This feature still needs to be activated on production.

aezell added a subscriber: aezell.Nov 5 2018, 7:43 PM

Can we release this wiki-by-wiki in case there are performance issues?

Can we release this wiki-by-wiki in case there are performance issues?

Sure. We can do this on all non-Wikipedia wikis first and then everywhere after a day. I would keep that work as part of this ticket though. I don't expect an immediate increase in performance because most SVGs currently on wikis have a lang parameter attached. But I don't have a full understanding of how this patch works.

I had a related question above for the engineers -

@Mooeypoo @MaxSem How are we planning to measure the increase in production load this feature causes, if any?

MaxSem added a comment.Nov 5 2018, 9:52 PM

Let's create a separate ticket for production rollout. There we can discuss schedule and performance metrics.

Niharika closed this task as Resolved.Nov 7 2018, 12:54 AM
Niharika moved this task from Needs Review/Feedback to Q2 2018-19 on the Community-Tech-Sprint board.

Let's create a separate ticket for production rollout. There we can discuss schedule and performance metrics.

T208899: Rollout SVGs in page language

This looks good. Closing it.

Change 495409 had a related patch set uploaded (by MaxSem; owner: MaxSem):
[mediawiki/core@master] Remove $wgMediaInTargetLanguage

https://gerrit.wikimedia.org/r/495409

Change 495409 merged by jenkins-bot:
[mediawiki/core@master] Remove $wgMediaInTargetLanguage

https://gerrit.wikimedia.org/r/495409