Page MenuHomePhabricator

Document and automate updating of static/project-logos in mediawiki-config
Open, Needs TriagePublic

Description

Status:

  • Decide where to document the logo updating process (e.g. https://meta.wikimedia.org/wiki/Requesting_wiki_configuration_changes, or maybe a README in mediawiki-config.git).
  • Decide on the requirements for updating a logo, and agree on how to meet those requirements. (e.g. file must be on Commons?, create 1.5x and 2x PNGs thumbs, optimise with optiPng/zopfli?, submit via SWAT, remember to purge)
  • Document the process in the decided location and ensure it is well advertised and links from various places.
  • Automate part of the process (e.g. a bash script that takes a Commons file name, and does download/optimisation).
  • Decide how to verify the commits in Gerrit (against malice and mistakes). E.g. a Jenkins job to run the script and confirm the committed file is the same or smaller (to confirm optimisation), and visual comparison by hand? Or also automated (SSIM?).
Original description by @ori

Change I8c9a6a567 moved the image files for project logos into the operations/mediawiki-config repository. Doing so allows us to set far future expires cache headers for these images. The impact of the change was bigger than I anticipated because it had social consequences which I did not fully understand.

  • Communities use the ability to upload a new version for a local file to set a temporary logo to mark some occasion. With I8c9a6a567 deployed, admins are still able to override the logo, but only via a Common.css rule. This needs to be communicated. @gpaumier is taking care of that. (Thanks!)
  • The use of Common.css hacks should be discouraged. Communities should follow the process described in meta:Requesting wiki configuration changes instead, and we should make sure we respond to such requests promptly. Note, configuration requests have always failed to work: see T28992, T7532, T28993.
  • The mapping of image files in the config repository to the Commons images from which they derive should be preserved. I propose to do that by having the Commons URL for each project's logo declared in a configuration file, and having a build script that (re-)populates the project-logos directory by retrieving each image from its canonical location and applying lossless image optimization.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I think we should make sure all of the files get marked as no longer in direct use as actual wiki logos. And pages like https://commons.wikimedia.org/wiki/Commons:Auto-protected_files/global/logos could do with a note at the top.

Centralised logo management is a PITA and I don't have suggestions on how to handle it. Removing myself.

Communities should follow the process described in meta:Requesting wiki configuration changes instead, and we should make sure we respond to such requests promptly.

With SWAT deploy windows, urgent changes (by tasks opened wih a priority set to high) requested Mon-Thu could be processed the same day. For example, T97281 were handled in 4 hours (deployed during the next SWAT window).

If the logo is for a short temporary event doing changes in the config files might not get through various caches fast enough to properly mark the event (Thinking of varnish for anons). Common.css has the benefit of being pretty instant.

"Common.css" updates and configuration updates are both instantaneous. Configuration updates actually roll out slightly faster even.

What matters is whether we change the file or upload a new file, and whether we use upload.wikimedia.org or "/static". If one overwrites a file, users with a pre-existing cache will continue to see the old logo for a while. If one uploads a new file, the update will be instantaneous for everyone.

For short-lived overrides, we can upload a new file to "/static" (e.g. dewikinews-20K.png for the 20,0000 article celebration) and update wgLogo configuration to point to that instead.

For indefinite changes, the default logo file should be overwritten. Server caches will update instantly thanks for Last-Modified and ETag. Browser caches may take a while to update (though users can clear their own cache) but that's inevitable. Given the change is indefinite, the rollout can be a bit slower.

@ori wrote:

The mapping of image files in the config repository to the Commons images from which they derive should be preserved. I propose to do that by having the Commons URL for each project's logo declared in a configuration file, and having a build script that (re-)populates the project-logos directory by retrieving each image from its canonical location and applying lossless image optimization.

Sounds great.

Krinkle renamed this task from Articulate workflows (technical and social) for project logo management to Document and automate updating of static/project-logos in mediawiki-config.Mar 9 2017, 10:22 PM
Krinkle edited projects, added Deployments; removed Wikimedia-Site-requests.
@ori wrote:

The mapping of image files in the config repository to the Commons images from which they derive should be preserved. I propose to do that by having the Commons URL for each project's logo declared in a configuration file, and having a build script that (re-)populates the project-logos directory by retrieving each image from its canonical location and applying lossless image optimization.

Sounds like this could be a build script in mediawiki-config that takes a static file that maps wikis to a commons url. Right now changes are made to it and where they came from and whether (and which) optimizations are applied is a hard to verify. In addition because Gerrit doesn't show images.

The build script could run from a Jenkins job to also verify the contents are up to date. (e.g. fail if dirty; which will happen if the image isn't the one from Commons or if the optim wasn't run)

hashar removed ori as the assignee of this task.Mar 17 2017, 11:18 PM

Change 346234 had a related patch set uploaded (by Krinkle):
[operations/mediawiki-config@master] [WIP] Document and automate sources of static/project-logos

https://gerrit.wikimedia.org/r/346234

Change 346234 abandoned by Krinkle:
[WIP] Document and automate sources of static/project-logos

https://gerrit.wikimedia.org/r/346234

I just had to go through this process and it was kind of a nightmare. Here are my (hopefully) actionable suggestions:

  • Require all (new) logos to have a source SVG on Commons
  • Have a script that:
    • downloads PNG thumbnails of the SVG in normal, 1.5x and 2x sizes
    • compresses the new images
      • we should provide a docker container for this
  • Generate the giant arrays in InitialiseSettings.php from a config file. E.g. instead of having to edit 3 lines manually, I should be able to say smth like:
enwiki:
  variant: enwiki20a
  • A tool (on Toolforge?) that takes a wiki's main page and injects a bit of CSS to have the logo from the Gerrit patch (think patchdemo). It should let you switch skins, toggle &safemode= and for bonus points, force 1.5x/2x screens for testing.

The workflow for changing a logo then becomes:

  • Upload SVG to commons
  • Update config file to point xxwiki to the Commons file. If it's a temporary variant, set the variant's name.
  • Run magic_logo_script xxwiki which downloads the PNGs, and compresses them. It regenerates the PHP arrays if the variant changes the filename
  • User makes first commit to update config and second to the generated PHP files.

@Legoktm LGTM. One additional point I would add, is for a CI job to run the same generate script and assert the working copy has no differences.

Change 657956 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/mediawiki-config@master] Add script to mostly automate logo management

https://gerrit.wikimedia.org/r/657956

My refactor ended up finding one bug, vepwiki was serving the 2x logo for 1.5x users. And I ended up punting on the docker container for now, I'll leave that for someone else.

@Legoktm LGTM. One additional point I would add, is for a CI job to run the same generate script and assert the working copy has no differences.

I think verifying that the images are fully compressed will need the docker container thing so that versions that a developer is using locally is consistent with what CI runs.

But there's one other issue, which is how we break this down into two commits. So if you're adding a new temporary logo, you 1) update config.yaml to the new SVG, download the PNGs, compress and commit them. and in patch 2) commit the updated logos.php. But in commit #1 config.yaml is intentionally out of sync with logos.php and would fail CI. I'm not sure how we should handle this case.

Thank you so much for looking into this and trying to simplify and streamline things a bit.

As the summary mentions docs I'd like to point out that https://wikitech.wikimedia.org/wiki/Wikimedia_site_requests#Change_the_logo_of_a_Wikimedia_wiki got created in the meantime (see T126330), so that on-wiki section might also welcome updates once some improvements have taken place. (I might try myself, but not sure how much I actually understand the process myself, hence dropping this here.)

  • A tool (on Toolforge?) that takes a wiki's main page and injects a bit of CSS to have the logo from the Gerrit patch (think patchdemo). It should let you switch skins, toggle &safemode= and for bonus points, force 1.5x/2x screens for testing.

Got most of this done in https://git.legoktm.com/legoktm/logo-test.git except the bonus points. (will deploy to Toolforge later tonight)

Thank you so much for looking into this and trying to simplify and streamline things a bit.

As the summary mentions docs I'd like to point out that https://wikitech.wikimedia.org/wiki/Wikimedia_site_requests#Change_the_logo_of_a_Wikimedia_wiki got created in the meantime (see T126330), so that on-wiki section might also welcome updates once some improvements have taken place. (I might try myself, but not sure how much I actually understand the process myself, hence dropping this here.)

Yeah, @Urbanecm pointed me at it yesterday. I'll make sure to update that page and [[m:Logos]] with the new instructions once this is merged.

  • A tool (on Toolforge?) that takes a wiki's main page and injects a bit of CSS to have the logo from the Gerrit patch (think patchdemo). It should let you switch skins, toggle &safemode= and for bonus points, force 1.5x/2x screens for testing.

Got most of this done in https://git.legoktm.com/legoktm/logo-test.git except the bonus points. (will deploy to Toolforge later tonight)

https://logo-test.toolforge.org/?wiki=en.wikipedia.org&logo=File%3AWikipedia-logo-v2-wordmark.svg

It proxies just enough of the interface so that all styles/skin loads (JS doesn't work) and injects the same MW CSS for logos to the end of the head. I ended up having it just pull the thumbs from Commons instead of Gerrit since the goal is mostly to get sizing and whatnot right, not actually test the compressed images. Plus this allows demos when discussing new logos instead of requiring a Gerrit patch to exist.

I've scheduled these changes to be merged and deployed in today's backport window (though they're all currently a no-op).

For documentation, here's what I propose to add to https://meta.wikimedia.org/wiki/Logo#Temporary_logo_variants:

  1. Upload your logo to Wikimedia Commons in SVG format. In most cases it should be a square.
  2. Use the logo-test tool to check that it displays as expected
  3. File a request in the Wikimedia-Site-requests project in Phabricator, providing links to the SVG image, logo-test and indicating when the temporary logo should be enabled and disabled
  4. Sysadmins will process your request, converting it into the proper formats in Gerrit. It is recommended you file a request at least a week in advance of wanting to change logos. Even if you don't know exactly when it should be enabled (e.g. waiting for the millionth article to be created), giving sysadmins a heads-up to prepare will help speed up your request when it is time.

@Aklapper maybe we could have a dedicated Phabricator form that asks for the fields in #3?

For documentation, here's what I propose to add to https://meta.wikimedia.org/wiki/Logo#Temporary_logo_variants:

LGTM, thanks!

@Aklapper maybe we could have a dedicated Phabricator form that asks for the fields in #3?

That's one option; the quicker and lazier option would be passing a bunch of URL parameters created with https://phabulous.toolforge.org :)

(For the records, the page I consider canonical for requesting any config changes is https://meta.wikimedia.org/wiki/Requesting_wiki_configuration_changes )

Change 657956 merged by jenkins-bot:
[operations/mediawiki-config@master] Add script to mostly automate logo management

https://gerrit.wikimedia.org/r/657956

@Urbanecm and I discussed what needs to be done for new Vector:

  • Setting in config.yaml to define the wordmark and tagline SVGs (pulling from Commons again)
  • Run svgo over said SVGs
  • Verify the dimensions of the SVG are the correct size (located in the opening <svg... tag)

@Urbanecm and I discussed what needs to be done for new Vector:

  • Setting in config.yaml to define the wordmark and tagline SVGs (pulling from Commons again)
  • Run svgo over said SVGs
  • Verify the dimensions of the SVG are the correct size (located in the opening <svg... tag)

Actually, not verify the dimensions are expected. If you open wmgSiteLogoVariants in IS.php, you'll see dimensions listed. Those should ideally match the native resolution of the SVG.

I went through the first row of https://commons.wikimedia.org/wiki/Wikipedia/2.0#SVG (minus enwiki because it has a special logo right now), setting a Commons source and updating the images and recompressing them (see https://gerrit.wikimedia.org/r/q/topic:%2522logo-update-recompress%2522)

It mostly was fine except that all of the 1.5x logos currently in git are 204×234, while the script is downloading 202×232 images. https://www.mediawiki.org/wiki/Manual:$wgLogos#Supported_variants says "The 1.5x version should be a square and should maximally have 202x202 pixels." but we haven't been doing that in practice?

Also it looks like the 1x logos are all getting smaller while it's a tossup for 1.5x/2x... maybe we do need that last pngout step?

I've updated https://www.mediawiki.org/wiki/Manual:$wgLogos#Supported_variants - this should make more sense and give you a better guideline what to do now.

It mostly was fine except that all of the 1.5x logos currently in git are 204×234, while the script is downloading 202×232 images.

204×234 is going to be slightly too big; 202x232 is correct. Basically, the logos should be nominally 135px by up to 155px, with the 1.5x and 2x versions being straight multiplications of whatever the resolutions of the 1x one are. This will keep them consistent across resolutions...

Also it looks like the 1x logos are all getting smaller while it's a tossup for 1.5x/2x... maybe we do need that last pngout step?

...because 1.5x and 2x logos are scaled to precisely 135px wide as their nominal size. Either the 1x one (which is not scaled) should also be 135px wide, or we need some way to tell RL the real nominal size, as the 135px is currently hardcoded into the RL module that renders this css.

It mostly was fine except that all of the 1.5x logos currently in git are 204×234, while the script is downloading 202×232 images.

204×234 is going to be slightly too big; 202x232 is correct. Basically, the logos should be nominally 135px by up to 155px, with the 1.5x and 2x versions being straight multiplications of whatever the resolutions of the 1x one are. This will keep them consistent across resolutions...

Thank you!

Also it looks like the 1x logos are all getting smaller while it's a tossup for 1.5x/2x... maybe we do need that last pngout step?

...because 1.5x and 2x logos are scaled to precisely 135px wide as their nominal size. Either the 1x one (which is not scaled) should also be 135px wide, or we need some way to tell RL the real nominal size, as the 135px is currently hardcoded into the RL module that renders this css.

My bad, I meant getting smaller in terms of file size.


I just discovered that zhwiki and probably other language converter wikis have different logos per variant, which is done in Common.css, but the logos are correctly hosted in mediawiki-config. So we'll also need to handle that in the management script too.

Change 661052 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/mediawiki-config@master] logos: Redo how variants work

https://gerrit.wikimedia.org/r/661052

Change 661052 merged by jenkins-bot:
[operations/mediawiki-config@master] logos: Redo how variants work

https://gerrit.wikimedia.org/r/661052