Page MenuHomePhabricator

Remove A/B test and launch to 100%
Closed, ResolvedPublic3 Story Points

Description

NOTE: tentative deployment date April 30, 2019

Background

Based on the results from our A/B test, we estimated the effect of sameAs to be a 1.4% increase (95% CI: 0.7-2.1) in average page views per day.

Acceptance Criteria

  • Delete PageSplitTester and PageSplitTesterTest from Wikibase (this will move to Core in T209319)
  • Delete PageRandomLookup and PageRandomLookupTest
  • Delete related configuration variables from Wikibase
  • Rollout directly to 100% on projects that were not a part of the A/B test

The effect should be to both eliminate some cruft introduced to Wikibase for "sameAs" and to rollout to 100%. If this is too dicey, submit an initial mediawiki-config patch that removes the "control" bucket and then do this task.

QA

  1. Pick a group 0 or 1 wiki such as Commons.
  2. Go to a random main namespace page on the wiki such as Charaxes jasius.
  3. Examine the page source and search for application/ld+json. You should see something like:
<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Charaxes jasius","url":"https:\/\/commons.wikimedia.org\/wiki\/Charaxes_jasius","sameAs":"http:\/\/www.wikidata.org\/entity\/Q1320946","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q1320946","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2005-10-14T21:53:21Z","dateModified":"2019-03-08T00:43:01Z","image":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/c\/cc\/Charaxes_jasius_MHNT_CUT_2013_3_12_male_Roquebrun_34_Dorsal.jpg","headline":"species of insect"}</script>
  1. If you don't find the LD+JSON blob, purge the cache and check again.
  2. Repeat the process 10 times.

QA Results

ACStatusDetails
1T209377#5183353

Related Objects

StatusAssignedTask
OpenNone
OpenNone
OpenNone
OpenNone
Resolvednray
Resolvedovasileva
Resolvedmpopov
Resolvedovasileva
Resolvedovasileva
ResolvedNone
Resolved Jdlrobson
ResolvedNone
ResolvedNone
ResolvedNone
ResolvedJdforrester-WMF
Resolved Tbayer
Resolved Tbayer
ResolvedNone
Resolvedovasileva
Invalid Tbayer
Resolvedmpopov
Resolvedmpopov

Event Timeline

Niedzielski triaged this task as Normal priority.
ovasileva raised the priority of this task from Normal to High.Apr 12 2019, 10:20 AM
ovasileva added a project: Readers-Web-Backlog.
ovasileva added a subscriber: ovasileva.

Based on the results from T209891#5008750 (1.4% increase in traffic on average), we've decided to roll out the change to all wikipedias

ovasileva updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)Apr 12 2019, 10:23 AM

NOTE: tentative deployment date April 30, 2019

@Tbayer, @cscott, @Addshore, @Gilles, @WMDE-leszek, @Smalyshev, @Tarrow, @CKoerner_WMF, @mpopov we're planning to complete the roll out of the "sameAs" page schema changes soon. Please comment if you have any concerns.

Thanks for the ping. I've added a note to the next issue of Tech News to let folks know of the change. It will go out on 2019-04-22

@Niedzielski all fine from WMDE side

Change 505050 had a related patch set uploaded (by Nray; owner: Nray):
[mediawiki/extensions/Wikibase@master] Remove sameAs A/B test logic

https://gerrit.wikimedia.org/r/505050

@nray's patch looks good to me I think we can merge it this week which would allow the train to carry the changes starting on Tuesday (April 30th) as per usual.

@WMDE-leszek, if it is possible, would someone from WMDE review and +2 if it's ok? After deployment, @nray is going to follow up with a mediawiki-config clean up patch to undo the configuration variables added.

Change 505999 had a related patch set uploaded (by Nray; owner: Nray):
[operations/mediawiki-config@master] Remove wikibase sameAs A/B test config

https://gerrit.wikimedia.org/r/505999

@nray's patch looks good to me I think we can merge it this week which would allow the train to carry the changes starting on Tuesday (April 30th) as per usual.

@WMDE-leszek, if it is possible, would someone from WMDE review and +2 if it's ok? After deployment, @nray is going to follow up with a mediawiki-config clean up patch to undo the configuration variables added.

That mediawiki-config patch is at https://gerrit.wikimedia.org/r/505999 if anyone has time to review it

@ovasileva, @mpopov, FYI in reviewing @nray's config patch, I noticed that the following wikis were at 0% and will jump to 100%:

	// Do not test on the Indonesian, Portuguese, Punjabi, Dutch, Korean, Bhojpuri, Cherokee,
	// Kazakh, Catalan, French, Yoruba, and Kalmyk Wikipedias. These Wikipedias have been identified
	// as good control candidates or are part of another ongoing test.
	'idwiki' => 0,
	'ptwiki' => 0,
	'pawiki' => 0,
	'pnbwiki' => 0,
	'nlwiki' => 0,
	'nds_nlwiki' => 0,
	'kowiki' => 0,
	'bhwiki' => 0,
	'chrwiki' => 0,
	'kkwiki' => 0,
	'cawiki' => 0,
	'frwiki' => 0,
	'yowiki' => 0,
	'xalwiki' => 0,

I believe all other wikis are at 50% and will jump to 100%.

Friendly ping to the WMDE folks: we're hoping to see this merged before Tuesday's train. If it's not, we'll have to write a special config patch in mediawiki-config to remove the control group from PageSplitTester which would not be ideal.

Change 505050 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Remove sameAs A/B test logic

https://gerrit.wikimedia.org/r/505050

Friendly ping to the WMDE folks: we're hoping to see this merged before Tuesday's train. If it's not, we'll have to write a special config patch in mediawiki-config to remove the control group from PageSplitTester which would not be ideal.

Apologies for sleeping on this one. Merged now.

Moving to QA as we'd like to QA this as soon as it hits production. We'll move back to to-do for necessary cleanup after this

Niedzielski updated the task description. (Show Details)May 1 2019, 5:44 PM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptMay 1 2019, 5:44 PM
Niedzielski reassigned this task from nray to Edtadros.May 1 2019, 5:44 PM
Niedzielski added subscribers: Edtadros, nray.

@Edtadros, I've added QA steps.

Edtadros reassigned this task from Edtadros to ovasileva.

Test Result

Status: ✅ PASS
OS: macOS Mojave
Browser: Chrome
Device: MBP
Emulated Device: iPhoneX

Test Artifact(s):

QA Steps

  1. Pick a group 0 or 1 wiki such as Commons.
  2. Go to a random main namespace page on the wiki such as Charaxes jasius.
  3. Examine the page source and search for application/ld+json.
  4. If you don't find the LD+JSON blob, purge the cache and check again.
  5. Repeat the process 10 times.

Iteration 1

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Ana Catarina Silva Pereira","url":"https:\/\/es.wikipedia.org\/wiki\/Ana_Catarina_Silva_Pereira","sameAs":"http:\/\/www.wikidata.org\/entity\/Q28973803","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q28973803","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2019-01-10T15:48:35Z","dateModified":"2019-03-22T13:43:26Z","headline":"jugadora portuguesa de f\u00fatbol sala"}</script>

Iteration 2

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Estaci\u00f3n de La Plana-Picamoixons","url":"https:\/\/es.wikipedia.org\/wiki\/Estaci%C3%B3n_de_La_Plana-Picamoixons","sameAs":"http:\/\/www.wikidata.org\/entity\/Q8779980","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q8779980","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2009-08-26T17:17:17Z","dateModified":"2019-05-12T22:02:43Z"}</script>

Iteration 3

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Terapia de choque (cortometraje)","url":"https:\/\/es.wikipedia.org\/wiki\/Terapia_de_choque_(cortometraje)","sameAs":"http:\/\/www.wikidata.org\/entity\/Q19245768","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q19245768","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2015-02-13T16:15:31Z","dateModified":"2015-08-15T20:32:14Z","headline":"Cortometraje espa\u00f1ol de 2010."}</script>

Iteration 4

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Galcasa","url":"https:\/\/es.wikipedia.org\/wiki\/Galcasa","sameAs":"http:\/\/www.wikidata.org\/entity\/Q16570508","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q16570508","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2014-04-21T05:30:32Z","dateModified":"2019-03-31T20:15:57Z"}</script>

Iteration 5

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Triforillonia","url":"https:\/\/es.wikipedia.org\/wiki\/Triforillonia","sameAs":"http:\/\/www.wikidata.org\/entity\/Q21447890","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q21447890","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2017-03-26T22:15:15Z","dateModified":"2018-02-26T08:07:41Z","image":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/4\/48\/Tribrachiidae.JPG"}</script>

Iteration 6

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Arteria cerebral media","url":"https:\/\/es.wikipedia.org\/wiki\/Arteria_cerebral_media","sameAs":"http:\/\/www.wikidata.org\/entity\/Q498179","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q498179","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2007-11-21T15:45:59Z","dateModified":"2018-11-07T13:20:58Z","image":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/c\/ca\/Circle_of_Willis_es.svg"}</script>

Iteration 7

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"He Touched Me","url":"https:\/\/es.wikipedia.org\/wiki\/He_Touched_Me","sameAs":"http:\/\/www.wikidata.org\/entity\/Q1754918","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q1754918","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2014-11-08T12:47:41Z","dateModified":"2019-04-04T09:27:35Z","headline":"\u00e1lbum de Elvis Presley"}</script>

Iteration 8

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Jristina Ioannidi","url":"https:\/\/es.wikipedia.org\/wiki\/Jristina_Ioannidi","sameAs":"http:\/\/www.wikidata.org\/entity\/Q27050157","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q27050157","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2019-05-02T03:09:15Z","dateModified":"2019-05-02T03:09:15Z","headline":"halter\u00f3fila griega"}</script>

Iteration 9

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Ingeborg Drewitz","url":"https:\/\/es.wikipedia.org\/wiki\/Ingeborg_Drewitz","sameAs":"http:\/\/www.wikidata.org\/entity\/Q71381","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q71381","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2006-06-18T17:27:05Z","dateModified":"2018-10-13T17:08:19Z","headline":"escritora alemana"}</script>

Iteration 10

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@type":"Article","name":"Artern","url":"https:\/\/es.wikipedia.org\/wiki\/Artern","sameAs":"http:\/\/www.wikidata.org\/entity\/Q163346","mainEntity":"http:\/\/www.wikidata.org\/entity\/Q163346","author":{"@type":"Organization","name":"Contributors to Wikimedia projects"},"publisher":{"@type":"Organization","name":"Wikimedia Foundation, Inc.","logo":{"@type":"ImageObject","url":"https:\/\/www.wikimedia.org\/static\/images\/wmf-hor-googpub.png"}},"datePublished":"2018-06-12T12:08:47Z","dateModified":"2018-06-12T12:08:47Z","image":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/4\/4a\/Bundesarchiv_Bild_146-2006-0013%2C_Luftbild_Artern.jpg"}</script>
Edtadros updated the task description. (Show Details)May 15 2019, 2:22 AM

Looks good, leaving for technical signoff for the remainder of the acceptance criteria

ovasileva updated the task description. (Show Details)May 20 2019, 9:26 AM
phuedx updated the task description. (Show Details)May 21 2019, 2:06 PM
phuedx updated the task description. (Show Details)
phuedx added a subscriber: phuedx.
  • Delete PageSplitTester and PageSplitTesterTest from Wikibase (this will move to Core in T209319)
  • Delete PageRandomLookup and PageRandomLookupTest
  • Delete related configuration variables from Wikibase

All three were done in 5d62583e179eb35bd406d669474094e6da02e246.

Change 505999 merged by jenkins-bot:
[operations/mediawiki-config@master] Remove wikibase sameAs A/B test config

https://gerrit.wikimedia.org/r/505999