Page MenuHomePhabricator

Archive search deployment plan
Closed, ResolvedPublic

Description

Since the code for T109561 is checked in, now we need to roll out this feature for all users. I propose the following plan:

  • Select several wikis with active admin community
  • Reindex those wikis to enable archive mapping (once datacenter switch has been completed)
  • Enable archive indexing configs for those wikis
  • Index archives for these wikis (forceSearchIndex.php --archive)
  • Post a notice on admin boards for those wikis asking for testing/feedback
  • Enable indexing for new archive deletes
  • Once feedback is collected, do the changes if needed (assuming it's ok to proceed)
  • Reindex rest of the wikis to enable the archive index again
  • Enable archive indexing configs for all wikis
  • Index archives for these wikis (forceSearchIndex.php --archive)
  • Post a notice on mailing lists (which ones?) and admin noticeboards about this capability
  • Wait for feedback and adjustments, if no negative pushback happening, enable ES search by default without fuzzy option (*)

Note that until the step marked with (*) the default search still does not use ElasticSearch and fuzzy=1 option is needed to enable it.

Event Timeline

Given the fact it's a long requested feature which doesn't break any workflows, it might not be unreasonable to not do a gradual rollout and just turn it on everywhere at once.

@CKoerner_WMF Do you have any thoughts on the deployment plan for this... and on my rather brave idea above? ;-)

Given that for now it's under a feature flag (fuzzy=1) it's definitely possible and won't mess anything up, since index is not going to change much, and UI can be tweaked on the same index.

Functionally, the rollout will still be gradual since reindexing wikis is slow, but we could skip feedback steps on the way if you think it's ok. I am open to opinions here.

@Deskana If I understand things correctly this will be making a lot of admins very happy. I think a gradual rollout and an announcement that the feature is available would be perfect.

A post to the Admin noticeboards (or equivalent) on the projects and perhaps a mailing list post (which mailing list I'm not sure).

A mention in tech news as we get closer would also be advisable.

I think we could use the same wikis as we did for testing sister projects: catalan, italian, russian, persian, german, french, polish and arabic; but also add in enwiki. :)

Deskana triaged this task as Medium priority.
Deskana moved this task from needs triage to Current work on the Discovery-Search board.

The creation of this plan is in progress.

Change 357236 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/mediawiki-config@master] Enable archive indexing on delete for select wikis

https://gerrit.wikimedia.org/r/357236

I've shared this news and asked for help testing on the list of wikis in T162302#3245957:

Wikipedias:

Arabic : https://ar.wikipedia.org/wiki/ويكيبيديا:إخطار_الإداريين/أخرى/الحالية#.D8.A7.D8.AE.D8.AA.D8.A8.D8.A7.D8.B1_.D8.A7.D9.84.D8.A8.D8.AD.D8.AB_.D9.81.D9.8A_.D9.85.D8.AD.D9.81.D9.88.D8.B8.D8.A7.D8.AA_.D8.A7.D9.84.D8.B5.D9.81.D8.AD.D8.A7.D8.AA_.D8.A7.D9.84.D9.85.D9.84.D8.BA.D8.A7.D8.A9

Catalan: https://ca.wikipedia.org/wiki/Viquipèdia:Sala_dels_administradors#Testing_search_in_deleted_pages_archive

English : https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard#Testing_search_in_deleted_pages_archive

Farsi: https://fa.wikipedia.org/wiki/ویکی%E2%80%8Cپدیا:تابلوی_اعلانات_مدیران#Testing_search_in_deleted_pages_archive

German: https://de.wikipedia.org/wiki/Wikipedia:Administratoren/Notizen#Testing_search_in_deleted_pages_archive

Italian: https://it.wikipedia.org/wiki/Wikipedia:Richieste_agli_amministratori/giugno_2017#Testare_la_ricerca_negli_archivi_delle_pagine_eliminate

Polish: https://pl.wikipedia.org/wiki/Wikipedia:Prośby_do_administratorów#Testujemy_wyszukiwanie_w_archiwum_usuni.C4.99tych_stron

Russian: https://ru.wikipedia.org/wiki/Википедия:Форум_администраторов#.D0.A2.D0.B5.D1.81.D1.82.D0.B8.D1.80.D0.BE.D0.B2.D0.B0.D0.BD.D0.B8.D0.B5_.D0.BF.D0.BE.D0.B8.D1.81.D0.BA.D0.B0_.D0.B2_.D0.B0.D1.80.D1.85.D0.B8.D0.B2.D0.B5_.D1.83.D0.B4.D0.B0.D0.BB.D1.91.D0.BD.D0.BD.D1.8B.D1.85_.D1.81.D1.82.D0.B0.D1.82.D0.B5.D0.B9

MediaWiki.org: https://www.mediawiki.org/wiki/Project:Requests#Testing_search_in_deleted_pages_archive

Smalyshev updated the task description. (Show Details)

Change 363669 merged by jenkins-bot:
[operations/mediawiki-config@master] Index deletes everywhere

https://gerrit.wikimedia.org/r/363669

Mentioned in SAL (#wikimedia-operations) [2017-07-12T23:08:31Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:363669|Index deletes everywhere]] T163235 (duration: 00m 47s)

We can announce after 1.30.0-wmf.10 is deployed.

Request for translations has gone out: https://lists.wikimedia.org/pipermail/translators-l/2017-July/004109.html

We expect to announce this change on Monday, July 24, 2017.

A lot of data munging and copy/paste later and the message has been sent to 475 wikis. First those that have an Admin noticeboard, then those that don't, but have a Village Pump (technical), and then finally just those that only have a Village Pump. In 6 languages.

Change 368819 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/mediawiki-config@master] Enable archive search via Elastic everywhere except Wikidata.

https://gerrit.wikimedia.org/r/368819

Change 368819 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable archive search via Elastic everywhere except Wikidata.

https://gerrit.wikimedia.org/r/368819

Change 368823 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/core@master] Enable archive search by default on Special:Undelete.

https://gerrit.wikimedia.org/r/368823

Mentioned in SAL (#wikimedia-operations) [2017-07-31T18:14:51Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:368819|Enable archive search via Elastic everywhere except Wikidata]] T163235 (duration: 00m 42s)

Change 368823 merged by jenkins-bot:
[mediawiki/core@master] Enable archive search by default on Special:Undelete.

https://gerrit.wikimedia.org/r/368823

Smalyshev updated the task description. (Show Details)