Archive search deployment plan
Closed, ResolvedPublic

Description

Since the code for T109561 is checked in, now we need to roll out this feature for all users. I propose the following plan:

  • Select several wikis with active admin community
  • Reindex those wikis to enable archive mapping (once datacenter switch has been completed)
  • Enable archive indexing configs for those wikis
  • Index archives for these wikis (forceSearchIndex.php --archive)
  • Post a notice on admin boards for those wikis asking for testing/feedback
  • Enable indexing for new archive deletes
  • Once feedback is collected, do the changes if needed (assuming it's ok to proceed)
  • Reindex rest of the wikis to enable the archive index again
  • Enable archive indexing configs for all wikis
  • Index archives for these wikis (forceSearchIndex.php --archive)
  • Post a notice on mailing lists (which ones?) and admin noticeboards about this capability
  • Wait for feedback and adjustments, if no negative pushback happening, enable ES search by default without fuzzy option (*)

Note that until the step marked with (*) the default search still does not use ElasticSearch and fuzzy=1 option is needed to enable it.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 18 2017, 6:48 PM

Given the fact it's a long requested feature which doesn't break any workflows, it might not be unreasonable to not do a gradual rollout and just turn it on everywhere at once.

@CKoerner_WMF Do you have any thoughts on the deployment plan for this... and on my rather brave idea above? ;-)

Given that for now it's under a feature flag (fuzzy=1) it's definitely possible and won't mess anything up, since index is not going to change much, and UI can be tweaked on the same index.

Functionally, the rollout will still be gradual since reindexing wikis is slow, but we could skip feedback steps on the way if you think it's ok. I am open to opinions here.

@Deskana If I understand things correctly this will be making a lot of admins very happy. I think a gradual rollout and an announcement that the feature is available would be perfect.

A post to the Admin noticeboards (or equivalent) on the projects and perhaps a mailing list post (which mailing list I'm not sure).

A mention in tech news as we get closer would also be advisable.

debt added a comment.Apr 18 2017, 8:38 PM

I think we could use the same wikis as we did for testing sister projects: catalan, italian, russian, persian, german, french, polish and arabic; but also add in enwiki. :)

Johan awarded a token.Apr 20 2017, 1:13 PM
Johan moved this task from To Triage to Not ready to announce on the User-notice board.
Deskana triaged this task as Normal priority.Apr 20 2017, 5:07 PM
Deskana moved this task from Needs triage to Current work on the Discovery-Search board.
Deskana assigned this task to Smalyshev.

The creation of this plan is in progress.

Smalyshev updated the task description. (Show Details)May 30 2017, 5:37 PM
Smalyshev updated the task description. (Show Details)Jun 5 2017, 6:27 PM

Change 357236 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/mediawiki-config@master] Enable archive indexing on delete for select wikis

https://gerrit.wikimedia.org/r/357236

CKoerner_WMF added a comment.EditedJun 6 2017, 7:25 PM

I've shared this news and asked for help testing on the list of wikis in T162302#3245957:

Wikipedias:

Arabic : https://ar.wikipedia.org/wiki/ويكيبيديا:إخطار_الإداريين/أخرى/الحالية#.D8.A7.D8.AE.D8.AA.D8.A8.D8.A7.D8.B1_.D8.A7.D9.84.D8.A8.D8.AD.D8.AB_.D9.81.D9.8A_.D9.85.D8.AD.D9.81.D9.88.D8.B8.D8.A7.D8.AA_.D8.A7.D9.84.D8.B5.D9.81.D8.AD.D8.A7.D8.AA_.D8.A7.D9.84.D9.85.D9.84.D8.BA.D8.A7.D8.A9

Catalan: https://ca.wikipedia.org/wiki/Viquipèdia:Sala_dels_administradors#Testing_search_in_deleted_pages_archive

English : https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard#Testing_search_in_deleted_pages_archive

Farsi: https://fa.wikipedia.org/wiki/ویکی%E2%80%8Cپدیا:تابلوی_اعلانات_مدیران#Testing_search_in_deleted_pages_archive

German: https://de.wikipedia.org/wiki/Wikipedia:Administratoren/Notizen#Testing_search_in_deleted_pages_archive

Italian: https://it.wikipedia.org/wiki/Wikipedia:Richieste_agli_amministratori/giugno_2017#Testare_la_ricerca_negli_archivi_delle_pagine_eliminate

Polish: https://pl.wikipedia.org/wiki/Wikipedia:Prośby_do_administratorów#Testujemy_wyszukiwanie_w_archiwum_usuni.C4.99tych_stron

Russian: https://ru.wikipedia.org/wiki/Википедия:Форум_администраторов#.D0.A2.D0.B5.D1.81.D1.82.D0.B8.D1.80.D0.BE.D0.B2.D0.B0.D0.BD.D0.B8.D0.B5_.D0.BF.D0.BE.D0.B8.D1.81.D0.BA.D0.B0_.D0.B2_.D0.B0.D1.80.D1.85.D0.B8.D0.B2.D0.B5_.D1.83.D0.B4.D0.B0.D0.BB.D1.91.D0.BD.D0.BD.D1.8B.D1.85_.D1.81.D1.82.D0.B0.D1.82.D0.B5.D0.B9

MediaWiki.org: https://www.mediawiki.org/wiki/Project:Requests#Testing_search_in_deleted_pages_archive

Smalyshev updated the task description. (Show Details)Jun 8 2017, 5:28 PM
Smalyshev updated the task description. (Show Details)Jul 11 2017, 6:18 PM
Smalyshev updated the task description. (Show Details)

Change 363669 merged by jenkins-bot:
[operations/mediawiki-config@master] Index deletes everywhere

https://gerrit.wikimedia.org/r/363669

Mentioned in SAL (#wikimedia-operations) [2017-07-12T23:08:31Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:363669|Index deletes everywhere]] T163235 (duration: 00m 47s)

Smalyshev updated the task description. (Show Details)Jul 13 2017, 12:38 AM

We can announce after 1.30.0-wmf.10 is deployed.

Smalyshev updated the task description. (Show Details)Jul 18 2017, 6:35 PM
debt added a comment.Jul 20 2017, 6:55 PM

Request for translations has gone out: https://lists.wikimedia.org/pipermail/translators-l/2017-July/004109.html

We expect to announce this change on Monday, July 24, 2017.

A lot of data munging and copy/paste later and the message has been sent to 475 wikis. First those that have an Admin noticeboard, then those that don't, but have a Village Pump (technical), and then finally just those that only have a Village Pump. In 6 languages.

Smalyshev updated the task description. (Show Details)Jul 25 2017, 7:48 PM

Change 368819 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/mediawiki-config@master] Enable archive search via Elastic everywhere except Wikidata.

https://gerrit.wikimedia.org/r/368819

Change 368819 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable archive search via Elastic everywhere except Wikidata.

https://gerrit.wikimedia.org/r/368819

Change 368823 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/core@master] Enable archive search by default on Special:Undelete.

https://gerrit.wikimedia.org/r/368823

Mentioned in SAL (#wikimedia-operations) [2017-07-31T18:14:51Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:368819|Enable archive search via Elastic everywhere except Wikidata]] T163235 (duration: 00m 42s)

Change 368823 merged by jenkins-bot:
[mediawiki/core@master] Enable archive search by default on Special:Undelete.

https://gerrit.wikimedia.org/r/368823

Smalyshev updated the task description. (Show Details)Aug 3 2017, 6:35 PM
Smalyshev closed this task as Resolved.