Page MenuHomePhabricator
Create Task
Maniphest T309379

🟪️ Investigate removing the archive ElasticSearch indices
Closed, ResolvedPublic
Actions

Description

We may be able to reduce the number of indices per wiki by removing the archive indices.

According to the Search team this should be possible by setting: $wgCirrusSearchPrivateClusters = []. Apparently the: "only change should be that Special:Undelete is harder to use".

However, to me, it seems like given that CirrusSearchEnableArchive is already disabled this may actually result in no change at all.

We should investigate if there is a change, present the outcome to @WMDE-leszek and if it meets with their approval make the change to MediaWiki

Related Objects

Event Timeline

Tarrow created this task.May 27 2022, 8:51 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 27 2022, 8:51 AM
• toan claimed this task.May 27 2022, 10:18 AM
• toan moved this task from Backlog to Doing on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
• toan added a comment.May 27 2022, 12:33 PM

This seems to work https://github.com/wbstack/mediawiki/pull/256

For some reason the wgCirrusSearchPrivateClusters doesn't accept just an empty array but rather converts that to null on startup.

• toan added a comment.EditedMay 27 2022, 1:14 PM

Experimenting a bit with this it seems indeed we are just creating this index for no good reason at all.

CirrusSearchEnableArchive is disabled by default so nothing ever ends up here.

Lets just get rid of it.

• toan removed • toan as the assignee of this task.May 27 2022, 1:15 PM
• toan moved this task from Doing to Review on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
• toan subscribed.
• toan claimed this task.Jun 2 2022, 10:55 AM
• toan moved this task from Review to Blocked/Stalled on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
• toan added a comment.Jun 2 2022, 10:59 AM

Will try manually deleting the archive on staging and then re-running ForceSearchIndex and see how that goes

• toan moved this task from Blocked/Stalled to Doing on the Wikibase Cloud (Launch Migration Kanban (2022)) board.Jun 2 2022, 1:38 PM
• toan added a comment.Jun 2 2022, 3:35 PM

Ok, since the mediawiki PR isn't merged (https://github.com/wbstack/mediawiki/pull/256) I succesfully removed the archive indices locally and nothing seems to be wrong or broken afterwards.

This can be done by running curl -X DELETE 'http://localhost:9200/*_archive_first' against the cluster

afterwards

  1. UpdateSearchIndexConfig runs
  2. ForceSearchIndex runs
  3. Searching works
  4. Updating entries work
  5. one less index
• toan removed • toan as the assignee of this task.Jun 2 2022, 3:35 PM
• toan moved this task from Doing to Blocked/Stalled on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
• toan moved this task from Blocked/Stalled to Review on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
Tarrow claimed this task.Jun 3 2022, 7:25 AM
Tarrow removed Tarrow as the assignee of this task.Jun 3 2022, 10:13 AM
Tarrow moved this task from Review to To Release on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
Tarrow claimed this task.Jun 7 2022, 9:00 AM
Tarrow moved this task from To Release to Deploy To Production on the Wikibase Cloud (Launch Migration Kanban (2022)) board.Jun 7 2022, 11:29 AM
Tarrow added a comment.Jun 7 2022, 11:42 AM

Deployed to production. Deletion of Archive indices not done yet so staying in this column. Hoping to do the deletion after https://phabricator.wikimedia.org/T309776 is merged so we can "watch" the heap drop

Tarrow moved this task from Deploy To Production to Done on the Wikibase Cloud (Launch Migration Kanban (2022)) board.Jun 7 2022, 2:25 PM
Tarrow closed this task as Resolved.Jun 23 2022, 9:13 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits