Page MenuHomePhabricator

Alternative storage for old published drafts
Open, MediumPublic

Description

In addition to deleting old drafts (T183890), as a long term solution for T183485 we may want to move old drafts to a better suited storage.

This may require the following steps (separate tickets to be defined in more detail as we focus in this work):

  • Update dumps script to read contents from the alternative storage for old published drafts
  • Create script to move from primary storage to alternative storage
  • Manual runs of that script
  • Automation of that script

Event Timeline

Pginer-WMF triaged this task as Medium priority.Mar 7 2018, 9:05 AM
Pginer-WMF added a project: ContentTranslation.

We may do this for everything, not just old published drafts.

We may do this for everything, not just old published drafts.

It depends on how you're tackling the problem. It can go to Cassandra but it's much more work but it also means you can store it and change it. If you want to push things to ExternalStorage (in mw it's called BlobStore store, implemented by SqlBlobStore), it's immutable so only stuff that has been published for example (old or not old). The good thing about using ES is that it's much less work to implement and I think just moving published stuff will take care of the bulk of the issue so anything unpublished would be quite a small fraction (I think, not backed up by data). and you would also get caching and compression on the fly for free by using the BlobStore service.