Page MenuHomePhabricator

cloudvps: commonsarchive project trusty deprecation
Closed, ResolvedPublic

Description

Ubuntu Trusty is no longer available in Cloud VPS since Nov 2017 for new instances. However, the EOL of Trusty is approaching in 2019 and we need to move to Debian Stretch before that date.

All instances in the commonsarchive project needs to upgrade as soon as possible.

The list of affected VMs is:

  • commonsarchive-backup.commonsarchive.eqiad.wmflabs
  • commonsarchive-production.commonsarchive.eqiad.wmflabs
  • commonsarchive-varnish.commonsarchive.eqiad.wmflabs

Listed administrator are:

More info in openstack browser: https://tools.wmflabs.org/openstack-browser/project/commonsarchive

Event Timeline

aborrero triaged this task as Medium priority.Sep 17 2018, 12:11 PM
aborrero created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 17 2018, 12:11 PM

I upgraded the main server to stretch. If there are no major issues (there may be some minor issues like T207351), and after I figure out a better way to take backups without needing a separate server, I'll delete the old trusty instances.

@Nemo_bis I'm thinking of using WikiTeam to backup this wiki to archive.org. Do you think that would work? And could you assist with that?

@Nemo_bis I'm thinking of using WikiTeam to backup this wiki to archive.org. Do you think that would work? And could you assist with that?

It's a very small wiki, so it should work. However, you could as well just use shell scripts:
https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_shell_access_to_server

Ok I think I got it mostly working... until the upload stage...

First run:

(venv)tools.commonsarchive@tools-bastion-05:/tmp/tmp.F7E4yGMIF6/wikiteam$ python uploader.py <(echo https://commonsarchive.wmflabs.org/w/api.php)
/mnt/nfs/labstore-secondary-tools-project/commonsarchive/wmflabs-dump/venv/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
  utils.DeprecatedIn23,
0 dumps uploaded previously
#########################################################################
# Uploading https://commonsarchive.wmflabs.org/w/api.php
#########################################################################
commonsarchivewmflabsorg_w-20181019-history.xml.7z found
commonsarchivewmflabsorg_w-20181019-history.xml.7z was not uploaded before
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w 20181019 commonsarchivewmflabsorg_w-20181019-history.xml.7z
 uploading commonsarchivewmflabsorg_w-20181019-history.xml.7z: [################################] 2/2 - 00:00:01
You can find it in https://archive.org/details/wiki-commonsarchivewmflabsorg_w
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w-20181019-history.xml.7z Error when uploading?

Second run:

(venv)tools.commonsarchive@tools-bastion-05:/tmp/tmp.F7E4yGMIF6/wikiteam$ python uploader.py <(echo https://commonsarchive.wmflabs.org/w/api.php)
/mnt/nfs/labstore-secondary-tools-project/commonsarchive/wmflabs-dump/venv/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
  utils.DeprecatedIn23,
0 dumps uploaded previously
#########################################################################
# Uploading https://commonsarchive.wmflabs.org/w/api.php
#########################################################################
commonsarchivewmflabsorg_w-20181019-history.xml.7z found
commonsarchivewmflabsorg_w-20181019-history.xml.7z was not uploaded before
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w 20181019 commonsarchivewmflabsorg_w-20181019-history.xml.7z
Item already exists.
 uploading commonsarchivewmflabsorg_w-20181019-history.xml.7z: [################################] 2/2 - 00:00:00
You can find it in https://archive.org/details/wiki-commonsarchivewmflabsorg_w
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w-20181019-history.xml.7z Error when uploading?

Looking at https://archive.org/details/wiki-commonsarchivewmflabsorg_w the item don't seem to include images. Given that they are archived by launcher.py to /tmp/tmp.F7E4yGMIF6/wikiteam/commonsarchivewmflabsorg_w-20181019-wikidump/images/ and inside /tmp/tmp.F7E4yGMIF6/wikiteam/commonsarchivewmflabsorg_w-20181019-wikidump.7z, how to I get the image backup uploaded?

Hey @zhuyifei1999 @Nemo_bis! Just an update to tell you that https://wikitech.wikimedia.org/wiki/News/Trusty_deprecation#Cloud_VPS_projects now says the deadline is 2018-12-18. Please get in contact if you need help. Nemo is there any chance you can help Zhuyifei with the above question?

https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w-20181019-history.xml.7z Error when uploading?

This means the file (or file hash) was not found online. Can also happen if you retry too soon, before IA has computed the hash.

how to I get the image backup uploaded?

Just go edit manually? https://archive.org/edit/wiki-commonsarchivewmflabsorg_w

Another ping. Deadline is approaching (2018-12-18).

zhuyifei1999 closed this task as Resolved.Dec 6 2018, 3:46 AM

I deleted the old trusty instances. Will work on backups later. Really lack time rn.