Page MenuHomePhabricator
Create Task
Maniphest T204499

cloudvps: commonsarchive project trusty deprecation
Closed, ResolvedPublic
Actions

Description

Ubuntu Trusty is no longer available in Cloud VPS since Nov 2017 for new instances. However, the EOL of Trusty is approaching in 2019 and we need to move to Debian Stretch before that date.

All instances in the commonsarchive project needs to upgrade as soon as possible.

The list of affected VMs is:

  • commonsarchive-backup.commonsarchive.eqiad.wmflabs
  • commonsarchive-production.commonsarchive.eqiad.wmflabs
  • commonsarchive-varnish.commonsarchive.eqiad.wmflabs

Listed administrator are:

More info in openstack browser: https://tools.wmflabs.org/openstack-browser/project/commonsarchive

Related Objects

Event Timeline

aborrero triaged this task as Medium priority.Sep 17 2018, 12:11 PM
aborrero created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 17 2018, 12:11 PM
zhuyifei1999 claimed this task.Sep 17 2018, 1:25 PM
bd808 edited projects, added Cloud-VPS (Ubuntu Trusty Deprecation); removed Cloud-VPS.Oct 12 2018, 10:21 PM
bd808 moved this task from Backlog to VPS Projects on the Cloud-VPS (Ubuntu Trusty Deprecation) board.Oct 12 2018, 10:32 PM
bd808 removed a parent task: T199003: Develop timeline for Cloud VPS wide deprecation of Trusty.Oct 12 2018, 10:42 PM
zhuyifei1999 added a comment.Oct 18 2018, 5:00 AM

I upgraded the main server to stretch. If there are no major issues (there may be some minor issues like T207351), and after I figure out a better way to take backups without needing a separate server, I'll delete the old trusty instances.

zhuyifei1999 added a subscriber: Nemo_bis.Oct 18 2018, 5:34 PM

@Nemo_bis I'm thinking of using WikiTeam to backup this wiki to archive.org. Do you think that would work? And could you assist with that?

Nemo_bis added a comment.Oct 18 2018, 8:31 PM
In T204499#4678193, @zhuyifei1999 wrote:

@Nemo_bis I'm thinking of using WikiTeam to backup this wiki to archive.org. Do you think that would work? And could you assist with that?

It's a very small wiki, so it should work. However, you could as well just use shell scripts:
https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_shell_access_to_server

zhuyifei1999 added a comment.Oct 19 2018, 2:00 AM

Ok I think I got it mostly working... until the upload stage...

First run:

(venv)tools.commonsarchive@tools-bastion-05:/tmp/tmp.F7E4yGMIF6/wikiteam$ python uploader.py <(echo https://commonsarchive.wmflabs.org/w/api.php)
/mnt/nfs/labstore-secondary-tools-project/commonsarchive/wmflabs-dump/venv/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
  utils.DeprecatedIn23,
0 dumps uploaded previously
#########################################################################
# Uploading https://commonsarchive.wmflabs.org/w/api.php
#########################################################################
commonsarchivewmflabsorg_w-20181019-history.xml.7z found
commonsarchivewmflabsorg_w-20181019-history.xml.7z was not uploaded before
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w 20181019 commonsarchivewmflabsorg_w-20181019-history.xml.7z
 uploading commonsarchivewmflabsorg_w-20181019-history.xml.7z: [################################] 2/2 - 00:00:01
You can find it in https://archive.org/details/wiki-commonsarchivewmflabsorg_w
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w-20181019-history.xml.7z Error when uploading?

Second run:

(venv)tools.commonsarchive@tools-bastion-05:/tmp/tmp.F7E4yGMIF6/wikiteam$ python uploader.py <(echo https://commonsarchive.wmflabs.org/w/api.php)
/mnt/nfs/labstore-secondary-tools-project/commonsarchive/wmflabs-dump/venv/local/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.
  utils.DeprecatedIn23,
0 dumps uploaded previously
#########################################################################
# Uploading https://commonsarchive.wmflabs.org/w/api.php
#########################################################################
commonsarchivewmflabsorg_w-20181019-history.xml.7z found
commonsarchivewmflabsorg_w-20181019-history.xml.7z was not uploaded before
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w 20181019 commonsarchivewmflabsorg_w-20181019-history.xml.7z
Item already exists.
 uploading commonsarchivewmflabsorg_w-20181019-history.xml.7z: [################################] 2/2 - 00:00:00
You can find it in https://archive.org/details/wiki-commonsarchivewmflabsorg_w
https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w-20181019-history.xml.7z Error when uploading?

Looking at https://archive.org/details/wiki-commonsarchivewmflabsorg_w the item don't seem to include images. Given that they are archived by launcher.py to /tmp/tmp.F7E4yGMIF6/wikiteam/commonsarchivewmflabsorg_w-20181019-wikidump/images/ and inside /tmp/tmp.F7E4yGMIF6/wikiteam/commonsarchivewmflabsorg_w-20181019-wikidump.7z, how to I get the image backup uploaded?

Krenair subscribed.Oct 22 2018, 1:54 PM

Hey @zhuyifei1999 @Nemo_bis! Just an update to tell you that https://wikitech.wikimedia.org/wiki/News/Trusty_deprecation#Cloud_VPS_projects now says the deadline is 2018-12-18. Please get in contact if you need help. Nemo is there any chance you can help Zhuyifei with the above question?

Nemo_bis added a comment.Oct 29 2018, 7:36 PM

https://commonsarchive.wmflabs.org/w/api.php commonsarchivewmflabsorg_w-20181019-history.xml.7z Error when uploading?

This means the file (or file hash) was not found online. Can also happen if you retry too soon, before IA has computed the hash.

how to I get the image backup uploaded?

Just go edit manually? https://archive.org/edit/wiki-commonsarchivewmflabsorg_w

aborrero added a comment.Nov 20 2018, 1:07 PM

Another ping. Deadline is approaching (2018-12-18).

zhuyifei1999 closed this task as Resolved.Dec 6 2018, 3:46 AM

I deleted the old trusty instances. Will work on backups later. Really lack time rn.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL