Page MenuHomePhabricator
Create Task
Maniphest T168490

upgrade planet instances to stretch
Closed, ResolvedPublic
Actions

Description

upgrade both planet1001 and planet2001 from jessie to stretch.

This task is for the actual reinstall of the prod instances.

A requirement for that was replacing the planet-venus software with rawdog but this has moved to subtask T180498.

was:

make needed changes in puppet role

also.. replace the whole software since planet-venus doesn't exist in stretch anymore and is largely unmaintained

https://tracker.debian.org/pkg/planet-venus

one candidate is "rawdog" which would be preferred since it's a package in stable

http://offog.org/code/rawdog/

the other is "planet pluto" which is not a package and uses Ruby (gem install...)

https://github.com/feedreader

rawdog uses Python and Feedparser just like planet-venus did.

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Dzahn created this task.Jun 21 2017, 1:22 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 21 2017, 1:22 AM
gerritbot subscribed.Jun 21 2017, 1:23 AM

Change 360596 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: planet-venus not in stretch, try using "rawdog" instead

https://gerrit.wikimedia.org/r/360596

gerritbot added a project: Patch-For-Review.Jun 21 2017, 1:23 AM
Dzahn added a comment.Jun 21 2017, 1:23 AM

https://gerrit.wikimedia.org/r/360584

Dzahn updated the task description. (Show Details)Jun 21 2017, 1:24 AM
Dzahn triaged this task as Medium priority.Jun 21 2017, 1:26 AM
gerritbot added a comment.Jun 21 2017, 1:27 AM

Change 360596 merged by Dzahn:
[operations/puppet@production] planet: planet-venus not in stretch, try using "rawdog" instead

https://gerrit.wikimedia.org/r/360596

gerritbot added a comment.Jun 21 2017, 1:47 AM

Change 360597 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: do not install libapache2-mod-php5

https://gerrit.wikimedia.org/r/360597

gerritbot added a comment.Jun 21 2017, 2:42 AM

Change 360597 merged by Dzahn:
[operations/puppet@production] planet: do not install libapache2-mod-php5

https://gerrit.wikimedia.org/r/360597

gerritbot added a comment.Jun 21 2017, 2:55 AM

Change 360598 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: disable codfw backend temporarily

https://gerrit.wikimedia.org/r/360598

gerritbot added a comment.Jun 21 2017, 2:59 AM

Change 360598 merged by Dzahn:
[operations/puppet@production] planet: disable codfw backend temporarily

https://gerrit.wikimedia.org/r/360598

gerritbot added a comment.Jun 21 2017, 3:26 AM

Change 360599 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: install python-tidylib for rawdog

https://gerrit.wikimedia.org/r/360599

gerritbot added a comment.Jun 21 2017, 3:30 AM

Change 360599 merged by Dzahn:
[operations/puppet@production] planet: install python-tidylib for rawdog

https://gerrit.wikimedia.org/r/360599

gerritbot added a comment.Jun 21 2017, 4:06 AM

Change 360600 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: add support/compat for stretch and rawdog

https://gerrit.wikimedia.org/r/360600

gerritbot added a comment.Jun 21 2017, 4:23 AM

Change 360600 merged by Dzahn:
[operations/puppet@production] planet: add support/compat for stretch and rawdog

https://gerrit.wikimedia.org/r/360600

gerritbot added a comment.Jun 21 2017, 4:34 AM

Change 360601 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: fix logo path and duplicate declare of config dir

https://gerrit.wikimedia.org/r/360601

Dzahn added a project: Wikimedia-Planet.Jun 21 2017, 4:35 AM
Dzahn added subscribers: Dereckson, Paladox, Framawiki.
Dzahn added a subscriber: Nemo_bis.
gerritbot added a comment.Jun 21 2017, 4:46 AM

Change 360601 merged by Dzahn:
[operations/puppet@production] planet: fix logo path and duplicate declare of config dir

https://gerrit.wikimedia.org/r/360601

Dzahn added a parent task: T168494: tracking task: jessie -> stretch.Jun 21 2017, 4:50 AM
gerritbot added a comment.Jun 21 2017, 5:47 AM

Change 360603 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: fix invalid relationship in cronjob, dirs on stretch

https://gerrit.wikimedia.org/r/360603

gerritbot added a comment.Jun 21 2017, 6:06 AM

Change 360603 merged by Dzahn:
[operations/puppet@production] planet: fix invalid relationship in cronjob, dirs on stretch

https://gerrit.wikimedia.org/r/360603

gerritbot added a comment.Jun 22 2017, 8:05 PM

Change 360910 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: make config templates and crons flexible for rawdog

https://gerrit.wikimedia.org/r/360910

gerritbot added a comment.Jun 22 2017, 10:12 PM

Change 360910 merged by Dzahn:
[operations/puppet@production] planet: make config templates and crons for rawdog

https://gerrit.wikimedia.org/r/360910

gerritbot added a comment.Jun 22 2017, 10:26 PM

Change 360990 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: fix "splitstate" option, config file name, icon

https://gerrit.wikimedia.org/r/360990

gerritbot added a comment.Jun 22 2017, 10:35 PM

Change 360990 merged by Dzahn:
[operations/puppet@production] planet: fix "splitstate" option, config file name, icon

https://gerrit.wikimedia.org/r/360990

gerritbot added a comment.Jun 23 2017, 9:19 PM

Change 361180 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: add HTML page/item/feed templates for rawdog

https://gerrit.wikimedia.org/r/361180

gerritbot added a comment.Jun 23 2017, 9:50 PM

Change 361180 merged by Dzahn:
[operations/puppet@production] planet: add HTML page/item/feed templates for rawdog

https://gerrit.wikimedia.org/r/361180

Paladox updated the task description. (Show Details)Nov 14 2017, 4:44 PM
Dzahn mentioned this in T180498: planet.wikimedia.org: replace planet-venus software with rawdog.Nov 14 2017, 4:59 PM
Dzahn created subtask T180498: planet.wikimedia.org: replace planet-venus software with rawdog.
Paladox updated the task description. (Show Details)Nov 14 2017, 5:03 PM
Dzahn updated the task description. (Show Details)Nov 14 2017, 5:28 PM
Dzahn updated the task description. (Show Details)
Dzahn mentioned this in T180978: Switch on http/2 in apache for gerrit.Nov 22 2017, 8:21 PM
gerritbot added a comment.Nov 22 2017, 11:14 PM

Change 392983 had a related patch set uploaded (by Paladox; owner: Paladox):
[operations/puppet@production] planet: Add support for http/2 on stretch

https://gerrit.wikimedia.org/r/392983

Paladox created subtask T181202: Enable http/2 for planet apache.Nov 22 2017, 11:14 PM
Dzahn closed subtask T181202: Enable http/2 for planet apache as Declined.Nov 23 2017, 5:53 PM
Dzahn added a comment.May 25 2018, 7:52 PM

15:33 <mutante> 15:31 <@James_F> mutante: Will the feed URL stay the same? If so, most won't notice the switchover.
15:33 <mutante> 15:33 < mutante> James_F: that's a good one. i will check that and if needed add some RewriteRule or so

Jdforrester-WMF subscribed.May 25 2018, 7:52 PM
Dzahn added a comment.May 25 2018, 8:00 PM

@Jdforrester-WMF I should have used T180498. That is the ticket for replacing the software. This here was just for the actual server upgrade .. then we ran into the blocker that planet-venus got dropped in stretch and then we found rawdog as the replacement. So i made it a subtask.

gerritbot added a comment.May 25 2018, 8:01 PM

Change 435218 had a related patch set uploaded (by Paladox; owner: Paladox):
[operations/puppet@production] Planet: Replace rss20.xml with atom.xml (backwards compat filename)

https://gerrit.wikimedia.org/r/435218

gerritbot added a comment.May 28 2018, 11:19 PM

Change 435218 abandoned by Paladox:
Planet: Replace rss20.xml with atom.xml (backwards compat filename)

Reason:
This is not atom format so we need to use rss20.

Which supports atom readers i think?

https://gerrit.wikimedia.org/r/435218

gerritbot added a comment.May 30 2018, 11:34 PM

Change 436426 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet/misc-varnish: add codfw backend to director

https://gerrit.wikimedia.org/r/436426

gerritbot added a comment.May 30 2018, 11:34 PM

Change 436427 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet/misc-varnish: comment out eqiad backend for reinstall

https://gerrit.wikimedia.org/r/436427

gerritbot added a comment.May 30 2018, 11:41 PM

Change 436426 merged by Dzahn:
[operations/puppet@production] planet/misc-varnish: add codfw backend to director

https://gerrit.wikimedia.org/r/436426

gerritbot added a comment.May 30 2018, 11:47 PM

Change 436427 merged by Dzahn:
[operations/puppet@production] planet/misc-varnish: comment out eqiad backend for reinstall

https://gerrit.wikimedia.org/r/436427

Paladox closed subtask T180498: planet.wikimedia.org: replace planet-venus software with rawdog as Resolved.May 31 2018, 1:40 AM
Dzahn added a comment.May 31 2018, 3:35 AM

Now planet2001 is on stretch, uses puppetized rawdog and is serving live traffic.

planet1001 is currently commented out in cache::misc and still has planet-venus on jessie.

For a short transitional period it will run like this to be able to revert easily if needed for any unexpected reason.

Next planet1001 will be reinstalled with stretch effectively removing planet-venus from production.

Finally the service can be active-active again in varnish which it hasn't been for a while since planet2001 has been upgraded to stretch for rawdog support.

Dzahn added a comment.May 31 2018, 3:38 AM

proper revert to go back to planet-venus would be to first revert

https://gerrit.wikimedia.org/r/#/c/436427/2/hieradata/role/common/cache/misc.yaml

and then, in a second change, comment out codfw and remove the comment from eqiad. for $reasons we should never switch a cache::misc backend in a single step from eqiad-only to codfw-only and vice versa.

MoritzMuehlenhoff subscribed.Jun 1 2018, 9:13 AM

There's quite a bit of crom spam by planet2001:

<planet@planet2001> http_proxy="http://url-downloader.codfw.wikimedia.org:8080" https_proxy=$http_proxy /usr/bin/rawdog -d /etc/rawdog/en/ -v -u -w > /var/log/planet/en-planet.log 2>&1

/bin/sh: 1: cannot create /var/log/planet/en-planet.log: Permission denied
Dzahn added a comment.Jun 1 2018, 12:52 PM

Fixed ownership of the "en" logfile. It was owned by root:root, all others by planet:planet. It was from manually running update command.

gerritbot added a comment.Jun 7 2018, 12:16 PM

Change 437975 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] install_server: switch planet1001 from jessie to stretch

https://gerrit.wikimedia.org/r/437975

gerritbot added a comment.Jun 7 2018, 12:18 PM

Change 437975 merged by Dzahn:
[operations/puppet@production] install_server: switch planet1001 from jessie to stretch

https://gerrit.wikimedia.org/r/437975

Stashbot subscribed.Jun 7 2018, 12:26 PM

Mentioned in SAL (#wikimedia-operations) [2018-06-07T12:26:55Z] <mutante> planet1001 - schedule downtime, boot to PXE, reinstall with stretch (ganeti) (T168490)

Stashbot added a comment.Jun 7 2018, 12:46 PM

Mentioned in SAL (#wikimedia-operations) [2018-06-07T12:46:08Z] <mutante> planet1001/puppetmaster: revoke old cert, sign new cert request, initial puppet run, reinstalled, will turn service active-active again once done (T168490)

gerritbot added a comment.Jun 7 2018, 12:49 PM

Change 437987 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] planet: make active-active again, reactivate eqiad backend

https://gerrit.wikimedia.org/r/437987

gerritbot added a comment.Jun 7 2018, 7:36 PM

Change 437987 merged by Dzahn:
[operations/puppet@production] planet: make active-active again, reactivate eqiad backend

https://gerrit.wikimedia.org/r/437987

Dzahn closed this task as Resolved.Jun 7 2018, 7:55 PM

done. planet is served again from both data centers at once. planet1001 and planet2001 are both on stretch and use rawdog. planet-venus has been decom'ed for being unmaintained upstream

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits