Page MenuHomePhabricator

Update wikitech wiki with deployment train
Closed, ResolvedPublic

Description

We can bikeshed on the right timing/cadence, but wikitech wiki is now in the WMF cluster (instead of hosted on a third-party machine).

We should make it be a part of the normal train deploys because...

A) Less busy work for Ops
B) Responding to a deployment-caused outage would (probably) be faster by someone who does this all the time (ie: no the Ops point person)
C) Why not? :)


Version: wmf-deployment
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=62496
https://bugzilla.wikimedia.org/show_bug.cgi?id=35826

Details

Reference
bz68751

Event Timeline

bzimport raised the priority of this task from to Normal.
bzimport set Reference to bz68751.
bzimport added a subscriber: Unknown Object (MLST).
greg created this task.Jul 28 2014, 4:20 PM

Bryan Davis volunteers to help with this.

bd808 added a comment.Jul 28 2014, 4:39 PM

I suppose the first step with this would be to get the wikitech configuration merged into operations/mediawiki-config.git. Settings for testwiki may be useful to look at to see what things we tweak when pinning a vhost to a particular machine.

Reedy added a comment.Jul 28 2014, 4:53 PM

Possibly the biggest issue here is the fact we don't include the SMW extensions in the extension-list, and hence, aren't included in our l10n rebuilds using scap.

I guess we just need extension-list-wikitech (or some such), and include that where necessary...

Change 155789 had a related patch set uploaded by Andrew Bogott:
Random stab at getting wikitech config in here.

https://gerrit.wikimedia.org/r/155789

Joe added a comment.Aug 28 2014, 1:54 PM

@Andrew: since wikitech is not hosted on an appserver, how exactly did you think about the access/deployment rights/overall security issue that this would raise?

Also, not discussing this openly here as it's a security issue.

Change 156956 had a related patch set uploaded by Andrew Bogott:
Add ::mediawiki::sync to virt1000.

https://gerrit.wikimedia.org/r/156956

Change 156956 merged by Andrew Bogott:
Add ::mediawiki::sync to virt1000.

https://gerrit.wikimedia.org/r/156956

Change 156957 had a related patch set uploaded by Andrew Bogott:
Moved ::mediawiki::sync to the Openstack manager class.

https://gerrit.wikimedia.org/r/156957

Change 156957 merged by Andrew Bogott:
Moved ::mediawiki::sync to the Openstack manager class.

https://gerrit.wikimedia.org/r/156957

Change 155789 merged by Chad:
Add wikitech config.

https://gerrit.wikimedia.org/r/155789

bd808 added a comment.Sep 5 2014, 2:10 AM
  • Bug 62496 has been marked as a duplicate of this bug. ***
bd808 added a comment.Sep 5 2014, 2:20 AM

Wikitech is now running from a MediaWiki version that is synced from tin and using production configuration system. There are a few more things that need to be done before we consider this "done":

  • Add virt1000 to the dsh group for scap (implying granting deployers shell access) or configure a cron job to run sync-common on a regular basis (at least once per day).
  • Merge as many configuration hacks in wikitech.php configuration file as possible into InitialiseSettings.php
  • Get "on the train" and sync versions with Group1 wikis (non-pedias)

A really nice followup, but separate issue in my opinion, would be to stop using an older release of SMW and get back to tracking their master branch.

There are a few more things that need to be done before we consider this "done":

  • Add virt1000 to the dsh group for scap (implying granting deployers shell access) or configure a cron job to run sync-common on a regular basis (at least once per day).

T75938: Add virt1000 to the dsh group for scap

  • Merge as many configuration hacks in wikitech.php configuration file as possible into InitialiseSettings.php

T75939: Merge as many configuration hacks in wikitech.php configuration file as possible into InitialiseSettings.php

  • Get "on the train" and sync versions with Group1 wikis (non-pedias)

I say, once the above are done, consider this bug that ^.

A really nice followup, but separate issue in my opinion, would be to stop using an older release of SMW and get back to tracking their master branch.

T75940: Stop using an older release of SMW and get back to tracking their master branch

hey all, any reason we did _not_ add it to dsh groups yet? i would merge my change but that means next scap run will try to update wikitech and i'm not sure if that would really work.

Dzahn changed the status of subtask T75938: Add virt1000 to the dsh group for scap from Open to Stalled.Dec 15 2014, 12:54 PM

I got poked today as a Scrum-of-Scrums pingback to see why this is still open and marked as blocked by Operations. Here's my $0.02:

Wikitech is using hetdeploy today as a result of work that @Andrew, @Reedy and I did in August/September. What it is not doing is "riding the train" to get weekly updates as a result of scap and sync-* commands executed by deployers on tin. @Andrew was concerned about letting the deployers group have ssh access to virt1000 (host that runs wikitech) which up until now has been an root-only host for ssh access.

Instead of getting changes pushed from tin, virt1000 is updated by a root running sync-common --verbose manually. The obvious problem with this is that it is an manual operation and thus subject to not being performed often enough to say get the latest security patches and bug fixes that have been pushed to the cluster.

The proper fix in my opinion would be to allow deployers access to virt1000 and get it properly on the train or we should disentangle it from hetdeploy again and go back to ops manually updating wikitech when they remember to do it.

I have no objection to allowing deployment access, at this point. What I'd /really/ like to do, though, is have wikitech hosted on the normal cluster rather than on virt1000.

I'm reminded of this especially this week as the puppetmaster on virt1000 has been hammering apache and causing brief wikitech outages.

If anyone has a notion of what's required to do the above, and has the time to help, I will cheerfully drop everything to make this happen!

-A

Joe added a comment.Jan 15 2015, 2:54 PM

We want it to be hosted on a separate host from the puppetmaster, we don't want it to be on the mediawiki cluster at all. Wikitech offers, if any, a completely different surface of attack than most of the (much more relevant) other sites, because of the different auth mechanism and of the use of OSM.

Please don't.

I think we can afford to dedicate one server to wikitech

That's fine with me as well.

Let's move this forward again. Getting a server to host wikitech is easy. Actually hosting wikitech on a different server than virt1000... I have no idea if it's possible. @Andrew, since you are the one most familiar with wikitech from what I see (do correct me if I am wrong), mind taking a look and figure out what's needed for this to happen ?

Andrew closed this task as Resolved.Feb 10 2015, 4:54 PM
Andrew claimed this task.

Wikitech is now hosted on silver, and we've just now verified that a sync-file on tin affects the wikitech install. So... done!

Hopefully this doesn't result in deployers breaking wikitech and not noticing.

Milimetric moved this task from Scheduled to Done on the Scrum-of-Scrums board.Feb 25 2015, 6:56 PM
greg moved this task from Backlog (Tech) to Done on the Deployments board.Mar 11 2015, 8:48 PM
Restricted Application added a subscriber: Matanya. · View Herald TranscriptJun 29 2015, 11:23 PM