Deploy wikimedia/portals with scap3
Open, LowPublic1 Story Points

Description

In order to be able to show the portal pages from Git as described in T110070, we need to deploy the contents of the repository wikimedia/portals to some space either in /srv/mediawiki or any other static directory. This deployment should be happening for any server that has roles mediawiki::web::beta_sites or mediawiki::web::sites.

Smalyshev created this task.Oct 5 2015, 6:31 PM
Smalyshev updated the task description. (Show Details)
Smalyshev raised the priority of this task from to High.
Smalyshev added a project: Discovery.
Smalyshev added a subscriber: Smalyshev.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 5 2015, 6:31 PM
mmodell added a subscriber: mmodell.

I'd really like to see this use Scap for deployment.

mmodell claimed this task.Oct 5 2015, 6:37 PM
mmodell added a project: Blocked-on-RelEng.
Krenair added a subscriber: Krenair.Oct 5 2015, 6:38 PM
mmodell edited a custom field.Oct 12 2015, 4:14 PM

For our planning purposes, what kind of timeline should we plan for to be able to deploy the portal via scap3?

bd808 added a subscriber: bd808.Oct 15 2015, 6:59 PM

I'd really like to see this use Scap for deployment.

If they want content on all Apache servers this seems like a job for scap/sync-dir today. All of the docroots are managed by scap in /srv/mediawiki/docroot

When we initially discussed it in #wikimedia-releng on IRC, there were shitloads of several conflicting opinions and disagreements, leading me to end up with an extreme lack of understanding, that is to say that I had/have essentially no clue what is actually desired and/or required.

If the portals are to live in /srv/mediawiki (which is what I originally suggested) and get deployed by wikidev, then indeed, it could be deployed with regular scap and there really isn't much to do here.

Again, during IRC discussion, there seemed to be a great deal of disagreement about whether it would be ok to require the help of a SWAT deployer to do the updates. I got the impression that the portals needed separate permissions, specifically, people without wikidev group membership should be able to deploy the portals, and do so independently from the mediawiki deployments.

Separate deployment groups, with separate shared ssh keys, is a feature that will be supported with Scap, however, it's unfortunately not ready to go quite yet. We could probably have that ready in the next week or two, optimistically.

Deskana added a subscriber: Deskana.EditedOct 16 2015, 3:02 PM

When we initially discussed it in #wikimedia-releng on IRC, there were shitloads of several conflicting opinions and disagreements, leading me to end up with an extreme lack of understanding, that is to say that I had/have essentially no clue what is actually desired and/or required.

Then perhaps as product owner I can clear up this confusion. :-)

If the portals are to live in /srv/mediawiki (which is what I originally suggested) and get deployed by wikidev, then indeed, it could be deployed with regular scap and there really isn't much to do here.

Again, during IRC discussion, there seemed to be a great deal of disagreement about whether it would be ok to require the help of a SWAT deployer to do the updates. I got the impression that the portals needed separate permissions, specifically, people without wikidev group membership should be able to deploy the portals, and do so independently from the mediawiki deployments.

That is the ideal. But it is not a requirement for the MVP.

Separate deployment groups, with separate shared ssh keys, is a feature that will be supported with Scap, however, it's unfortunately not ready to go quite yet. We could probably have that ready in the next week or two, optimistically.

I would prefer to go with regular scap then, and migrate over to scap3 (with all its nice features) later. This has taken much longer than I'd hoped, so if there's anything we can do to get it working sooner, I'd definitely want to go with that.

As far as I know this is ready to go on our end, I just need the go-ahead and I can start deploying this repo along with the train.

added @MaxSem as a subscriber as he is now leading up the backend portion of the portals work.

MaxSem lowered the priority of this task from High to Low.Nov 3 2015, 12:53 AM

Lowering priority: with https://gerrit.wikimedia.org/r/#/c/248526/ , we're fine for now. Having this repository deployed via scap3 (and potentially automatically) would be nice, but it's not blocking us right now.

ksmith renamed this task from Create deployment for wikimedia/portals to Create scripts for automatic deployment for wikimedia/portals.Nov 3 2015, 12:55 AM
mxn added a subscriber: mxn.Nov 4 2015, 5:26 AM
greg edited projects, added scap2; removed Deployment-Systems.Feb 9 2016, 11:51 PM
dduvall moved this task from Experiments to Adoption on the Scap board.Feb 12 2016, 7:25 PM
mmodell moved this task from Adoption to Scap3-Adoption-Phase1 on the Scap board.Mar 4 2016, 7:01 PM
mmodell edited projects, added Scap (Scap3-Adoption-Phase1); removed Scap.
greg added a subscriber: greg.Mar 11 2016, 10:04 PM

Does deploying MW with scap3 actually block this? Or can we migrate portals before we migrate MW itself?

portals are completely independent from MW, the submodule just happens to live in the mediawiki-config repository because that's also where the docroot used by our apache's lives. We can almost certainly migrate them before

mmodell renamed this task from Create scripts for automatic deployment for wikimedia/portals to Deploy wikimedia/portals with scap3.Mar 13 2016, 5:02 AM
mmodell removed mmodell as the assignee of this task.