We haven't reached to each other this week. Jan has set up a checkin for Monday 11/6.

Change 389468 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[integration/config@master] [WIP] JJB for building wikimedia portals

https://gerrit.wikimedia.org/r/389468

Change 389479 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[wikimedia/portals@master] Add NPM script to run all portal build steps

https://gerrit.wikimedia.org/r/389479

Mentioned in SAL (#wikimedia-releng) [2017-11-07T15:15:00Z] <hashar> Created VPS account "PortalsBuilder" - T179694

Mentioned in SAL (#wikimedia-releng) [2017-11-07T16:15:02Z] <hashar> Created portalsbuilder in Gerrit, generated a ssh key pair for it and stored in Jenkins credentials store - T179694

Status

Jan and I had a chat yesterday about it.

https://gerrit.wikimedia.org/r/#/c/389479/ introduces an entry point build-all-portals to generate the assets. Jenkins will call it.

Jan and Hashar paired on the Jenkins job https://gerrit.wikimedia.org/r/389468 . It will:

• clone the repos on Monday at 7am
• run npm build-all-portals
• craft a git commit and send it for review to Gerrit as user PortalsBuilder
• report by email

https://integration.wikimedia.org/ci/job/wikimedia-portals-build/4/console

So eventually we had a change ending up in Gerrit:

Assets build - 2017-11-09 12:47:38+00:00
https://gerrit.wikimedia.org/r/#/c/390234/

:D

Change 389479 merged by jenkins-bot:
[wikimedia/portals@master] Add npm script to run all portal build steps

https://gerrit.wikimedia.org/r/389479

Lets see how it behaves Monday.

I guess next we will want to look at pushing changes to a new /deploy repository and have operations/mediawiki-config use that.

Change 389468 merged by jenkins-bot:
[integration/config@master] Job for building wikimedia portals

https://gerrit.wikimedia.org/r/389468

@hashar when thinking about splitting the deployment assets into their own repo, I had some concerns regarding developer workflow with a git submodule. Specifically, it seemed confusing to me that the artifacts repo would be updated by the jenkins job, then mediawiki-config would pull to the head of that repo, but the source repo would be stuck at whatever commit the submodule was added at (unless a patch was submitted to update the submodule). Luckily, I discovered that git submodules can also track branches, https://stackoverflow.com/questions/9189575/git-submodule-tracking-latest/9189815#9189815 and that sounds like a pretty good option to me :)

Change 391189 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[integration/config@master] Changing time of wikimedia-portals-build job

https://gerrit.wikimedia.org/r/391189

Change 391189 merged by jenkins-bot:
[integration/config@master] Changing time of wikimedia-portals-build job

https://gerrit.wikimedia.org/r/391189

In T179694#3741663, @Stashbot wrote:

Mentioned in SAL (#wikimedia-releng) [2017-11-07T16:15:02Z] <hashar> Created portalsbuilder in Gerrit, generated a ssh key pair for it and stored in Jenkins credentials store - T179694

@hashar This gerrit user's email is our team internal list, so we're all getting gerrit update emails for each patch. We should do something about that. Can we use a different email/list for that?

In T179694#3761179, @greg wrote:

Can we use a different email/list for that?

Really, the account should be owned by the Portals team, not RelEng, right?

In T179694#3761318, @greg wrote:

Really, the account should be owned by the Portals team, not RelEng, right?

@Jdrewniak what would make sense from your side?

In T179694#3763562, @greg wrote:

In T179694#3761318, @greg wrote:

Really, the account should be owned by the Portals team, not RelEng, right?

@Jdrewniak what would make sense from your side?

Not sure. In reality, the portals team is just me and @debt, and we don't have an email/list for the portals. It probably shouldn't be a personal email, so I guess we can make a portals email list and use that instead?

In T179694#3769202, @Jdrewniak wrote:

Not sure. In reality, the portals team is just me and @debt, and we don't have an email/list for the portals. It probably shouldn't be a personal email, so I guess we can make a portals email list and use that instead?

+1

@Jdrewniak and @greg — I'll ask OIT to create an email list for this purpose. :)

There is now a portals@wikimedia.org however it is public so we cant really have a Wikitech account attached to it. Else one can trivially retrieve credentials by asking for a temporary password. Grrr :-(

I think what we did for other bots is to have a list that is an alias / relay to several emails. But not a proper list with archives etc. I am not quite sure how to set that up. Alternatively, the current list can be restricted / made private and that would solve the issue.

From there, we can:

• change the email for the PortalsBuilder user on wikitech
• adjust the git author/committer email in the job
• update the email the jenkins job send spam too

In T179694#3778354, @hashar wrote:

There is now a portals@wikimedia.org however it is public so we cant really have a Wikitech account attached to it. Else one can trivially retrieve credentials by asking for a temporary password. Grrr :-(

I think what we did for other bots is to have a list that is an alias / relay to several emails. But not a proper list with archives etc. I am not quite sure how to set that up. Alternatively, the current list can be restricted / made private and that would solve the issue.

From there, we can:

• change the email for the PortalsBuilder user on wikitech
• adjust the git author/committer email in the job
• update the email the jenkins job send spam too

Ahh....I didn't realize that we didn't want it public. I'll see what I can do to change the portals@wikimedia.org list.

Nice! I will change the registered email / jenkins job etc :)

Hi @hashar - did the email get setup? I don't think I received one yesterday after the build. Thanks for checking! :)

In T179694#3792995, @debt wrote:

Hi @hashar - did the email get setup? I don't think I received one yesterday after the build. Thanks for checking! :)

Nevermind....! We found the email and it all seemed to work, yay!

Mentioned in SAL (#wikimedia-releng) [2017-12-21T13:51:47Z] <hashar> wikitech: change email of PortalsBuilder user from releng@lists.wikimedia.org to portals@lists.wikimedia.org | Credentials come from https://phabricator.wikimedia.org/D872 | T179694

Change 399630 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Update PortalsBuilder email

https://gerrit.wikimedia.org/r/399630

• change the email for the PortalsBuilder user on wikitech
• adjust the git author/committer email in the job
• update the email the jenkins job send spam too

I have updated the PortalsBuilder password on wikitech. Which triggers:

A confirmation email has been sent to the specified email address. Before any other email is sent to the account, you will have to follow the instructions in the email, to confirm that the account is actually yours.

So @Jdrewniak / @debt would have received a mail from Wikitech to portals@lists.wikimedia.org asking for confirmation of the email.

I have logged in to Gerrit and confirmed ( https://gerrit.wikimedia.org/r/#/settings/ ) that the email address for the account is updated. One can check by searching in Gerrit for owner:portalsbuilder which would autocomplete to owner:"PortalsBuilder <portals@lists.wikimedia.org>"

The Jenkins job wikimedia-portals-build is updated via https://gerrit.wikimedia.org/r/399630

I have took on me to update the job and trigger a build https://integration.wikimedia.org/ci/job/wikimedia-portals-build/22/console

https://gerrit.wikimedia.org/r/#/c/399631/ the commit metadata has:

Author: PortalsBuilder <portals@lists.wikimedia.org>
Committer: PortalsBuilder <portals@lists.wikimedia.org>

So I guess it is all fine now!

Change 399630 merged by jenkins-bot:
[integration/config@master] Update PortalsBuilder email

https://gerrit.wikimedia.org/r/399630

Left to do:

• validate the new email in wikitech based on the email sent to portals@lists.wikimedia.org

• allow jenkins-bot@wikimedia.org to post to portals@lists.wikimedia.org

Jenkins send the email with jenkins-bot <jenkins-bot@wikimedia.org>. However the portals list is private and messages sent by non-members are held into moderation. So I guess that email should be made allowed to post or added as a member (with message delivery disabled).

jenkins-bot@wikimedia.org is an alias to release engineering if I remember properly.

Hi @hashar -- I've done the following:

jenkins-bot@wikimedia.org has been successfully subscribed to Portals.
gerrit@wikimedia.org has been successfully subscribed to Portals.

and I can confirm that we did get the emails send from the PortalsBuilder.

Let me know if there is anything else you'd like us to do! :)

Nice! I have rebuild the job once again ( build #23). That should have caused Gerrit to emit a few notifications to the list. If so I guess we can finally mark this resolved :]

Hmmm...I don't see that an email was posted to the list, but as long as it didn't bounce back to Gerrit, that's good? Maybe? Or, the list is setup not in a way that is truly useful?

I have commented on https://gerrit.wikimedia.org/r/#/c/399687 which should have sent an email to the list. Maybe the message is still hold into moderation? :(

In T179694#3855451, @debt wrote:

Hmmm...I don't see that an email was posted to the list, but as long as it didn't bounce back to Gerrit, that's good? Maybe? Or, the list is setup not in a way that is truly useful?

I diagnosed this with @debt on a hangout. I think this is now resolved? Resolve this loooong task now? :) :)

Sweet, @hashar! I think we'll need to take a look at this next week to be sure nothing is off/odd about it, since January 1st wasn't a normal deploy day (due to it being New Years Day). :)