Page MenuHomePhabricator

Consider moving the Wikidata Query Builder repository from github to gerrit
Closed, ResolvedPublic

Description

As part of getting ready for deployment of the Query Builder we should consider moving the code from github to a gerrit repository.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I don't know why we decided to go with github (maybe @Jakob_WMDE remembers?) I'm definitely fine with moving to gerrit.

I don't know why we decided to go with github (maybe @Jakob_WMDE remembers?) I'm definitely fine with moving to gerrit.

IIRC we chose github over gerrit to keep code review on one platform (with wikit) and for the easier CI setup. We also said if for production we need it to be on a WMF-hosted server, we could mirror it to Diffusion as we did for Termbox before.

Things to maybe do:

  • come up with a list of options
  • talk to engineering managers about what to do
  • talk to other wmde engineers about what to do
  • maybe talk to wmf security people about what is required for security approval
  • ...

Automated mirroring to diffusion is a no-go and got banned already by wmf security. Termbox is now migrated to gerrit. We can have the deploy repo in gerrit and manually deploy it (similar to what's happening to wdqs gui) but still it's better to keep it under github.com/wikimedia org IMO (mostly because of access rules and reducing attack vectors).

Options:

  • Move it to gerrit
  • Keep it in github but have a deploy repo manually updated in gerrit.
    • If it's going to stay in github: It can be in wikimedia or wmde. It's going to be complicated. IIRC security team really wants the code to be in one org so they can apply the same policies (like mandatory 2FA for admins) and also reducing the attack vector.

For original development, it made sense since the CI was pretty easy to setup but if we want to deploy this in WMF production, I think it makes sense to move it to gerrit due to security and integration reasons. I don't know about others. I can ask security team to comment

I can ask security team to comment

Yes, an official statement from them would be appreciated.

Hello security team, it would be great if we can have a comment on this ticket on whether it's okay to have it on github or not. We are planning to deploy this to production as a static site.

Aklapper renamed this task from consider moving the repository from github to gerrit to Consider moving the Wikidata Query Builder repository from github to gerrit.Jan 29 2021, 5:36 PM

Hello security team, it would be great if we can have a comment on this ticket on whether it's okay to have it on github or not. We are planning to deploy this to production as a static site.

@Ladsgroup @Michael - we'll chat about this as a team at our clinic meeting this Monday, but I don't think we'd have too many security concerns (at least I don't) about canonically hosting Wikimedia-related repos at github, since we already do that for a handful of repos anyways (service-template-node et al). I believe there is a preference to use gerrit for canonical Wikimedia-related repos, but there's no official policy governing this AFAIK, and as long as best practices around development, CI and security issues are being followed, that should be fine. Finally - this will all change anyways once projects begin migrating to Gitlab over the next year or so.

@Ladsgroup @Michael - The Security-Team discussed this task this morning. Is the plan to pull from github for deployments upon Wikimedia production hardware? If so, I believe SRE does not allow for that.

@sbassett Thank you and the team so much for looking into this!
Having certainty about the move away from GitHub allows us to make progress on other questions as well, and we can start the migration process now, with more than enough time to spare until we actually want to go live.

Let's migrate it once we migrated the browser tests to cypress.

@Ladsgroup @Michael - The Security-Team discussed this task this morning. Is the plan to pull from github for deployments upon Wikimedia production hardware? If so, I believe SRE does not allow for that.

Not quite, here is the summary

So, this will be deployed via a build in jenkins (ideally), so that it uses the same process and the query gui.
This is just about to be created by the campsite as a push button trigger in https://phabricator.wikimedia.org/T210286
I guess it's only for a similar job to exist fetching code from github to create the build that would then be deployed?

Another alternative would be github actions to make the build and push a change to gerrit?
I don't see a big difference between the two as either way the build is triggered by a human, and the change is still 2ed by a human.
The one difference would be that npm install is running in a different place for each.

For original development, it made sense since the CI was pretty easy to setup

I don't think there is anything that different to normal CI things for this repository that would make it hard to run in jenkins (but I could be wrong)

So, this will be deployed via a build in jenkins (ideally), so that it uses the same process and the query gui.
This is just about to be created by the campsite as a push button trigger in https://phabricator.wikimedia.org/T210286
I guess it's only for a similar job to exist fetching code from github to create the build that would then be deployed?

Another alternative would be github actions to make the build and push a change to gerrit?
I don't see a big difference between the two as either way the build is triggered by a human, and the change is still 2ed by a human.
The one difference would be that npm install is running in a different place for each.

While not ideal, I think either of these approaches would be fairly low risk given the current realities of how code with build steps has to be managed and deployed to Wikimedia production, especially if said code's canonical repo exists outside of gerrit. And obviously any QA and/or security-minded review which can happen post-build (automated or otherwise) is strongly encouraged, prior to deployment.