Page MenuHomePhabricator

TransparencyReport repository master in Gerrit silently made private
Closed, ResolvedPublic

Description

Puppet started failing on zirconium on 2015-02-16 10:59:33 due to:

Error: /Stage[main]/Role::Transparency/Git::Clone[wikimedia/TransparencyReport]/Exec[git_pull_wikimedia/TransparencyReport]/returns: change from notrun to 0 failed: /usr/bin/git pull --quiet returned 1 instead of one of [0]

a quick debugging showed during git pulling the repo, a username/password is required. Obviously puppet does not provide one and git pull returns with an error code and puppet is complaining

Event Timeline

akosiaris assigned this task to Dzahn.
akosiaris raised the priority of this task from to Medium.
akosiaris updated the task description. (Show Details)
akosiaris added a subscriber: akosiaris.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 16 2015, 11:19 AM
akosiaris set Security to None.
faidon raised the priority of this task from Medium to High.Feb 17 2015, 11:44 AM
faidon added a subscriber: faidon.
Dzahn added a comment.Feb 17 2015, 8:44 PM

I moved the docroot of that around recently to make it consistent with other docroots on zirconium. It might be related, checking. I had copied it with rsync though...

Dzahn added a comment.Feb 17 2015, 8:56 PM

yea, no. the changed i was referring to was https://gerrit.wikimedia.org/r/#/c/190036/ but that doesn't seem related. i'm also getting the password question when just cloning that repo on my local laptop without puppet being involved.

and i can confirm that must have changed recently only.. investigating..

Dzahn added a comment.EditedFeb 17 2015, 9:00 PM

It seems like this project has been deleted on Gerrit. I don't know who or why deleted it though.

edit: it still exists in the filesystem on ytterbium:

@ytterbium:/var/lib/gerrit2/review_site/git/wikimedia/TransparencyReport.git

but i don't see it in the gerrit ui at all. not a gerrit admin though. looks like gerrit restricted it

hashar added a subscriber: hashar.

The project is no more in Gerrit: gerrit ls-projects -m ransparency yields nothing. #Wikimedia-Git-or-Gerrit watchers might know.

hashar added a subscriber: demon.Feb 17 2015, 9:25 PM

+ @Chad our Gerrit guru :-]

Dzahn added a comment.Feb 17 2015, 9:56 PM

i put a request on the wiki page to have it (re-)created but with a warning message that it existed before and linking back here

https://www.mediawiki.org/w/index.php?title=Git/New_repositories/Requests/Entries&diff=prev&oldid=1413269

gerritbot added a subscriber: gerritbot.

Change 191198 had a related patch set uploaded (by QChris):
Disable cloning of TransparencyReport until the repo is public again

https://gerrit.wikimedia.org/r/191198

Patch-For-Review

Change 191198 merged by Dzahn:
Disable cloning of TransparencyReport until the repo is public again

https://gerrit.wikimedia.org/r/191198

Krenair reassigned this task from Dzahn to QChris.Feb 17 2015, 10:35 PM
Krenair added a subscriber: Dzahn.

That says:

The repo is currently not publicly clonable by private request.
Hence, turning cloning off for now, without removing the content
until the repo is public again

Who sent this request and why?

QChris added a subscriber: Prtksxna.EditedFeb 17 2015, 10:38 PM

The repo was requested to be hidden (for the time being) in private mails.

@Prtksxna: I let you speak to the "why".

Cloning is turned off for now and should make puppet pass again.

Krenair renamed this task from Puppet failing on zirconium due to inability to git pull Transparency Report to TransparencyReport repository master in Gerrit silently made private.Feb 17 2015, 10:43 PM
Krenair removed a subscriber: Legoktm.
Krenair added a subscriber: Legoktm.

The puppet run on zirconium is fixed, because cloning is temp. disabled now.

Who sent this request and why?

I sent the request on behalf of the legal team.

We are working on the new version of the the TransparencyReport and will start making content changes soon. The legal team doesn't want the stories and numbers to be visible to the press before the launch.

That's... Not how Git is supposed to work. There is no need to hide Wikimedia's master copy of the repository just because you want to prepare a private patch and get a load of things ready/approved before releasing it.

The only other way that I know of is to submit all changes as draft patches (I hear aren't exactly private either) and merge them right before launch.

I am all ears to better alternatives.

The only other way that I know of is to submit all changes as draft patches (I hear aren't exactly private either) and merge them right before launch.

I am all ears to better alternatives.

You should be able to fork it to a local (or private) repo and then merge it in later.

So I guess we can close this bug now. The repository has been made private while some reports are being crafted for later public release.

We could. Then again I don't see us having reached a consensus about how this bug got resolved. @QChris was kind enough to provide https://gerrit.wikimedia.org/r/#/c/191198/ but it has the downside that when a new TransparencyReport version is ready to be released, various hoops will have to be jumped through by at various teams to actually release it.

  1. Legal will have to update the repo
  2. Legal will have to ping someone (@QChris again? someone else? ops ?) to turn the repo public again
  3. Someone (possibly ops) will have to revert https://gerrit.wikimedia.org/r/#/c/191198/
  4. Ops will have to merge the above.
  5. Depending on a possible requested time and date of release, ops will have to run puppet so that the update happens in a specific point into time.

There is a variation on the above on step 3

  1. git pull manually and avoid 4,5

Btw this variation only makes it worse since it is not clear if this variation should be followed or the previous one.

And then:

Rinse and repeat for the next version.

We got 2 possible "someones", 2 teams and a possible requested time and date of release. If humor was easy to convey over a phabricator ticket, now would be the time to ask "What could possibly go wrong?"

Normally, the proper solution would be a deployment process for this, but honestly I do not want to subject Legal to either scap or trebuchet or any of that.

So, the way things are right now my solution to the problem would be:

  1. Setup a way for Legal to have a workable copy of the <next_version>. A private gerrit repo with a slightly different name would probably work fine, a local repo (as in the owner's computer) would work fine as well.
  2. Clear the repo of anything Legal does not want in that repo. e.g git reset <now_released_version>, rebase etc
  3. Make the repo public again
  4. Undo https://gerrit.wikimedia.org/r/#/c/191198/

So when the time comes for <next_version> to be released.

  1. Legal updates the public repo
  2. Legal pings ops (beforehand) to run puppet at the possible specific point in time Legal wants <next_version> to be released.

If Legal is OK with about max 20 minutes of inaccuracy from the above specific point in time, 2 is not needed.

So up to 2 teams(possibly just one) and 0 someones.

Does it sound OK ?

a local repo (as in the owner's computer) would work fine as well

@MSyed and I need to collaborate on this so a local copy won't suffice.

If Legal is OK with about max 20 minutes of inaccuracy from the above specific point in time, 2 is not needed.

I didn't understand what this meant. No changes have been made to the repository since it was made private.


Just to be clear, are both @akosiaris and @Jalexander suggesting that — we fork to a separate gerrit repo (private) for the next release, and merge its changes into the current repo right before release?

I agree to this solution.

a local repo (as in the owner's computer) would work fine as well

@MSyed and I need to collaborate on this so a local copy won't suffice.

A (different and newly created) private gerrit repo then. In fact a clone of the current one. Then when you are ready and all the work is done, you can push the changes to the public repo

If Legal is OK with about max 20 minutes of inaccuracy from the above specific point in time, 2 is not needed.

I didn't understand what this meant. No changes have been made to the repository since it was made private.

I was referring to the release of the report. As in:

You want it to be online at 01:00 PM UTC on March 1 of 2015, but you are OK with it being in the public repo like 20-30 minutes before that time. We can make it so all the rest (the repo pulling on the server's side) is automatic. That way, no need to urgently (or beforehand) find Ops and coordinate with us. That way you are pretty much in full control of what happens


Just to be clear, are both @akosiaris and @Jalexander suggesting that — we fork to a separate gerrit repo (private) for the next release, and merge its changes into the current repo right before release?

I agree to this solution.

Exactly!!! Happy that we agree. I can create the private gerrit repo as a clone of the current one and change the policy of the "public" one to actually well... public and undo the change. I 'll proceed with this unless someone objects

A (different and newly created) private gerrit repo then. In fact a clone of the current one. Then when you are ready and all the work is done, you can push the changes to the public repo

Yup. Sounds good. Should I request one at https://www.mediawiki.org/wiki/Git/New_repositories/Requests?

You want it to be online at 01:00 PM UTC on March 1 of 2015, but you are OK with it being in the public repo like 20-30 minutes before that time. We can make it so all the rest (the repo pulling on the server's side) is automatic. That way, no need to urgently (or beforehand) find Ops and coordinate with us. That way you are pretty much in full control of what happens

I think I understand. Just merge to the other repo when we're ready, for a while the changes will be visible to everyone, but I guess that is fine (looks at @Mpaulson).

Exactly!!! Happy that we agree. I can create the private gerrit repo as a clone of the current one and change the policy of the "public" one to actually well... public and undo the change. I'll proceed with this unless someone objects

Sounds good. Let me know if yo need anything from me.

hashar removed a subscriber: hashar.Feb 19 2015, 11:36 AM

Hello,
sorry for the long wait, I just managed to work on this today.

I have create the gerrit repo wikimedia/TransparencyReport-private in gerrit. It is viewable only by:

  • The people in group wikimedia-TransparencyReport which are:
    1. MSyed
    2. Prtksxna
    3. Ori.livneh
  • The people in the gerrit Administrators group (selected people in wmf). I opted to leave the project viewable to the this group so they don't go around wondering how to help for a project they can't see in gerrit if a support request comes in from Legal.

I populated the new repo with the contents of the old one. Pushing to either one is possible for the wikimedia-TransparencyReport group so Legal can use it for internal collaboration (either via git remotes or local merging - whatever is everyone's preference) while retaining the capability to pushing to the non-private one at their will.

I have turned the "non-private" repo... well "non-private" again.

@Prtksxna, I think that covers everyting so I will revert https://gerrit.wikimedia.org/r/#/c/191198/ and close this unless you need something from us.

Thanks!

akosiaris closed this task as Resolved.Feb 27 2015, 9:48 AM

I am currently unable to clone from the private respository—

~/wmf
▶ git clone ssh://prtksxna@gerrit.wikimedia.org:29418/wikimedia/TransparencyReport-private
Cloning into 'TransparencyReport-private'...
Checking connectivity... done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.

@Prtksxna. OK, fixed.

I had tested cloning the repo as my user, but not actually cloning the repo as a member of the wikimedia-TransparencyReport group. I added read permissions for the group in the repo and it now works.

So I gave the group (prateek, moiz and ori) direct push access to the public repo, and they can update it as they see fit. The commit from qchris has been reverted, so after things are pushed puppet will pull them in, or someone from ops can pull them in.

@MSyed and I will be doing the deployment on Tuesday, 7 April 2015, at approximately 16:00:00 UTC. We will —

@akosiaris, as you had described earlier we'll be able to do all of this on our own. Even then, would it be possible for you to be available during this time?

@yuvipanda, thanks for all your help!

@Prtksxna, yeah I will be around, just make sure to reach me if you need be before 17:00 UTC as I will be unavailable for about an hour after that time.

demon removed a subscriber: demon.Apr 7 2015, 12:39 PM
matmarex removed a subscriber: matmarex.Apr 7 2015, 1:02 PM

They Legal team needs this to be made public in the next few days to launch the new version of Transparency Report. @Prtksxna and I are working on it but will be done in the next few days. We will need some help to make this public, @akosiaris.

Let us know if you can help and make the transparent report less private ;)

Thanks!

Restricted Application added a subscriber: Matanya. · View Herald TranscriptJul 13 2015, 6:12 AM

Specifically, we need is to force push the private repository to the public one once we are ready, so that the website updates at the right time.

Hello,

Just force pushing to the public repo instead of the private should be sufficient. I am not sure how you have setup your local environment but, assuming <username> stands for a username of either of you:

git push -f ssh://<username>@gerrit.wikimedia.org:29418/wikimedia/TransparencyReport.git master

should be sufficient.

As before, doing the push up to 30 mins before the actual desired time of release will mean you will not need to ping ops or anyone else to coordinate with you. In fact, with wikimania this week it might (or might not, I have no idea of what schedule you want to adhere to) be more difficult to find someone, so stopping by #wikimedia-operations and informing people before hand in case something does not work is advised. For what is worth though, I am pretty confident that since nothing has changed since last time, everything will work just fine.

Just force pushing to the public repo instead of the private should be sufficient.

Gerrit confuses me, wanted to make sure that I have the sufficient rights to force push.

As before, doing the push up to 30 mins before the actual desired time of release will mean you will not need to ping ops or anyone else to coordinate with you.

Thanks @akosiaris! Last time Yuvi was around in case the something went wrong, I'll be sure to hang out on #wikimedia-operations while force pushing this time.

I've forced push to the public repository and can see my latest commits on Github. Its been half an hour now, but the website itself hasn't updated.

@akosiaris, any ideas?

I've forced push to the public repository and can see my latest commits on Github. Its been half an hour now, but the website itself hasn't updated.

The website wasn't updating because we hadn't built it (using middleman). It got updated soon after @MSyed submitted the build patch.

@akosiaris, any ideas?

Thanks for all your help again!