Here's what I'm thinking:

• A GitHub app can provide more instant feedback than a daily script
• The logic could look something like this: a PR is created -> app living on toolforge is triggered via a webhook -> app checks if .gitreview exists in repo (to not act in non-mirrors) -> (app creates a patch in Gerrit?) -> app closes PR with a helpful message

I can implement this if needed.

@Majavah: Yay, thanks for your interest and for sharing your knowledge about a better approach! Help is definitely welcome as I am not a developer and don't use Github very often. :)

In my understanding the problem to solve before this one is to find a reliable way how to identify those repositories on Github which are *not* mirrored from Wikimedia Gerrit (random example: https://github.com/wikimedia/wikipedia-ios/ ). T109939 and T237470 basically.
We don't want to interrupt the workflow of those developers who use Github instead of Wikimedia Gerrit. :-/

For the record: I've created https://toolsadmin.wikimedia.org/tools/id/github-pr-closer.

Process update: it's working on my test repository!

The code is at rLTPRC tool-github-pr-closer. I'm not a writer, so I'd appreciate if someone could help with writing the closing message.

ping @Aklapper

While it’s very much not a common thing, I think we might still have some repos that are canonically on phabricator/diffusion which use an .arcconfig (off the top of my head)

We should check with Release-Engineering-Team about that

Regarding the message, it should definitely link to one of the gerrit getting starred guides etc. Can definitely find people to help write that (we have a few lovely tech writers who I'm sure would be more than happy to help)

Can I ask where you found the basis for your instructions?

Go to organization settings -> github apps -> new github apps

That doesn't seem to be right in our org:

And on "Installed GitHub Apps", there is no "New" type button (AFAIK they want you to install them via https://github.com/marketplace)

And the items on the WebHook page don't match the options you're saying to set

Thanks!

I'd probably change message_template.md to something like this:

:wave: Hi @{{author}},
	
Thank you for contributing to Wikimedia! This Github repository is a read-only mirror.
The project uses [Gerrit](https://gerrit.wikimedia.org) for code review.
Please [submit your pull request to Gerrit](https://www.mediawiki.org/wiki/Gerrit/Tutorial), so your code changes can be reviewed.

If you are interested in sticking around within the Wikimedia community, please take a look at https://www.mediawiki.org/wiki/New_Developers
Thanks in advance!

But let me ping a technical writer first! :)

In T249703#6042877, @Aklapper wrote:

I'd probably change message_template.md to something like this:

[...]

This sounds great, thanks! I think that at least in the beginning it would be nice to have some "come yell here if this is broken" link, do you think this should have its own project here or should it just eg. link to this task?

Is there a way we can run this on demand/interactively/similar? There's potentially some PRs in some less viewed github repos that haven't already been manually closed, and would be nice to get back to zero on them

In T249703#6042921, @Reedy wrote:

Is there a way we can run this on demand/interactively/similar? There's potentially some PRs in some less viewed github repos that haven't already been manually closed, and would be nice to get back to zero on them

This could be run as an one-off script: running a search for is:pr is:open org:wikimedia currently yields 332 results, which is an amount that can be just run in a go (with some delays to stay within rate limits).

In T249703#6042973, @Majavah wrote:

In T249703#6042921, @Reedy wrote:

Is there a way we can run this on demand/interactively/similar? There's potentially some PRs in some less viewed github repos that haven't already been manually closed, and would be nice to get back to zero on them

This could be run as an one-off script: running a search for is:pr is:open org:wikimedia currently yields 332 results, which is an amount that can be just run in a go (with some delays to stay within rate limits).

Cool! I note some repos do use github as their canonical repo, so not all of those open PR will want closing, but as the script will be following the same logic to leave comments and close PR, this shouldn't be an issue :)

In T249703#6042834, @Reedy wrote:

Can I ask where you found the basis for your instructions?
[...]
That doesn't seem to be right in our org:

For me "GitHub apps" is under "Developer settings", could you check again?

Also it looks like it's not possible to give permission to only 2 different files, it's one file or all files. That means that this will require to have read-only contents permission, does this cause any issues?

In T249703#6042985, @Majavah wrote:

In T249703#6042834, @Reedy wrote:

Can I ask where you found the basis for your instructions?
[...]
That doesn't seem to be right in our org:

For me "GitHub apps" is under "Developer settings", could you check again?

Might be worth clarifying in your documentation a little, it wasn't immediately obvious :)

In T249703#6042985, @Majavah wrote:

Also it looks like it's not possible to give permission to only 2 different files, it's one file or all files. That means that this will require to have read-only contents permission, does this cause any issues?

I honestly don't think so. All our repos are public, so the bot/script isn't going to be able to access anything that anyone (the script or otherwise) couldn't just access directly via the web etc, or by cloning the repo.

If we had private repos (with for example passwords), I could understand there being a need to have a restriction. But for our use case, I don't see this being an issue at all

In T249703#6042834, @Reedy wrote:

While it’s very much not a common thing, I think we might still have some repos that are canonically on phabricator/diffusion which use an .arcconfig (off the top of my head)

We should check with Release-Engineering-Team about that

T191182: Migrate active repositories in Phabricator Differential to GitLab - It's definitely deprecated

In T249703#6043379, @Reedy wrote:

In T249703#6042834, @Reedy wrote:

While it’s very much not a common thing, I think we might still have some repos that are canonically on phabricator/diffusion which use an .arcconfig (off the top of my head)

We should check with Release-Engineering-Team about that

T191182: Migrate active repositories in Phabricator Differential to GitLab - It's definitely deprecated

If support for Diffusion is not added the only downside is that PRs in repositories hosted there will not get automally closed - so not false positives here, just false negatives. Supporting adds an extra API call to GitHub, so any objections if that's not supported (initially, at least)?

In T249703#6043285, @Reedy wrote:

In T249703#6042985, @Majavah wrote:

In T249703#6042834, @Reedy wrote:

Can I ask where you found the basis for your instructions?
[...]
That doesn't seem to be right in our org:

For me "GitHub apps" is under "Developer settings", could you check again?

Might be worth clarifying in your documentation a little, it wasn't immediately obvious :)

{{done}}

@Aklapper the text of that message looks good!

Thanks @srodlund. @Reedy can you do the configuration on GitHub side or should I contact someone else?

I created the script for closing down existing pull requests. AFAIK this is now ready for deployment.

Thanks!

As always, some code review is a good idea

I note https://phabricator.wikimedia.org/source/tool-github-pr-closer/browse/master/close-old-prs.py seems to have a lot of duplicated code compared to https://phabricator.wikimedia.org/source/tool-github-pr-closer/browse/master/github.py

If you've structured the code like you have to have functions in github.py, why not make use of them in close-old-prs.py? I know the code isn't all exactly the same, but it's mostly the same, doing the same token requests etc

In T249703#6049108, @Reedy wrote:

Thanks!

As always, some code review is a good idea

[...]

`Thanks, I dryed the code. Anything else?

@Aklapper @Reedy What would be the next steps in getting this deployed?

In T249703#6042896, @Majavah wrote:

This sounds great, thanks! I think that at least in the beginning it would be nice to have some "come yell here if this is broken" link, do you think this should have its own project here or should it just eg. link to this task?

Hmm, for now I'd say link to this task should be fine if as I cannot imagine too many requests? I might be wrong in hindsight. :) If you want a project that's fine to of course, see https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects :)

In T249703#6051117, @Majavah wrote:

@Aklapper @Reedy What would be the next steps in getting this deployed?

As I have no clue about Github, I'd love Release-Engineering-Team to take a look at this. (Or maybe even @thcipriani as I just stumbled across T245826?) :)

I don't mind setting it up

Webhook: active=true, url=https://(base-url)/github, secret=(generate random string here)

What's the URL for the bot/tool?

GitHub app name, Description, Homepage URL: these are displayed to the user, so please put something helpful here. app name is used for when naming the bot account.

We should write a description and decide what URL to use for the homepage

App name? "Wikimedia PR Closer"

Or keep something more generic?

@Aklapper have we filed a task for doing something similar for issues?

I understand the app Name will be the bot name, so rather than "Wikimedia PR Closer" I would prefer something more user-friendly, such as "Wikimedia Welcome Bot for GitHub users"

Well, it's not really welcoming anyone ;)

"Wikimedia Helper Bot for GitHub users"

Heh, calling it "Welcome Bot" sounded nicer, even if it would still be saying go away from GitHub ;)

T249703#6042877 seemed like a welcome to me, but "Wikimedia Helper Bot for GitHub users" is good to go, too.

In T249703#6069939, @Reedy wrote:

Webhook: active=true, url=https://(base-url)/github, secret=(generate random string here)

What's the URL for the bot/tool?

The webhook on the Toolforge side should probably use the new --canonical flag to webservice and its *.toolforge.org hostname so that the GitHub side of the config doesn't need to be changed once the toolforge.org migration reaches the forced migration stage.

https://github-pr-closer.toolforge.org/

(edit: forgot to add the URL)

In T249703#6069939, @Reedy wrote:

I don't mind setting it up

Great, thanks!

Webhook: active=true, url=https://(base-url)/github, secret=(generate random string here)

What's the URL for the bot/tool?

Webhook URL should be https://github-pr-closer.toolforge.org/github

GitHub app name, Description, Homepage URL: these are displayed to the user, so please put something helpful here. app name is used for when naming the bot account.

We should write a description and decide what URL to use for the homepage

App name? "Wikimedia PR Closer"

Or keep something more generic?

+1 for naming it "Wikimedia Helper Bot for GitHub users". How about using https://www.mediawiki.org/wiki/Gerrit as the homepage URL?

have we filed a task for doing something similar for issues?

Issues can easily be disabled from repository settings.

Wikimedia Helper Bot for GitHub users

is too long, so going with

Wikimedia Helper Bot for GitHub

It's alive!

https://github.com/wikimedia/mediawiki/pull/117

Big Thanks to everybody who worked on this (especially Majavah for diving into this with a speed and a passion that I unfortunately could not always follow and support myself because too many things on the to-do list).
@Majavah: I really appreciate that you made this happen and I owe you a drink once there are physical Hackathons and conferences again.

@Majavah: I read this entire ticket again wondering which logic identifies (and excludes) repos which are canonical on Github and not mirrored to Github.
Asking because we've been wondering in T109939 how to identify them, so any knowledge sharing is welcome in T109939! :)

In T249703#6157956, @Aklapper wrote:

@Majavah: I read this entire ticket again wondering which logic identifies (and excludes) repos which are canonical on Github and not mirrored to Github.
Asking because we've been wondering in T109939 how to identify them, so any knowledge sharing is welcome in T109939! :)

It looks for .gitreview and possibly .arcreview I think

In T249703#6157960, @RhinosF1 wrote:

In T249703#6157956, @Aklapper wrote:

@Majavah: I read this entire ticket again wondering which logic identifies (and excludes) repos which are canonical on Github and not mirrored to Github.
Asking because we've been wondering in T109939 how to identify them, so any knowledge sharing is welcome in T109939! :)

It looks for .gitreview and possibly .arcreview I think

This tool only looks for .gitreview due to T191182: Migrate active repositories in Phabricator Differential to GitLab. That file looks like this:

[gerrit]
host=gerrit.wikimedia.org
port=29418
project=wikimedia/meet-accountmanager.git
track=1
defaultrebase=0

where project is the repository name in Gerrit with a .git suffix. I'm happy to write a script for T109939 if you can decide the format and landing page.

It would be nice to link to or at least name the Gerrit repository name in the closing message. (It would be even cooler to provide specific instructions on how to submit the pull request to Gerrit, not sure how feasible that is though.)

In T249703#6157963, @Majavah wrote:

This tool only looks for .gitreview

Ah, thanks! :) Alright, that means checking for existence of a file, so looks like we cannot use that approach in T109939 because that's not API metadata...

I am slightly hijacking this closed task. We got a notification from Github warning about the removal of the events integration_installation and integration_installation_repositories:

We're contacting you regarding upcoming changes to GitHub Apps owned by your organization:

• Wikimedia Helper Bot for GitHub

We no longer support two events which your GitHub Apps may rely on, integration_installation and integration_installation_repositories.

These events can be replaced with the installation and installation_repositories events respectively.

The integration_installation and integration_installation_repositories events will be removed after October 1st, 2020.

Please visit https://developer.github.com/changes/2020-04-15-replacing-the-installation-and-installation-repositories-events for more information about suggested changes, brownouts, and removal dates.

From a quick look at the code, nothing matches integration so maybe we are not affected?

@hashar see T261294 T264138

In T249703#6504238, @Majavah wrote:

@hashar see T261294 T264138

Guess I should have done a search first. thank you!