Page MenuHomePhabricator

Create a release monitoring bot for Phabricator
Closed, ResolvedPublic

Assigned To
Authored By
Legoktm
Sep 25 2018, 12:56 AM
Referenced Files
None
Tokens
"Love" token, awarded by Quiddity."Orange Medal" token, awarded by Aklapper."Like" token, awarded by Jdforrester-WMF."Like" token, awarded by Mainframe98."Orange Medal" token, awarded by Krinkle.

Description

I currently subscribe to RSS feeds for some of the upstream libraries we maintain to know when updates are available. However this doesn't really scale past me.

This bot would follow upstream projects, and if a new release is detected, it would file a task indicating that a new version is available. It would then be up to the maintainers of that project to follow-up on the ticket.

If a new version is released the bot will check to see if the previous version's ticket was already closed, and if so, file a new bug. If it wasn't closed, it'll leave a comment indicating that an even newer version is available.

The bot's configuration will have a list of upstream projects, and for each upstream, it'll have a list of Phabricator projects to add to the ticket and people to subscribe.

It would be good if we could reuse the APIs of existing projects like https://release-monitoring.org/ and https://libraries.io/.

This would be similar to what Fedora does: https://bugzilla.redhat.com/show_bug.cgi?id=1525004.

Event Timeline

Legoktm triaged this task as Lowest priority.Sep 25 2018, 12:56 AM
Legoktm created this task.

There's a decent chance I end up implementing this in LibUp since it's already tracking upstream projects and pulling down the latest version of each on a regular basis.

Change 679284 had a related patch set uploaded (by Legoktm; author: Legoktm):

[labs/libraryupgrader@master] Add Phabricator release monitoring bot

https://gerrit.wikimedia.org/r/679284

Change 679285 had a related patch set uploaded (by Legoktm; author: Legoktm):

[labs/libraryupgrader/config@master] Add release monitoring configuration

https://gerrit.wikimedia.org/r/679285

Change 679285 merged by jenkins-bot:

[labs/libraryupgrader/config@master] Add release monitoring configuration

https://gerrit.wikimedia.org/r/679285

Change 679284 merged by jenkins-bot:

[labs/libraryupgrader@master] Add Phabricator release monitoring bot

https://gerrit.wikimedia.org/r/679284

OK, this is hopefully ready for usage now. I triggered it manually and it filed two tasks: T280116: New upstream release for squizlabs/PHP_CodeSniffer, T280117: New upstream release for Pygments (I have *not* enabled it to be automatic yet). It will keep track of tasks that it files and add comments to those if they're still open (it's not smart enough to search for tasks filed by humans).

You can see in https://gerrit.wikimedia.org/r/plugins/gitiles/labs/libraryupgrader/config/+/refs/heads/master/monitoring.json what the configuration should look like.

I ended up going with release-monitoring.org because when I checked libraries.io, it was missing the PHPCS release from 5 days ago. We may need to look into what the process for adding new projects there is, hopefully it works out.

Any feedback before we turn this on properly? It'll run every 24h when libup kicks off its daily run.

OK, this is hopefully ready for usage now. I triggered it manually and it filed two tasks: T280116: New upstream release for squizlabs/PHP_CodeSniffer, T280117: New upstream release for Pygments (I have *not* enabled it to be automatic yet). It will keep track of tasks that it files and add comments to those if they're still open (it's not smart enough to search for tasks filed by humans).

You can see in https://gerrit.wikimedia.org/r/plugins/gitiles/labs/libraryupgrader/config/+/refs/heads/master/monitoring.json what the configuration should look like.

I ended up going with release-monitoring.org because when I checked libraries.io, it was missing the PHPCS release from 5 days ago. We may need to look into what the process for adding new projects there is, hopefully it works out.

Any feedback before we turn this on properly? It'll run every 24h when libup kicks off its daily run.

It'd be good if the bot's Phab post could say what the current version it thinks we're running is, but that feels like it'd be complicated.

It'd be good if the bot's Phab post could say what the current version it thinks we're running is, but that feels like it'd be complicated.

I couldn't really think of a generic way to implement this so for now I've punted on it, but I do think it would be useful (we could then generate git changelog links). Note that it doesn't check Git repositories whatsoever, just the upstream info and the Phabricator project. I think this gives us more flexibility on what we can track.

Like I plan to add notifications for php-parallel-lint & covers-validator for LibUp/MediaWiki-General so we can bump the version in libup/config rather than looking at the version in individual repositories.

It'd be good if the bot's Phab post could say what the current version it thinks we're running is, but that feels like it'd be complicated.

I couldn't really think of a generic way to implement this so for now I've punted on it, but I do think it would be useful (we could then generate git changelog links).

Yeah, that's what I was thinking.

Note that it doesn't check Git repositories whatsoever, just the upstream info and the Phabricator project. I think this gives us more flexibility on what we can track.

Ack. Hence "complicated" compared to the current code. :-)

Like I plan to add notifications for php-parallel-lint & covers-validator for LibUp/MediaWiki-General so we can bump the version in libup/config rather than looking at the version in individual repositories.

Sounds great.

Change 680833 had a related patch set uploaded (by Legoktm; author: Legoktm):

[labs/libraryupgrader/config@master] Add documentation for monitoring.json

https://gerrit.wikimedia.org/r/680833

Change 680839 had a related patch set uploaded (by Legoktm; author: Legoktm):

[labs/libraryupgrader@master] Enable monitoring checks in nightly run

https://gerrit.wikimedia.org/r/680839

Change 680839 merged by jenkins-bot:

[labs/libraryupgrader@master] Enable monitoring checks in nightly run

https://gerrit.wikimedia.org/r/680839

Change 680833 merged by jenkins-bot:

[labs/libraryupgrader/config@master] Add documentation for monitoring.json

https://gerrit.wikimedia.org/r/680833

This is now enabled and will run every day. Once I finish the documentation I'll send an email to wikitech-l announcing the feature.

I ended up going with release-monitoring.org because when I checked libraries.io, it was missing the PHPCS release from 5 days ago. We may need to look into what the process for adding new projects there is, hopefully it works out.

You need a Fedora or OpenID account, which I think shouldn't be too burdensome. Once I found my Fedora account credentials, it took me ~30 seconds to create https://release-monitoring.org/project/192621/.

Small note: I wonder if the Phab task title could include the version info (allows easier differentiation when looking at lists of tasks), like New upstream release %version for %library

Small note: I wonder if the Phab task title could include the version info (allows easier differentiation when looking at lists of tasks), like New upstream release %version for %library

I can do that. Would the bot then update the title when it finds a new version and leaves a comment or just leave the title alone? Based on https://bugzilla.redhat.com/show_activity.cgi?id=1525004 it looks like Fedora updates the title.

No particular opinion - I had naively expected that the bot would create a new task everytime there's a new upstream release, and that it would not keep track of previous tasks (and their status whether a task is still open and should be 'bumped', or if a task has been resolved and a new task should be created). :)