IRC bot to report Phabricator activity exactly like Wikibugs does with Bugzilla
Closed, ResolvedPublic

Description

Wikibugs IRC bot reports Bugzilla activity to some IRC channels. We need a solution to get exactly the same type of reports from Phabricator.

This ticket is for the Bugzilla migration only. A separate ticket will exist for rt. And further down the road for gerrit or anything.

Bot parity for phabricator with bugzilla is the goal here.

Details

Reference
fl221

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes

yuvipanda wrote on 2014-07-24 10:53:39 (UTC)

How configurable is this? Right now it relays gerrit merges to operations/puppet.git into #wikimedia-dev, which is uh oh. Our current bots are also *very* configurable in what channel to send what thing, so if those features don't exist we should work with upstream to add them.

mmodell wrote on 2014-08-05 16:48:55 (UTC)

@yuvipanda phabricator's chat bot is somewhat less configurable. It should be reasonably easy to add some filtering capabilities.

dzahn wrote on 2014-08-20 20:39:47 (UTC)

I believe replacing the current wikibugs isn't as easy of a task as it might have sounded above. There is a very specific configuration that maps Bugzilla products to IRC channel names, and people have been taken this stuff pretty serious in the past (read: tons of discussion which bot should be where and what output is too little, what is too much).

dzahn wrote on 2014-08-20 20:40:55 (UTC)

it might be easier to have phab generate mails and adapt wikibugs to understand those mails (like it currenlty does for BZ) than using the phabricator bot

yuvipanda wrote on 2014-08-20 21:24:36 (UTC)

Nah, we should just upstream fixes to make the phab bot more configurable and then port the config.

aklapper wrote on 2014-08-20 22:07:27 (UTC)

In T221#45, @Dzahn wrote:

it might be easier to have phab generate mails and adapt wikibugs to understand those mails (like it currenlty does for BZ) than using the phabricator bot

In case that's considered the (or one) way to go, Phabricator now provides an "X-Phabricator-Projects" mail header to parse.

dzahn wrote on 2014-08-28 00:55:18 (UTC)

yuvipanda, not sure how well the "just upstream" part worked so far though

Rush wrote on 2014-09-05 17:02:59 (UTC)

I would rather keep this ticket as solely transitioning existing logic. Another ticket for making behavior not previously available can be made as well.

qgil wrote on 2014-09-05 17:06:00 (UTC)

Wikimedia Phabricator Day 1 is getting closer, progress on this task is not blocked by any of the big topics we are dealing with, and yet there is little progress seen here.

@Aklapper marked this task as Ready to Go, however...

  • Who is going to work on this, @mmodell (assigned) or @yuvipanda. I have no opinion other than having the task assigned to whoever is going to drive the work.
  • Are we using wikiphabot or not? Looking at the task description we see two wikiphabot tasks blocking; reading the discussion it looks like we are actually not using wikiphabot to resolve this task. Again, no opinion other than having a single plan agreed.

qgil wrote on 2014-09-05 17:10:28 (UTC)

In T221#56, @Rush wrote:

I would rather keep this ticket as solely transitioning existing logic. Another ticket for making behavior not previously available can be made as well.

Agreed. I just wanted to make the title and description more precise, with "Wikibugs" instead of "Bugzilla bots".

Rush wrote on 2014-09-05 17:10:47 (UTC)

All good questions, but I stll would like to keep this ticket as reflecting only to mirror the existing logic. Saying this is to stream activity is a very broad topic. When this ticket is closed something that mimics existing behavior needs to exist. Can we please change the title back to reflecting this and anything not 1:1 can go somewhere else.

qgil wrote on 2014-09-07 10:08:03 (UTC)

◀ Merged tasks: T647.

qgil wrote on 2014-09-07 10:22:04 (UTC)

In T221#45, @Dzahn wrote:

it might be easier to have phab generate mails and adapt wikibugs to understand those mails (like it currenlty does for BZ) than using the phabricator bot

Relevant: T647: wikibugs should Ignore CC changes

qgil wrote on 2014-09-07 10:22:04 (UTC)
Relevant: T647: wikibugs should Ignore CC changes

For the records, that is now T453.

flimport added a subscriber: mmodell.
flimport added a subscriber: Qgil.Oct 2 2014, 9:48 PM
flimport added a subscriber: Dzahn.Oct 2 2014, 9:55 PM
flimport added a subscriber: greg.Oct 2 2014, 9:58 PM
flimport added a subscriber: Legoktm.
flimport added a subscriber: Elitre.
mmodell removed mmodell as the assignee of this task.Oct 4 2014, 3:53 PM

@Dzahn have you had a chance to look into this?

mmodell lowered the priority of this task from High to Lowest.Oct 4 2014, 3:53 PM
flimport added a subscriber: scfc.Oct 7 2014, 3:00 AM

Okay, I've taken a further look at a possible migration path for pywikibugs. The main problem is that, even though we now have emails, they are very hard to parse. They are great for humans... but not so much for computers. Transferring the channel config to wikiphabot (T324) looks like a more promising solution.

Okay, I have the phab IRC bot running under the wikibugs tools account now ('phawikibugs' in #wikimedia-feed). This is also a test to see if it arrives on IRC ;-)

If that works, the T324 certainly looks like the right thing to do.

flimport added a subscriber: revi.Oct 10 2014, 9:00 PM
Qgil assigned this task to valhallasw.Oct 10 2014, 9:40 PM
Qgil raised the priority of this task from Lowest to Normal.
Qgil set Security to None.
Qgil moved this task from Waiting For Migration Start to Doing on the Bugzilla-Migration board.
valhallasw removed valhallasw as the assignee of this task.Oct 23 2014, 6:43 PM

Un-assigning this task again. There is a task list at https://phabricator.wikimedia.org/tag/phawikibugs/ and I'd be happy to help anyone get started, but I don't feel like I can commit to actually making sure this task gets finished in a reasonable time frame.

Qgil moved this task from Doing to Backlog on the Bugzilla-Migration board.Oct 30 2014, 8:22 AM
Qgil added a comment.Oct 30 2014, 8:35 AM

Thank you @valhallasw for your help kicking this task. This is probably the last Phabricator migration task that still doesn't have a clear owner. Anybody willing to claim it and drive it to a level that it is not a launch blocker anymore?

I've tried to clarify the current blockers of this task. Feel free to edit/revert to get a more accurate picture and/or a better setup for your work.

I would be glad to work on this, however, I'm currently kinda booked up with other stuff that seems at least as important if not more so. I can put this into my backlog but I can't be sure how soon it'll get finished.

Qgil lowered the priority of this task from Normal to Lowest.Nov 1 2014, 7:37 AM
Qgil added a subscriber: yuvipanda.

I am asking @yuvipanda and @Legoktm (or their managers, or someone else they know at the WMF) to be officially responsible of fixing this task, and get some time allocated accordingly. Chase and Mukunda have their hands full with Bugzilla & RT migrations tasks.

This is kind of urgent. We want to start the migration on 21 Nov, and it would be odd that after all the actual blockers are IRC and wikibugs-l...

I must say it looks quite odd to see priority lowered to Needs Volunteer at the same time as you say it's urgent and you want a WMF team (though a different one) to handle it. If this is expected work flow we may want to reconsider the needs volunteer phrase...

Qgil raised the priority of this task from Lowest to Normal.Nov 1 2014, 3:49 PM

You might be right.

After investigating and attempting to adapt the existing Phabricator IRC bot to what we need (https://github.com/legoktm/phabricator/commit/7b93dd1bb9eae1a350e52d691a2ddfbaddc44dac), I decided it wasn't worth it since we'd have to re-write most of it and what existed was trivial to re-implement.

My current WIP is at https://github.com/legoktm/wikibugs2. There are two parts, the first part converts the Phabricator API responses into useful, structured data and puts it into a redis queue. The second part reads from the queue, creates a formatted IRC message and sends it out.

Qgil awarded a token.Nov 3 2014, 7:21 AM

@Legoktm wait, the way is for us to poll phabricator every second or two? Is that what the current phab bot is also doing? Ewww...

Qgil moved this task from Backlog to Doing on the Bugzilla-Migration board.Nov 3 2014, 9:33 PM
Qgil assigned this task to yuvipanda.

As an update, we currently have a stream of phab changes coming in and being filtered. Just need to figure out how to emit them to IRC properly.

Thanks to @Legoktm for most of the work :)

I have turned off phawikibugs now, and wikibugs2 should be reporting the same changes now, in a better format :)

It's currently running in the wikibugs tool account, so myself, Yuvi, valhallasw, and 20after4 have access to kill it and start up phawikibugs if necessary.

Qgil added a comment.Nov 10 2014, 2:43 PM

Is this task complete now, or is there anything that MUST to be done before the Bugzilla migration?

Unless we consider T1120 blocking, I think this is now complete.

Qgil added a comment.Nov 10 2014, 9:10 PM

T1120 doesn't correspond to any Bugzilla use case, so in my opinion this is not a blocker.

valhallasw closed this task as Resolved.Nov 10 2014, 10:09 PM

Sounds good to me.