Page MenuHomePhabricator

Figure out how to match phabricator milestone tags for wikibugs reporting
Closed, ResolvedPublic

Description

We recently moved to using milestone tags to track services tasks across their lifecycle (see T91716 for background). In Phabricator, those are modeled as sub-tags of the Services tag, and display as Services (next). These milestones are exclusive, which means that the plain services tag is removed when a task is categorized within services.

We would like to continue reporting on those tasks using wikibug. https://gerrit.wikimedia.org/r/#/c/315830/ has an attempt to match anything starting with services.*, but this does not match the milestone tags. We don't have visibility in how phabricator reports milestone tags to wikibugs, so would appreciate help figuring out how to match those.

Event Timeline

It seems the configuration in git is probably correct (see notes below), but just not deployed. I have manually pulled in the latest changes -- that should help.

(notes below for if something else is also wrong in the end)
It has been a while...

Phabricator side:

    def get_tags(self, task_page):
        soup = BeautifulSoup(task_page)
        alltags = {}

        for tag in soup.findAll(class_='phabricator-handle-tag-list-item'):
            taglink = tag.find('a', class_='phui-tag-view')
            if not taglink:
                continue

            marker = taglink.find('span', class_='phui-icon-view')

            classes = taglink['class'] + marker['class']

            shade = [cls.split("-")[3] for cls in classes if cls.startswith("phui-tag-shade-")][0]
            disabled = shade == "disabled"
            tagtype = [cls.split("-")[1] for cls in classes if cls.startswith("fa-")][0]
            uri = taglink['href']
            name = taglink.text
            alltags[name] = {'shade': shade,
                             'disabled': disabled,
                             'tagtype': tagtype,
                             'uri': uri}
        return alltags


[...]

 projects = self.get_tags(task_page)

        useful_event_metadata = {
            'url': phid_info['uri'] + anchor,
            'projects': projects,
            'user': self.get_user_name(event_info['authorPHID']),
        }

[...]

self.rqueue.put(useful_event_metadata)

IRC end:

    def channels_for(self, projects):
        """
        :param project: Get all channels to spam for given projects
        :type project: iterable
        :returns: dict[channel: matched projects]
        """
        channels = collections.defaultdict(list)
        for channel in self.config['channels']:
            for project in projects:
                if self.config['channels'][channel].match(project):
                    channels[channel].append(project)

        if not channels:
            channels[self.default_channel] = []

        channels.pop('/dev/null', None)
        channels[self.firehose_channel] = []

        return channels

channels = bot.chanfilter.channels_for(useful_info['projects'])

If I read that correctly, it matches on the text of the link (the keys of the dictionary -- in this case: "Services (blocked)"), and this should match the regexps specified in the configuration.

import requests, wikibugs, configfetcher, channelfilter
bugs = wikibugs.Wikibugs2(        configfetcher.ConfigFetcher()    )
tags = bugs.get_tags(requests.get("https://phabricator.wikimedia.org/T148650").text)
tags

{'Wikibugs': {'disabled': False, 'shade': 'blue', 'tagtype': 'briefcase', 'uri': '/tag/wikibugs/'}, 'Services (blocked)': {'disabled': False, 'shade': 'violet', 'tagtype': 'map', 'uri': '/project/view/2312/'}, 'Release-Engineering-Team': {'disabled': False, 'shade': 'violet', 'tagtype': 'users', 'uri': '/tag/release-engineering-team/'}}

chanfilter = channelfilter.ChannelFilter()
chanfilter.channels_for(tags)

defaultdict(<class 'list'>, {'#mediawiki-feed': [], '#wikimedia-releng': ['Release-Engineering-Team'], '#wikimedia-labs': ['Wikibugs'], '#wikimedia-devtools': ['Wikibugs']})

so even though it's in the config, it doesn't match?!

>>> chanfilter.config['channels']['#wikimedia-services']
re.compile('^(restbase(-.*)?|services(-.*)?|blocked-on-services)$', re.IGNORECASE)

That's not right...

Pchelolo claimed this task.
Pchelolo subscribed.

@valhallasw Looks like we've started to get the notifications in the services channel. Thank you, resolving!