Page MenuHomePhabricator

AI which suggests best reviewers for a patch ("Patch wrangler")
Open, LowestPublic

Description

Build an AI that suggests the best code reviewers for a patch.

Wiki thing it helps with:

  • Speed up code review by assigning most likely reviewers.
  • Santhosh: Learn from who reviewed this kind of code/path of repo in the past and responsiveness(time taken to do first review)
  • Alternatively: try to partition existing code into areas and automatically "assign" "maintainers"
  • Stephen: would be cool if it suggested related Phabricator tickets to tag based on files changed and commit message

Things that might helps us get this AI built:

  • Niklas's code review statistic tool.

Event Timeline

There's quite some research existing on the topic of proposing assignees for code review or bugs; I guess you're aware? :)

I wrote https://tools.wmflabs.org/reviewers/ a while back to try and address this problem and I've found that it's pretty reliable/accurate for most of our repos.

Suggested reviewers should also be able to teach (un-train) the AI with patches they wrongly got suggested for.

@Aklapper, could you link to some of this existing research. It's not exactly my field ;)

Not sure it works well for mediawiki/core. For some other extensions (Translate, Babel) it seems to have me as top suggestion even though I'm already struggling to do reviews (but it's not really wrong to suggest me).

Not sure it works well for mediawiki/core. For some other extensions (Translate, Babel) it seems to have me as top suggestion even though I'm already struggling to do reviews (but it's not really wrong to suggest me).

Yeah, I had some ideas/plans on how to specially handle giant repos like mediawiki/core and ops/puppet using directories/files but didn't have time to finish it up...it worked well for MediaWiki extensions since there usually are only 2-3 people for each one.

There is a gerrit plugin that recommends reviewers based on blaming the changed lines. (See also T101131: Enable Gerrit reviewers-by-blame plugin.)

The ideal way to suggest reviewers would be some combination of their blame presence + their recent activity.

[Kind of offtopic comment: Posting too many links to academic papers about (mostly pretty technical) bug assignee recommendation approaches - feel free to skip]

@Aklapper, could you link to some of this existing research. It's not exactly my field ;)

@Halfak: Alright... I'm just dropping a long list of papers here which I read a while ago. And this might be very unhelpful - please tell me if/how I could be of more help. :)
Also, I referred to research about assigning to specific developers, not teams/groups. And I obviously focus on assigning tasks, not patches to review but some concepts are similar.

  • John Karsten Anvik. Assisting bug report triage through recommendation. PhD thesis, University of British Columbia, November 2007. Also available at http://www.cs.ubc.ca/grads/resources/thesis/May08/Anvik_John.pdf.
  • Davor Čubranić and Gail C. Murphy. Automatic bug triage using text categorization. In Frank Maurer and Günther Ruhe, editors, Proceedings of the 16nd International Conference on Software Engineering & Knowledge Engineering (SEKE), pages 92–97, 2004. Also available at http://www.cs.ubc.ca/labs/spl/papers/2004/seke04-bugzilla.pdf.
  • Jifeng Xuan, He Jiang, Zhilei Ren, Jun Yan, and Zhongxuan Luo. Automatic bug triage using semi-supervised text classification. In Proceedings of the 22nd International Conference on Software Engineering & Knowledge Engineering (SEKE), pages 209–214, Redwood City, USA, July 2010. Knowledge Systems Institute Graduate School. Also available at http://researchers.lille.inria.fr/~xuan/page/paper/seke_10.pdf.
  • Jifeng Xuan, He Jiang, Zhilei Ren, and Weiqin Zou. Developer prioritization in bug repositories. In Proceedings of the 34th International Conference on Software Engineering, ICSE ’12, pages 25–35, Piscataway, USA, 2012. IEEE.
  • John Anvik, Lyndon Hiew, and Gail C. Murphy. Who should fix this bug? In Proceedings of the 28th international conference on Software engineering, ICSE ’06, pages 361–370, New York, USA, 2006. ACM. Also available at http://www.cs.ubc.ca/labs/spl/projects/bugTriage/papers/icse2006.pdf.
  • John Anvik and Gail C. Murphy. Determining implementation expertise from bug reports. In Fourth International Workshop on Mining Software Repositories, ICSE Workshops MSR ’07, pages 9–16, 2007.
  • John Anvik and Gail C. Murphy. Reducing the effort of bug report triage: Recommenders for development-oriented decisions. ACM transactions on software engineering and methodology, 20(3):10:1–10:35, August 2011.
  • Gerardo Canfora and Luigi Cerulo. Supporting change request assignment in open source development. In Hisham M. Haddad, editor, Proceedings of the ACM symposium on Applied computing, SAC ’06, pages 1767–1772, Dijon, France, 2006. ACM. Also available at http://pdf.aminer.org/000/566/815/supporting_change_request_assignment_in_open_source_development.pdf.
  • Dominique Matter, Adrian Kuhn, and Oscar Nierstrasz. Assigning bug reports using a vocabulary-based expertise model of developers. In Michael W. Godfrey and Jim Whitehead, editors, MSR, pages 131–140. IEEE, 2009. Also available at http://flosshub.org/sites/flosshub.org/files/131AssigningBugReports.pdf.
  • Ahmed Tamrawi, Tung Thanh Nguyen, Jafar M. Al-Kofahi, and Tien N. Nguyen. Fuzzy set and cache-based approach for bug triaging. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC/FSE ’11, pages 365–375, New York, USA, 2011. ACM. Also available at http://home.engineering.iastate.edu/~atamrawi/publications/bugzie-fse11.pdf.
  • Kalyanasundaram Somasundaram and Gail C. Murphy. Automatic categorization of bug reports using latent dirichlet allocation. In Proceedings of the 5th India Software Engineering Conference, ISEC ’12, pages 125–130, New York, USA, 2012. ACM.
  • Tao Zhang and Byungjeong Lee. A hybrid bug triage algorithm for developer recommendation. In Proceedings of the 28th Annual ACM Symposium on Applied Computing, SAC ’13, pages 1088–1094, New York, USA, 2013. ACM.
  • Ramin Shokripour, Zarinah M. Kasirun, Sima Zamani, and John Anvik. Automatic bug assignment using information extraction methods. In Advanced Computer Science Applications and Technologies (ACSAT), 2012 International Conference on, pages 144–149, November 2012.
  • Ramin Shokripour, John Anvik, Zarinah M. Kasirun, and Sima Zamani. Why so complicated? simple term filtering and weighting for location-based bug report assignment recommendation. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR ’13, pages 2–11, Piscataway, USA, 2013. IEEE Press. Also available at http://www.cwu.edu/~janvik/papers/msr2013.pdf.
  • Gaeul Jeong, Sunghun Kim, and Thomas Zimmermann. Improving bug triage with bug tossing graphs. In Proceedings of the 7th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. ACM, August 2009.
  • Liguo Chen, Xiaobo Wang, and Chao Liu. An approach to improving bug assignment with bug tossing graphs and bug similarities. Journal of Software, 6(3), 2011.
  • Pamela Bhattacharya, Iulian Neamtiu, and Christian R. Shelton. Automated, highly-accurate, bug assignment using machine learning and tossing graphs. Journal of systems and software, 85(10):2275–2292, October 2012. Also available at http://www.cs.ucr.edu/~cshelton/papers/docs/bugtossml.pdf.
Aklapper renamed this task from Patch-wrangler -- suggests the best reviewers for a patch to AI which suggests best reviewers for a patch ("Patch wrangler").Sep 7 2020, 7:27 AM