Page MenuHomePhabricator

Weekly reports for Extension:SmiteSpam
Closed, ResolvedPublic

Description

Week 1 (25th May - 31st May)

  • Created minimum viable product (work continuing at https://gerrit.wikimedia.org/r/#/c/214303/)
    • Extension uses 3 checks to evaluate pages: a) ratio of external links to words, b) number of repeated external links, c) whether the page is new or not.
    • These pages are listed on the Special:SmiteSpam page and can be deleted by selecting the corresponding checkbox and hitting "Delete selected".
  • Tests set up for jenkins to run on new patch submission
  • Some significant issues:
    • A lot of the logic from WikiPage.php is being duplicated in SSWikiPage.php, I need to figure out a way to use the existing logic.
    • I'm not sure I'm using i18n messages correctly yet, but everything is still heavily under development, so I hope I'll be able to clean that up later.
    • The view to delete the pages is very rudimentary, just a table and checkboxes. The final version must, obviously, include some filtering and sorting features.
    • The tests are still not too useful on real spam. The most important test, I think, will be the one that deals with checking how much wikitext is being used. I would love suggestions on implementing that test.
    • The values returned by the checks are rather arbitrary, a convention/standard needs to be decided.

Week 2 (1st June - 7th June)

  • Updated minimum viable product. In another patch set or two, it will be merged and future development will continue as separate tasks and patches.
    • Deleted about ~100 lines of code after they were identified as duplicating functionality of classes already implemented in core, such as WikiPage, Title and Revision.
    • Added smitespam- prefix to i18n messages and converted a few more hardcoded strings to use i18n messages instead.
    • Changed some class structure and names
  • Lessons learnt:
    • A lot of work has already gone into implementing classes for MediaWiki core and almost every functionality can be satisfied by them. Do not forget to check once if what you're trying to implement has already been implemented.
    • The efficiency of the extension is probably not the primary concern, the ease of use of the deletion interface is.
    • Note that a lot of the functionality needs to be moved to use Jobs in later patches.

Week 3 (8th June - 14th June)

  • The minimum viable product was completed and merged. Significant changes were made to:
    • The analyzer code, checks were added, false positives were reduced, etc.
    • In order to prevent memory limit exceeding as well as timeouts, ~2000 (configurable) random pages were picked and analyzed.
  • A patch was submitted to also add a 1) preview of the page's contents and 2) the username of the creator of the page, to the output of the Special page. (https://gerrit.wikimedia.org/r/#/c/218268/)
  • Major lesson learnt: The extension is working with a lot of data, lots of unforeseen problems are going to occur because of this.

Week 4 (15th June - 21st June)

  • After a few more patch sets, the patch was merged.
  • Work started on switching from the Special Page itself performing the queries to using an API extension and paging analysis of the pages in the wiki (also removing randomization). This patch is, as of June 20th, very close to completion. (https://gerrit.wikimedia.org/r/#/c/218595/)

Week 5 (22nd June - 28th June)

  • After a few revisions, the patch to switch to loading pages using AJAX and an API extension was merged.
  • Fixed JS coding style errors and enabled JSCS and JSHint tests in I4788a56ef2f1.
  • Even though an API extension was being used, pages were still being deleted by normal form submission and the subsequent POST request. In another patch, the extension switched to deleting pages using MW core's API.

Week 6 (29th June - 5th July)

  • Improved evaluation code
    • 55eeb6f Merge different wikitext checkers into one
    • 869c1d0 Add config option to ignore small pages
  • Allow blocking of users using Special:SmiteSpam
    • eb1e400 Group results by creator on special page
    • 63460a4 Modify API response to also include whether user is blocked
    • 1af8712 Add option to block user in special page
  • Misc. improvements:
    • 70c3b7f Improve special page UI a little
    • 42f5424 Don't sort pages in Analyzer if accessing from API
    • 8b10f2d Add missing global in Analyzer

Week 7 (6th July - 12th July)

  • Started a patch to display reasons for why a page is spammy on the special page, but that was stalled due to the fact that it should probably be done after a pending big UI discussion.
  • A massive update, including a lot of refactoring which finally resulted in:
    • f3ef6ff Modify implementation and refactor JS of special page

Week 8 (13th July - 19th July)

Did not work the entire week. Was travelling a lot and didn't have my laptop with me.

Week 9 (20th July - 26th July)

  • 150ad70 Add option to trust users: Added an option to allow admins to mark users as "trusted". Trusted users are ignored by SmiteSpam and hence can be used to reduce false positives.

Week 10 (27th July - 2nd Aug)

  • Fell ill (dengue) and was unable to work pretty much the entire week. :/

Week 11 (3rd Aug - 9th Aug)

  • 800571a Select all delete checkboxes on selecting a user's block checkbox
  • Began discussions with mentors on a big UI update
  • College began

Week 12 (10th Aug - 16th Aug)

  • Settling into college turned out to be rather tiring and time consuming, in addition, a lot of initial college work made it difficult to work on anything.
  • Made a basic mockup of a new UI design at http://vghaisas.me/mockups/smitespam-cards/
  • Started work on updating the UI of the extension itself

Week 13 (17th Aug - 19th Aug)

  • 821a072 Change UI of Special:SmiteSpam: Big UI update. Could certainly use more work, but is probably easier on the eye than the earlier version.
  • Created T109509, T109510, T109511 to list out some remaining tasks.

Week 14 (20th Aug - 26th Aug)

  • 0d14c21 Update i18n messages
  • 4ec7481 Add unicode support for wikitext preview in Special:SmiteSpam
  • 6c1fa2a Add select all, select none options to Special:SmiteSpam
  • ef346f3 Display creation timestamp of pages in Special:SmiteSpam

Also wrote the documentation page on mw.org.

Event Timeline

polybuildr claimed this task.
polybuildr raised the priority of this task from to Normal.
polybuildr updated the task description. (Show Details)
polybuildr added subscribers: Niharika, Yaron_Koren, jan.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 2 2015, 1:02 PM
jan added a comment.Jun 2 2015, 9:55 PM

You have made a nice work :-)

It's good that you see the problem of duplicate the code from WikiPage. I think it is very important to centralize code and reusing it.

polybuildr updated the task description. (Show Details)Jun 9 2015, 3:04 PM
polybuildr set Security to None.
polybuildr updated the task description. (Show Details)Jun 20 2015, 8:15 PM
polybuildr updated the task description. (Show Details)Jun 29 2015, 7:57 PM
polybuildr updated the task description. (Show Details)
polybuildr updated the task description. (Show Details)Jul 24 2015, 3:06 PM
polybuildr updated the task description. (Show Details)Jul 24 2015, 5:32 PM
polybuildr updated the task description. (Show Details)Jul 24 2015, 5:40 PM
polybuildr updated the task description. (Show Details)Aug 18 2015, 8:55 PM
polybuildr updated the task description. (Show Details)Aug 21 2015, 1:50 PM
polybuildr updated the task description. (Show Details)Aug 21 2015, 2:05 PM
jan closed this task as Resolved.Aug 21 2015, 9:20 PM

Last week report was uploaded.

Thanks @polybuildr, @jan. Great job, folks!

jan added a comment.EditedAug 23 2015, 1:53 PM

Do not forget @Yaron_Koren :-) He was the mentor who has done the most work.

Right, a big thanks to @Yaron_Koren for his work. :)