Change Details

= Week 1 (25th May - 31st May) = - Created minimum viable product (work continuing at https://gerrit.wikimedia.org/r/#/c/214303/) - Extension uses 3 checks to evaluate pages: a) ratio of external links to words, b) number of repeated external links, c) whether the page is new or not. - These pages are listed on the Special:SmiteSpam page and can be deleted by selecting the corresponding checkbox and hitting "Delete selected". - Tests set up for jenkins to run on new patch submission - Some significant issues: - A lot of the logic from WikiPage.php is being duplicated in SSWikiPage.php, I need to figure out a way to use the existing logic. - I'm not sure I'm using i18n messages correctly yet, but everything is still heavily under development, so I hope I'll be able to clean that up later. - The view to delete the pages is very rudimentary, just a table and checkboxes. The final version must, obviously, include some filtering and sorting features. - The tests are still not too useful on real spam. The most important test, I think, will be the one that deals with checking how much wikitext is being used. I would love suggestions on implementing that test. - The values returned by the checks are rather arbitrary, a convention/standard needs to be decided. = Week 2 (1st June - 7th June) - Updated minimum viable product. In another patch set or two, it will be merged and future development will continue as separate tasks and patches. - Deleted about ~100 lines of code after they were identified as duplicating functionality of classes already implemented in core, such as `WikiPage`, `Title` and `Revision`. - Added `smitespam-` prefix to i18n messages and converted a few more hardcoded strings to use i18n messages instead. - Changed some class structure and names - Lessons learnt: - A lot of work has already gone into implementing classes for MediaWiki core and almost every functionality can be satisfied by them. Do not forget to check once if what you're trying to implement has already been implemented. - The efficiency of the extension is probably not the primary concern, the ease of use of the deletion interface is. - Note that a lot of the functionality needs to be moved to use `Job`s in later patches. = Week 3 (8th June - 14th June) - The minimum viable product was completed and merged. Significant changes were made to: - The analyzer code, checks were added, false positives were reduced, etc. - In order to prevent memory limit exceeding as well as timeouts, ~2000 (configurable) random pages were picked and analyzed. - A patch was submitted to also add a 1) preview of the page's contents and 2) the username of the creator of the page, to the output of the Special page. (https://gerrit.wikimedia.org/r/#/c/218268/) - Major lesson learnt: The extension is working with **a lot** of data, lots of unforeseen problems are going to occur because of this. = Week 4 (15th June - 21st June) = - After a few more patch sets, the patch was merged. - Work started on switching from the Special Page itself performing the queries to using an API extension and paging analysis of the pages in the wiki (also removing randomization). This patch is, as of June 20th, very close to completion. (https://gerrit.wikimedia.org/r/#/c/218595/) = Week 5 (22nd June - 28th June) = - After a few revisions, the patch to switch to loading pages using AJAX and an API extension was merged. - Fixed JS coding style errors and enabled JSCS and JSHint tests in [[ https://gerrit.wikimedia.org/r/#/c/220414/ | I4788a56ef2f1 ]]. - Even though an API extension was being used, pages were still being deleted by normal form submission and the subsequent POST request. In [[ https://gerrit.wikimedia.org/r/#/c/220742/ | another patch ]], the extension switched to deleting pages using MW core's API. = Week 6 (29th June - 5th July) - Improved evaluation code - 55eeb6f Merge different wikitext checkers into one - 869c1d0 Add config option to ignore small pages - Allow blocking of users using Special:SmiteSpam - eb1e400 Group results by creator on special page - 63460a4 Modify API response to also include whether user is blocked - 1af8712 Add option to block user in special page - Misc. improvements: - 70c3b7f Improve special page UI a little - 42f5424 Don't sort pages in Analyzer if accessing from API - 8b10f2d Add missing `global` in Analyzer == Week 7 (6th July - 12th July) == - Started [[ https://gerrit.wikimedia.org/r/#/c/222918/ | a patch ]] to display reasons for why a page is spammy on the special page, but that was stalled due to the fact that it should probably be done after a pending big UI discussion. - A **massive** update, including a lot of refactoring which finally resulted in: - [[ https://gerrit.wikimedia.org/r/#/c/223750/ | f3ef6ff ]] Modify implementation and refactor JS of special page == Week 8 (13th July - 19th July) == Did not work the entire week. Was travelling a lot and didn't have my laptop with me.

= Week 1 (25th May - 31st May) = - Created minimum viable product (work continuing at https://gerrit.wikimedia.org/r/#/c/214303/) - Extension uses 3 checks to evaluate pages: a) ratio of external links to words, b) number of repeated external links, c) whether the page is new or not. - These pages are listed on the Special:SmiteSpam page and can be deleted by selecting the corresponding checkbox and hitting "Delete selected". - Tests set up for jenkins to run on new patch submission - Some significant issues: - A lot of the logic from WikiPage.php is being duplicated in SSWikiPage.php, I need to figure out a way to use the existing logic. - I'm not sure I'm using i18n messages correctly yet, but everything is still heavily under development, so I hope I'll be able to clean that up later. - The view to delete the pages is very rudimentary, just a table and checkboxes. The final version must, obviously, include some filtering and sorting features. - The tests are still not too useful on real spam. The most important test, I think, will be the one that deals with checking how much wikitext is being used. I would love suggestions on implementing that test. - The values returned by the checks are rather arbitrary, a convention/standard needs to be decided. = Week 2 (1st June - 7th June) - Updated minimum viable product. In another patch set or two, it will be merged and future development will continue as separate tasks and patches. - Deleted about ~100 lines of code after they were identified as duplicating functionality of classes already implemented in core, such as `WikiPage`, `Title` and `Revision`. - Added `smitespam-` prefix to i18n messages and converted a few more hardcoded strings to use i18n messages instead. - Changed some class structure and names - Lessons learnt: - A lot of work has already gone into implementing classes for MediaWiki core and almost every functionality can be satisfied by them. Do not forget to check once if what you're trying to implement has already been implemented. - The efficiency of the extension is probably not the primary concern, the ease of use of the deletion interface is. - Note that a lot of the functionality needs to be moved to use `Job`s in later patches. = Week 3 (8th June - 14th June) - The minimum viable product was completed and merged. Significant changes were made to: - The analyzer code, checks were added, false positives were reduced, etc. - In order to prevent memory limit exceeding as well as timeouts, ~2000 (configurable) random pages were picked and analyzed. - A patch was submitted to also add a 1) preview of the page's contents and 2) the username of the creator of the page, to the output of the Special page. (https://gerrit.wikimedia.org/r/#/c/218268/) - Major lesson learnt: The extension is working with **a lot** of data, lots of unforeseen problems are going to occur because of this. = Week 4 (15th June - 21st June) = - After a few more patch sets, the patch was merged. - Work started on switching from the Special Page itself performing the queries to using an API extension and paging analysis of the pages in the wiki (also removing randomization). This patch is, as of June 20th, very close to completion. (https://gerrit.wikimedia.org/r/#/c/218595/) = Week 5 (22nd June - 28th June) = - After a few revisions, the patch to switch to loading pages using AJAX and an API extension was merged. - Fixed JS coding style errors and enabled JSCS and JSHint tests in [[ https://gerrit.wikimedia.org/r/#/c/220414/ | I4788a56ef2f1 ]]. - Even though an API extension was being used, pages were still being deleted by normal form submission and the subsequent POST request. In [[ https://gerrit.wikimedia.org/r/#/c/220742/ | another patch ]], the extension switched to deleting pages using MW core's API. = Week 6 (29th June - 5th July) - Improved evaluation code - [[ https://gerrit.wikimedia.org/r/#/c/221736/ | 55eeb6f ]] Merge different wikitext checkers into one - [[ https://gerrit.wikimedia.org/r/#/c/222277/ | 869c1d0 ]] Add config option to ignore small pages - Allow blocking of users using Special:SmiteSpam - [[ https://gerrit.wikimedia.org/r/#/c/222116/ | eb1e400 ]] Group results by creator on special page - [[ https://gerrit.wikimedia.org/r/#/c/222552/ | 63460a4 ]] Modify API response to also include whether user is blocked - [[ https://gerrit.wikimedia.org/r/#/c/222840/ | 1af8712 ]] Add option to block user in special page - Misc. improvements: - [[ https://gerrit.wikimedia.org/r/#/c/222710/ | 70c3b7f ]] Improve special page UI a little - [[ https://gerrit.wikimedia.org/r/#/c/222701/ | 42f5424 ]] Don't sort pages in Analyzer if accessing from API - [[ https://gerrit.wikimedia.org/r/#/c/222551/ | 8b10f2d ]] Add missing `global` in Analyzer == Week 7 (6th July - 12th July) == - Started [[ https://gerrit.wikimedia.org/r/#/c/222918/ | a patch ]] to display reasons for why a page is spammy on the special page, but that was stalled due to the fact that it should probably be done after a pending big UI discussion. - A **massive** update, including a lot of refactoring which finally resulted in: - [[ https://gerrit.wikimedia.org/r/#/c/223750/ | f3ef6ff ]] Modify implementation and refactor JS of special page == Week 8 (13th July - 19th July) == Did not work the entire week. Was travelling a lot and didn't have my laptop with me.

= Week 1 (25th May - 31st May) = - Created minimum viable product (work continuing at https://gerrit.wikimedia.org/r/#/c/214303/) - Extension uses 3 checks to evaluate pages: a) ratio of external links to words, b) number of repeated external links, c) whether the page is new or not. - These pages are listed on the Special:SmiteSpam page and can be deleted by selecting the corresponding checkbox and hitting "Delete selected". - Tests set up for jenkins to run on new patch submission - Some significant issues: - A lot of the logic from WikiPage.php is being duplicated in SSWikiPage.php, I need to figure out a way to use the existing logic. - I'm not sure I'm using i18n messages correctly yet, but everything is still heavily under development, so I hope I'll be able to clean that up later. - The view to delete the pages is very rudimentary, just a table and checkboxes. The final version must, obviously, include some filtering and sorting features. - The tests are still not too useful on real spam. The most important test, I think, will be the one that deals with checking how much wikitext is being used. I would love suggestions on implementing that test. - The values returned by the checks are rather arbitrary, a convention/standard needs to be decided. = Week 2 (1st June - 7th June) - Updated minimum viable product. In another patch set or two, it will be merged and future development will continue as separate tasks and patches. - Deleted about ~100 lines of code after they were identified as duplicating functionality of classes already implemented in core, such as `WikiPage`, `Title` and `Revision`. - Added `smitespam-` prefix to i18n messages and converted a few more hardcoded strings to use i18n messages instead. - Changed some class structure and names - Lessons learnt: - A lot of work has already gone into implementing classes for MediaWiki core and almost every functionality can be satisfied by them. Do not forget to check once if what you're trying to implement has already been implemented. - The efficiency of the extension is probably not the primary concern, the ease of use of the deletion interface is. - Note that a lot of the functionality needs to be moved to use `Job`s in later patches. = Week 3 (8th June - 14th June) - The minimum viable product was completed and merged. Significant changes were made to: - The analyzer code, checks were added, false positives were reduced, etc. - In order to prevent memory limit exceeding as well as timeouts, ~2000 (configurable) random pages were picked and analyzed. - A patch was submitted to also add a 1) preview of the page's contents and 2) the username of the creator of the page, to the output of the Special page. (https://gerrit.wikimedia.org/r/#/c/218268/) - Major lesson learnt: The extension is working with **a lot** of data, lots of unforeseen problems are going to occur because of this. = Week 4 (15th June - 21st June) = - After a few more patch sets, the patch was merged. - Work started on switching from the Special Page itself performing the queries to using an API extension and paging analysis of the pages in the wiki (also removing randomization). This patch is, as of June 20th, very close to completion. (https://gerrit.wikimedia.org/r/#/c/218595/) = Week 5 (22nd June - 28th June) = - After a few revisions, the patch to switch to loading pages using AJAX and an API extension was merged. - Fixed JS coding style errors and enabled JSCS and JSHint tests in [[ https://gerrit.wikimedia.org/r/#/c/220414/ | I4788a56ef2f1 ]]. - Even though an API extension was being used, pages were still being deleted by normal form submission and the subsequent POST request. In [[ https://gerrit.wikimedia.org/r/#/c/220742/ | another patch ]], the extension switched to deleting pages using MW core's API. = Week 6 (29th June - 5th July) - Improved evaluation code -- [[ https://gerrit.wikimedia.org/r/#/c/221736/ | 55eeb6f ]] Merge different wikitext checkers into one -- [[ https://gerrit.wikimedia.org/r/#/c/222277/ | 869c1d0 ]] Add config option to ignore small pages - Allow blocking of users using Special:SmiteSpam -- [[ https://gerrit.wikimedia.org/r/#/c/222116/ | eb1e400 ]] Group results by creator on special page -- [[ https://gerrit.wikimedia.org/r/#/c/222552/ | 63460a4 ]] Modify API response to also include whether user is blocked -- [[ https://gerrit.wikimedia.org/r/#/c/222840/ | 1af8712 ]] Add option to block user in special page - Misc. improvements: -- [[ https://gerrit.wikimedia.org/r/#/c/222710/ | 70c3b7f ]] Improve special page UI a little -- [[ https://gerrit.wikimedia.org/r/#/c/222701/ | 42f5424 ]] Don't sort pages in Analyzer if accessing from API -- [[ https://gerrit.wikimedia.org/r/#/c/222551/ | 8b10f2d ]] Add missing `global` in Analyzer == Week 7 (6th July - 12th July) == - Started [[ https://gerrit.wikimedia.org/r/#/c/222918/ | a patch ]] to display reasons for why a page is spammy on the special page, but that was stalled due to the fact that it should probably be done after a pending big UI discussion. - A **massive** update, including a lot of refactoring which finally resulted in: - [[ https://gerrit.wikimedia.org/r/#/c/223750/ | f3ef6ff ]] Modify implementation and refactor JS of special page == Week 8 (13th July - 19th July) == Did not work the entire week. Was travelling a lot and didn't have my laptop with me.