Hello,

In T139810#2487919, @Harej wrote:

Let's do a simple exercise:
What is your role on the wiki where you use CheckUser?

Wikimedia Steward and CheckUser at two other projects.

What prompts you to use CheckUser? (Vandalism, suspected sockpuppetry, etc.)

To deter and prevent instances of vandalism, spam, crosswiki abuse and abuse of multiple accounts, across many Wikimedia projects due to my role as steward.

What frustrates you when you try to use CheckUser?

May I point you to the workboard?

The only specific complaint I see listed here is that the interface is bad, which is a very valid complaint. If that's so, there should be a task for re-designing the interface, with an assessment of who uses the tool and what the tool is used for so that the interface can be optimized for its use cases.

I refer you to the introduction of this RFC. There is a bug for this T132892: CheckUser UI revamp.

This is a very strong assertion, and I do not see specifically why this is the case. Is there something that makes it structurally incompatible with MediaWiki? Does it not work at all? Does it have an unacceptable false positive or false negative rate?

There have been attempts to create a global checkuser tool. I can't quote vermatim from the source because it's prohibited, but we've been told that with the current state of the tool is impossible to do so.

So, to recap, we have two great issues: the first is the UI redesign which will be a great improvement, but we also need the second, which is a global CU tool; and to the best of my knowledge, the second one can't happen just with just improving the UI.

In T139810#2485737, @RobLa-WMF wrote:

An analysis of the age and turnover of the bugs tagged CheckUser would make this a much stronger RFC. Would you be willing to do that work?

Yes, I can create a table listing the open bugs with the date of creation. So far, we've 42, which are (ordered by ticket number):

• T11858: Recursive checkuser feature needed
• T16699: More versatile searching in CheckUser log
• T18306: Allow user to specify block settings from CU form
• T21796: Checkuser watchlist feature
• T21964: Provide a link to the user on account creation
• T24119: Enhance Blocking and tagging from checkuser interface: add new page
• T24120: Enhance Blocking and tagging from checkuser interface: add dropdowns
• T24890: Checkuser mass blocker does not affect already blocked users
• T26231: Allow CheckUsers to filter checks for account creations only
• T26232: Allow CheckUsers to filter results to new accounts only
• T26411: List recent User-Agents for a user or IP
• T26552: Wiki-email to store a hash of the message text
• T27053: Replace user / talk pages feature of "Get users" creates a user/talk page if it doesnt exist
• T29431: Drop legacy installer script from CheckUser extension (update.php hooks should already exist)
• T33712: CheckUser User-Agent check feature
• T36372: New User hook broken for PG, breaks normal logging
• T36438: Reverse DNS lookup
• T39612: CheckUser displays 2001:6a0:200:121::2/128 and 127.0.0.1/32 on PostgreSQL
• T39613: CheckUser records XFF: 0 on PostgreSQL
• T39971: No results on CheckUser query from 127.0.0.1 on PostgreSQL
• T41013: Incomplete i18n for log entries in CheckUser
• T46800: IPv6 data loaded from recentchanges table is in compact instead of full address format
• T49505: CheckUser results (userlinks) cannot be customized
• T54849: Checkuser option "get users" should point out when user password is changed
• T61677: FlaggedRevs reviews should be visible in CheckUser
• T65011: the messages checkuser-email-action and checkuser-reset-action need to support GENDER
• T69811: CheckUsers "Get users" design needs improvements
• T94735: Rewrite Special:CheckUser using server side templates
• T99056: Improve GENDER support for checkuser logs
• T106282: action=query list=checkuserlog API response is inconsistent with other action=query responses
• T107651: Mark accounts that hit limit of account creation in one IP for functionaries
• {T112751}
• T113205: checkuserlog API documentation references non-existent "culstart" and "culend"
• T117801: Checkuser should provide option to expose registered email address
• T121190: Add an option to show only IPs or accounts in get users
• T122296: Allow checking by last 64 bits of an IPv6 address
• T123114: CheckUser stores two rows for account creations on WMF wikis
• T125324: Allow checkusers to compare the watchlists of two accounts
• T131207: Create a checkuser entry for global rename requests
• T132892: CheckUser UI revamp
• T139809: Bad output of AbuseFilter blocks in CU get edits
• T139810: RFC: Overhaul the CheckUser extension

It'll get me some more time to get the creation dates of the bug, as many of them were filled under the old Bugzilla service.

Best regards.

Since some of them are quite old, I think that we need to clear that list of no longer valid bugs, just in case the issues reported got already resolved. Then focus on the UI and i18n bugs IMHO.

"Rewrite from scratch" is one potential solution to big problems with an existing codebase. I'd still love to see an improved explanation providing specific problems with the current extension.

@MarcoAurelio: T139810#2444212 states that "CU is obsolete, and has flaws that require a major rewrite to fix it." This feels like a statement that welcomes more elaboration - due to which specific architecture issues or specific bugs / missing functionality did you come to this conclusion?

Relatedly, T139810#2485937 asks "What's the most important problem to solve from your perspective?" and T139810#2487919 asks "What frustrates you when you try to use CheckUser?"
These are good questions, as every non-trivial piece of software has bugs and unfulfilled feature requests. Hence 'just' looking at the entire list of open tasks does not provide a strong argument for the proposed 'solution' (rewriting from scratch; which might introduce new regressions as explained in the link in T139810#2443633).

T139810#2443688 mentions that the "most useful addition would be a global checkuser". Feasibility whether this could be achieved via changes to the existing codebase should be discussed in a dedicated task. (Is T131207: Create a checkuser entry for global rename requests about this?)
And the initial description also mentions T132892: CheckUser UI revamp.
Currently these seem to be the only two specific issues addressing the question.

(In addition, there might be (a perception of?) a low level of maintainership. This has been challenged by provided numbers for Git activity in T139810#2460521.)

I also don't think we need to start from scratch and write a new extension. Trying to do that would almost certainly be more work than working on refactoring and improving the current code. It's also not like CheckUser is the only extension deployed on our wikis with bugs that are not quickly fixed (and so needing completely new extensions). The lack of an active maintainer also doesn't seem to be much of an issue during this year so far as I have received reviews quicker than previously. I think I will have some free time in the upcoming weeks so assuming that others (both devs and users) are interested in this, I can attempt to work on improving the extension as promised on the other task.

In T139810#2443633, @Legoktm wrote:

At Wikimania I had the opportunity to sit down with @Ajraddatz and understand how he uses CU, and got some good notes and workflow ideas from it.

Is it possible to share it with us? Maybe in a private paste if it cannot be made public.

https://www.mediawiki.org/wiki/User:Legoktm/Wikimania_2016_steward_tools

In T139810#2489777, @Aklapper wrote:

(In addition, there might be (a perception of?) a low level of maintainership. This has been challenged by provided numbers for Git activity in T139810#2460521.)

I'm not sure what data git-blame uses to create such info, but I think that table is not accurate since nearly all commits are translation updates to the i18n files which a bot do import from translatewiki.net. "Human" commits to the repo are just 36 this year and not 840 as mentioned by Tim above; however I accede to the point that my affirmation that "nobody" works on the extension was not accurate; and of course let me say again that the contributions are appreciated.

"Rewrite from scratch" is one potential solution to big problems with an existing codebase. I'd still love to see an improved explanation providing specific problems with the current extension.

Well, I think we all concurr that the current status of the UI needs an update and that we also would benefit from new features in the future, which are being discussed separately. Can we focus on UI for now at least, if a rewrite from scratch is not optimal (I don't know, I'm not an engineer again).

T139810#2443688 mentions that the "most useful addition would be a global checkuser". Feasibility whether this could be achieved via changes to the existing codebase should be discussed in a dedicated task. (Is T131207: Create a checkuser entry for global rename requests about this?)

Nope, global CU is a wanted new tool. I'm not sure why there's not a task created about it but work was done. Sadly the entire project is deceased. What is being requested there is that the software should create a CU entry and register global rename requests requested through Special:GlobalRenameRequest.

I don't think listing ever single task in the E:CheckUser workboard is the best way forward for this task.

I think the list should be split up in the goals, for example, Tasks that should be worked in the Short/Medium/Long terms, It should also be looked at what would have benefits to the end users of the tools, back-end users (eg: ops team for maintenance scripts) and non wmf-cluster users.

Also we should talk about the scope of the tasks that should be looked at, For example there are several PostgreSQL related tasks so those probably shouldn't be classed as a priority in a WMF priority schedule (If they are fixed in the course of other tasks and improvements such as Database abstraction improvements, great!) as that DB back-end is not used on the cluster.

There is also a couple of tasks that at quick scan, I don't believe would align with our Privacy Policy, Legal guidelines or have consensus (but could be desired by other external users) so they should be prioritized lower as well.

I am pragmatic (that was a disclaimer), and therefore I see little value discussing whether we should rewrite from scratch or modify the existing code when the end product is unclear. I think instead of using this task to discuss philosophical questions about rewrite vs modify or is anyone contributing to this code at all, we should use it to reach consensus about what CU tool should look like. And depending on what we agree on, we can decide if a rewrite is needed or not.

Again, I am pragmatic! So I don't see much value in telling others what we should do; I see value in actually doing that thing, in this case in actually drawing a picture of what CU should look like.

I think the CU tool should allow the following (it currently does not):

• Sorting the results by IP (I often want to find the high level "ranges" used by a user, that gives me an easy way to screen if two users are similar at all (if their ranges are not even close, then they are not). Right now, results are always returned sorted by date; I need to be able to sort by IP.
• A list of distinct UAs used by a user. In fact, I think the "get IPs" function should be replaced by a "get summary" page which shows to you distinct IPs (sortable by time or by IP), distinct UAs (sortable by time or UA), and distinct IP-UA combos (sorted by time).
• More decision support for UAs is needed. I don't think it'll be that much extra time to replicate what http://useragentstring.com/ or similar websites do. In the ideal world, I want CU tool to analyze the UA for me and already show me information like:
  • What browser, OS, etc does this UA represent
  • What year and month was that particular browser version released, and what year and month was a next version released (I find the date at which a user upgrades their browser a good clue for matching accounts or rejecting their similarity)
  • Does it look like a valid or a forged UA?
• I know I might be asking for something that will never happen but: more decision support for IPs! I want CU tool to already tell me which country an IP belongs to, which ISP, etc. I know this information changes over time, and is not publicly and freely available, but I think at the very least, the CU code should be modified such that you could "extend" it by providing a CSV file containing IP-to-country and/or IP-to-ISP mappings. That way, we don't need to publish such a mapping as part of the code, but major users of the CU tool such as WMF can pay for proprietary lists like that and install them for their wiki. I envision a day that I run the CU tool and don't have to run fifty WHOISes right after.

I can think of many other front- and backend improvements. I encourage us to discuss those here and mock up the CU 2.0 rather than discussing what to do with old code.

Hi @Huji, thanks for the many clarifications, and bringing this back to a specific design. Given that this is not going to be an easy RFC (where "easy" means: very clear question, very little prose necessary), it seems like this should be drafted on mediawiki.org. Here's a deep link to the submission template. We can use this task to keep track of the state of the RFC.

I created the RFC as advised: https://www.mediawiki.org/wiki/Requests_for_comment/CheckUser_requirements

Dropping this off the RFC board, as per discussion during the ArchCom meeting. There is no architectural or strategic/cross-cutting technical issue to be discussed here. Once there are concrete proposals for implementations, these can be covered by RFCs, if appropriate.

The intent behind this ticket seems to be to get resources to overhaul an old part of the software, which currently has no owner, and has been unmaintained for a while. This is indeed a problem, but it's not one that can be solved by an ArchCom RFC. It's an organizational issue, not a technical one. RFCs should come with a commitment of resources for implementation, they cannot be used to acquire resources.

I think the fact that admin tools currently have no clear owner among the WMF engineering teams is an issue that needs to be addressed by the VP of Product.