I forgot to add the ANALYZE results of the proposed query (without inner limit):

I did a bit of testing on the proposed query changes (the first row in the table above), and it does perform significantly better.

I don't see a reason to disable the autocomplete on that field.

The schema in T245499#5907670 could be simplified a bit more by removing cua_ip (and use cua_ip_hex instead). I can't see a reason why that would be needed?

In T245499#5908431, @Tchanders wrote:

Only solution that gets all the data. (Data may be paginated, but it will all be there.)

In T237299#5907995, @Niharika wrote:

@Tchanders To confirm - sticky highlights persist across paginated tabs right?

Perhaps an even better way to do this is to create a new cu_changes table instead of altering the existing one, something like:

CREATE TABLE /*_*/cu_revision (
  -- CheckUser Actor ID
  cua_id BIGINT NOT NULL,

Another proposal is to create another table, something like this:

CREATE TABLE /*_*/cu_actor (
  -- Primary key
  cua_id bigint unsigned NOT NULL PRIMARY KEY AUTO_INCREMENT

I kind of like the idea of using the existing logging table because then as the user adds additional targets during the course of their investigation, those targets can be added as individual log entries, rather than modifying the existing log entry.

@Tchanders pointed out that there is an existing "type" field on the cu_log table:

-- String indicating the type of query, may be:
-- "useredits", "userips", "ipedits", "ipusers", "ipedits-xff", "ipusers-xff"
cul_type varbinary(30) not null,

In T245662#5898816, @aezell wrote:

Maybe we could "tag" the logs from the new tool and keep them in the same DB, he said without looking at the schema.

In T245662#5898781, @Niharika wrote:

There will be a new special page for the log, associated with Special:Investigate so we can keep track of the lookups happening through the new page.

@Niharika will this be a separate log or will we use the existing log?

In T245499#5897637, @Tchanders wrote:

As mentioned in the task description, indexing doesn't solve the problem of setting a limit on the number of rows being accessed.

We should probably keep in mind the schema changes that are happening in T233004. Not necessarily a blocker though.

I ran the query above on Wikidata with my two user accounts, my home IP address, and an 8 IP range around my IP address

7 rows in set (4 min 16.03 sec)

clearly what we have isn't going to work.

In T244157#5892968, @dbarratt wrote:

In T244157#5892494, @Tchanders wrote:

If we were certain that timestamp always increased with ID, we could even paginate on the single ipb_id column. It seems this is not the case (see DatabaseBlock::updateTimestamp), so we can paginate on timestamp and ID together.

Yeah the timestamp is non-unique as well, so you could (in theory I suppose) have two different blocks with the same timestamp.

In T244157#5892494, @Tchanders wrote:

If we were certain that timestamp always increased with ID, we could even paginate on the single ipb_id column. It seems this is not the case (see DatabaseBlock::updateTimestamp), so we can paginate on timestamp and ID together.

In T245181#5892530, @Tchanders wrote:

@dbarratt We should discuss that as a team. Does it impact on whether we do this task though?

In T244579#5884024, @Mooeypoo wrote:

explore the method that does not involve using OFFSET with GROUP BY for this product.

A long time ago (perhaps too long ago!) we talked about making code coverage an implied part of Anti-Harassment tasks acceptance criteria, did we abandon that?

@Catrope That totally makes sense to me, it would be great if we picked a direction either way.

In T245167#5881935, @Jdforrester-WMF wrote:

That'd be fine by me if that label works for you; it's a good name for it.

@Tchanders now that T244492 has been resolved I imagine this can be done?

@Qwertysdf There are a lot of different ways in which you might want to setup MediaWiki using the docker image. Forcing a specific setup by specifying the VOLUMES might be helpful to some use-cases, but is a detriment to others. For instance, I personally might mount the entire /var/www/html directory for development as a volume, but on my server, I might create a new image that extends this one and copies the extensions I want at build time and does not mount anything. In either instance, I don't want the VOLUMES being declared/forced. If docker provided a way to remove the volumes, I think it would be fine to define the most popular use case in the Dockerfile, but that isn't the case. If you want the VOLUMES to be declared, I would suggest extending the image with the FROM directive.

In T242945#5846144, @Niharika wrote:

@dbarratt We had a conversation about this in a team meeting but I don't remember the outcome - will the user be able to run multiple investigations at once or will they not? I am perfectly happy either way but that will influence my choice of the mock we go with.

In T244579#5879638, @tstarling wrote:

I see T244579 is done: the change is merged so it presumably should be closed. Does that answer all use cases in T236225, or is there something remaining that still needs OFFSET?

Perhaps move the existing tasks to a new tag like Docker-Hub-MediaWiki since it's hosted at https://hub.docker.com/_/mediawiki ?

In T239680#5880251, @dom_walden wrote:

@dbarratt (After cherry picking your latest patch) On the preliminary check table, when I attempt to go to a previous or next page I get:

The style fixes in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CheckUser/+/571794 should fix this problem (I think).

I can wrap the elements being added to the content panel in a TabPanelLayout but in my testing that didn't make a difference to the spacing (but maybe I did something wrong).

In T240366 I was forced to bypass the IndexLayout::addTabPanels() method because it is hard-coded to use a <span> instead of an <a> and from my testing the label would not allow an unescaped string of HTML.

and @Catrope and @Jdforrester-WMF !

Boldly closing as it appears to have been merged... thanks @Tchanders !

Updated example (removed the iv): https://jwt.io/?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1ODE0NzczMjgsImRhdGEiOiJ1NFFpaWdubkM1NTVCUjdNRE42WEtjNzZVVlp6cWt2WE8veXZHQTMvNi9EVGlBPT0ifQ.MjLKVAWEffgQ1qm4IFGnQOvp705hXDjZt2mYsXX32vk

In T239680#5867028, @sbassett wrote:

@dbarratt - One comment for you on the patch. Sorry I don't have any good suggestions :/ Also, I'm curious how large the data from these CU queries can get on-wiki. Playing around with str_repeat() a bit, I can pretty quickly generate some massive base64 strings with both aes-256-ctr and aes-256-cbc, which then gets base64-encoded again as part of the token, I believe. I'm not sure if all web browsers will be fine with passing potentially massive strings around as a query parameter. Have you noticed any issues during your testing?

Here's an example with the changes:
https://jwt.io/?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1ODE0NTEwOTgsIml2IjoiSXFjNU1HQ051Q1Z2cHlcL1B0dlwvVDNRPT0iLCJkYXRhIjoiOHk1bWk4Qm94MFJXMzZ3M2ZcLzhVMlV1T3dKdCtXSjFFb1hzQ21KWWRRS1ZiT1E9PSJ9.w0SgWkClfhAdMBvLKb7jhpvDPk_kqFMZDAXfLDy0fbI

@sbassett FYI we've made some improvements to the way the token is generated https://gerrit.wikimedia.org/r/571337

This error only occurs if CentralAuth is disabled.

From the Vue docs

When using in-DOM templates or in-JavaScript template strings, the template-to-render-function compilation is performed on the fly. This is usually fast enough in most cases, but is best avoided if your application is performance-sensitive.

In T241180#5827364, @egardner wrote:

@dbarratt regarding need for a build step: there's some discussion of this in the original RFC; it's true that React can be used sans build-step with the CreateElement API.

In T241180#5821811, @egardner wrote:

• It will be much easier to integrate Vue into the Mediawiki tech stack as it presently exists because it doesn't require a build step.

In T241180#5812363, @Addshore wrote:

I don't believe anyone has said any of these frameworks are the "best in the world", so this analogy probably doesn't make sense.

In T241180#5812071, @Ladsgroup wrote:

[citation needed]
(An example, yahoo being the most popular search engine back in early days)

In T241180#5805149, @Tonina_Zhelyazkova_WMDE wrote:

Indeed, we can cite companies using either framework all day. I assume the underlying point of these examples is that React is more popular than Vue. And yes this is a fact. But is being most popular an argument in favor of choosing React for MW? If yes, why? What does the most popular framework has that Vue doesn't provide?

I've updated the JWT's payload, which now looks like this:

{
  "iss": "local",
  "sub": "Jim Gordon",
  "exp": 1579233518,
  "data": "u4QjmB7yHcgwfAbdQf+dMImkUTA4jQL9dMmkBQey2K8="
}

In T242793#5803402, @Krenair wrote:

Are we okay with the idea of stuff running in labs accessing data from private wikis via OAuth?

@Tonina_Zhelyazkova_WMDE my point is that it doesn't mitigate uncertainty, the BDFL could change the direction of Vue just as easily as a public corporation could (I would argue even easier because he has little risk involved in changing, where Facebook has a lot of risk). The reality is, is that the point being made (about mitigating uncertainty) is a business decision, not a technical one, and I don't think it takes all of the factors into account.

I forgot to mention that both WordPress (as of version 5.0) and Drupal (future) use React. I think it might be wise to consult with these two projects and see why they chose React over Vue.

Overall I think this is awesome. I think it's an overdue change. 👏 Thanks for all of your hard work. 😃

In T239680#5799744, @Niharika wrote:

@dbarratt Can you elaborate a bit on what the user-facing impact of such an expiry would be?

errrrr... to further clarify, we like #2 as a solution over #3 because it allows us a place to put things like filters that are not necessarily in the log itself, but could reveal information. For instance, if a CheckUser filters by a User Agent, on its own, that's not PII, but if you knew what user they were investigating at the time of the request, you could pretty easily put two and two together. These filters aren't currently recorded in the CheckUserLog. However, again, if someone has checkuser permission, they could repeat the investigation and they would have access to all of the data that was encrypted in the token in the first place.