I drafted a patch for this, but won't have enough time to clean it up .. it works fine now, but the code is not the prettiest. Specifically, it might require some clean-up around where TermCollisionDetector instances come from, as for these two special pages we are getting them directly from WikibaseRepo (where it is determined whether we are using new store or not) but inside TermsValidatorFactory it is done using another layer of indirection (with ByIdFingerprintUniquenessValidator) and so on.. but probably it is better to do such clean up when migration is done and temporary stuff are removed.

Oh, apparently SpecialNewEntity creates entities without even creating change ops. That would explain it, as then it will by-pass the whole validation of fignreprint uniqueness that is only executed inside ChangeOpFingerprintResult::validate(). Guess that needs to change too.

Same goes for items. API calls fail manually as expected, but I can create conflicting items through special pages. I also confirm that SingleEntitySourceServices::getTermSearchInteractorFactory and SingleEntitySourceServices::getPrefetchingTermLookup need fixing. Will be working on that this evening.

I added two test cases to the integration test editentity fingerprint uniqueness. They pass for API calls, which I could also confirm sending API requests manually.

In T232040#5774510, @Addshore wrote:

While reviewing https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/553524/ I noticed that both at least 2 more things need fixing:

• SingleEntitySourceServices::getTermSearchInteractorFactory
• SingleEntitySourceServices::getPrefetchingTermLookup

Until that happens its hard to really verify the final patch etc.

If reviewing the patch in the current state is too much effort or difficult (it can be very much the case), also let me know I can try to split it up further into couple of smaller patches

@Addshore @Ladsgroup
Are we waiting on this until we start reading a range (first 1000 items iirc) from new store? Is there more code review to be done?

Not sure @Ladsgroup your comments in the review do make sense.. can you please double check them and read through the code again? I may of course have misunderstood them, maybe little more elaboration will help

Last patch turning on validation of uniqueness in new store is ready for review now
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/553524
It has 3 little fixes below it too

In T240321#5727257, @Ammarpad wrote:

Added. It is not really difficult :). I believe you can even do so with your employee account if you wanted to.

In T232595#5635743, @Addshore wrote:

I guess this is still just in beta?

what kind of impact the solving of the issue will generate and why?

who/what is suffering from it ?

problem one

• developers productivity not catching up with modern javascript, making it become slower with time as the projects expands.
• on boarding young developers increase in cost too as they need to get to learn and work with semi-outdated and limitef APIs and standardbof JS.

what the current problem(s) are ?

problem one
we cannot use es5/6 features that are not supported by browsers yet and cannot be polyfilled in runtime (like URI api. ref: T117651: Align mediawiki.Uri more with the native URL constructor and <the one with browser.URI>)

@WMDE-leszek that's great! I remember seeing it differently locally when I tested adding the id back a while back, or maybe I just assumed it will land on the a tag from the code (that seems to do some magic there). Either way, I should've waited and checked on beta :)

@Ladsgroup yeah probably.. There might still some that might break because the id now is on the a tag and not a wrapper element like it used to be.

yeah we need some column for such cases.

Thanks @waldyrious very much for your contribution on the spot!

Moved back to be briefly discussed, estimated and broken down.

Wow in minutes .. it took a query on wb_terms >5 minutes to get 50 results only. Same on Query Service. How did you find those?

oh yeah sure that won't be running in production like that .. I just was wondering if there are any extra optimization here that I could've missed re using indexes or using the sub-queries.

yes I did. I should add that the only teats failing for me now are WikibaseLexeme ones, specifically mwbot logins. There is an invalid edit token (+\\ feels truncated or smth) being stored and reused by the test sometimes, causing Invalid CSRF Token error.