@Lydia_Pintscher This was requested during the Vienna Hackathon.

The problem seems no longer reproducible, so I guess the module(s) in question have been fixed.

In T64925#3346908, @Ladsgroup wrote:

This is indeed resolved: https://www.wikidata.org/w/api.php?action=wbparsevalue&datatype=string&values=%1Ffoo|bar%1Fbaaar|foo0

My suggestion is, once the fix for not introducing new duplicates (https://gerrit.wikimedia.org/r/354550) is in, to list all duplicate row ids in a text file (generated via a query similar to the one I already mentioned). Then we can delete these in batches using a maintenance script reading from the file of term_row_id to delete.

All placeholders up to (including) https://cy.wikipedia.org/wiki/Arbennig:Am_y_Pwnc/Q16956 are now indexable!

Thinking about this more, we could provide a mw.wikibase.getUserLanguageSitelink( entityId, 'wikipedia' )/mw.wikibase.entity:getUserLanguageSitelink( entityId, 'wikipedia' ) which would return the sitelink on the Wikipedia which has the user's language or nil if there's none. Implementing a fallback here for finding other potentially relevant Wikipedias would be possible (based on Language fallbacks), but I'm not sure that's very useful.

I ran the numbers briefly and both kowiki and trwiki don't look as interesting to me. Therefore I would suggest going with elwiki, which has the following usages currently:

Changes by @simon04 applied to meta: https://meta.wikimedia.org/w/index.php?diff=16792401&oldid=15240696.

+1 Definitively

Workaround is in place, now we can just wait for someone to fix this in HHVM.

When discussing this with @Fnielsen he mentioned that there might be Items which are very interesting for the certain audience of a Wikipedia (like Danish paintings might be very interesting to readers of the Danish Wikipedia). But these Items often fail to meet our notability criteria, as they are mostly interesting for a certain audience.

We plan to deploy this on Wednesday May 31.

According to @Smalyshev the service has recently been reloaded:

SELECT (COUNT(*) as ?count) WHERE {
  ?a wikibase:statements ?b
}

How is the Lua module supposed to estimate the size of the box? Is that even feasible?

After the next Wikidata deploy, you should be able to see what exactly is causing the problems in question so that it should be easier to locate the problem.

I don't like to have stable links with BETA in them (like https://dumps.wikimedia.org/wikidatawiki/entities/latest-all-BETA.ttl.bz2), thus I would prefer to add https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2.

We don't have a hard limit in the suggester, but we have a probability cut-off. So if an item has a lot of statements already, we might just not find any further relevant properties.

I don't think we should add this before we declare the RDF dumps stable, thus I added T112127: [Story] Move RDF ontology from beta to release status as a subtask/ dependency for this.

With the current state, we still have the same amount of connections to the master DBs, but we don't use GET_LOCK etc. on them anymore. The queries are unchanged also (despite the GET_LOCK etc.).

2fa has been removed, you should be able to log in by just using your user name + password again.

In T163815#3238627, @Lydia_Pintscher wrote:

@hoo is there anything we can do here?

In T160506#3231024, @Lucas_Werkmeister_WMDE wrote:

Since the API and user script announcement, several constraints have been fixed, such as this one which used to report two violations on every date of birth in Wikidata. Can we do another reload? We should also announce it to the community so that they know they should get in any necessary changes before the date in question (say, next Wednesday, 2017-05-10).

Will be fixed with the next deploy.

Any updates here? For us it's good enough if we can manually update the vendor "build" when needed, we don't need any automatism here that reacts upon Wikibase wmf-branch changes.

Works in production (on test.wikidata), will go live on Wikidata later today.

In T163770#3208914, @TTO wrote:

Those block changes were to the local block, though, weren't they?

In T163748#3208792, @thiemowmde wrote:

The semantics of Title::isDeleted seem to be utterly broken. It does not say if a page is currently deleted, but if there was ever a revision deleted, and how much. Can we please deprecate this shit and rename it to hasDeletedRevisons or something?

Wikidata confirmed: https://www.wikidata.org/w/index.php?diff=478497603

What I meant is that: You can't turn an Item into a redirect if it has one or more deleted revisions.

Probably at least 10% of the table are duplicates (at least 129M/2).

Number of rows which occur at least twice (excludes aliases, for performance reasons):

Both problems have been addressed.

In T163144#3205058, @Lydia_Pintscher wrote:

It likely has not been deployed yet. Thanks for checking.
@hoo when will it be deployed?

In T101253#3204508, @Lydia_Pintscher wrote:

Is this still relevant?

In T110528#3204506, @Lydia_Pintscher wrote:

Is this still relevant?

I'm looking into this and will see what we can do.

The first truthy nt dump can be found at https://dumps.wikimedia.org/wikidatawiki/entities/20170418/. New truthy nt dumps will appear weekly on https://dumps.wikimedia.org/wikidatawiki/entities/ at about the same time (mid to late Wednesday UTC).

Wikidata specific steps taken and confirmed: https://www.wikidata.org/w/index.php?diff=476527413.

@Reedy We have our "dependency template" in the wikidata/build-resources repo. It's mostly just straight-forward pulling things in via composer, but we also do some evil-ish magic in build/tasks/updatecomposer.js.

In T105398#3193023, @Marostegui wrote:

Is this still needed?

In T151717#3184079, @jcrespo wrote:

We want to collect additional information on one of these wikis for a while

If that doesn't involve a schema change, sure.

The first truthy nt dump should appear next Tuesday (probably late UTC).