I have also observed this behaviour in various projects of mine, for instance EditGroups, which also uses the Python sseclient library to consume the stream.
My workaround is ugly: I have another script which checks how old the last event ingested in the tool's local database is, and restarts the listener if it gets too old.
It looks like we could do with a small Python library which would reliably expose WMF streams as Python iterators.

Thanks a lot @Mvolz! The new endpoint running at https://wikidata.reconci.link/ is much more stable but I assume that it is even harder for you to rely on something that is outside the WMF ecosystem (compared to a toolforge project).

Perhaps that's good enough indeed! And I might propose a patch specifically for the description of this wbsearchentities action, just to clarify that uselang should be used there.

Fantastic, thanks a lot. Please proceed with the password reset whenever you find it convenient, then!

@Tgr how about adding a single link to a page giving some information about this huge amount of "inherited" parameters? That would not make the help page less readable, would it?

Thanks, that sounds ideal!

It does use tools-redis indeed!

Yesterday the https://wdreconcile.toolforge.org/ service returned 504 errors for all URLs (after a waiting time - they did not appear "cached" as in this ticket). Restarting the service seems to temporarily mitigate the issue, so it's probably a different sort of outage. I do not commit to restarting the service when this happens - I no longer receive notifications when the service is down. If people want to volunteer to do it, I can give them access to the tool. To restart the service, a single command is needed: webservice --backend kubernetes python3.7 restart. It is very simple!

Oh I see, sorry for the misunderstanding!

It looks great! Here are a few comments:

• I assume the constraint ids don't appear there because the quality constraints extension is not installed on this wiki, but would appear otherwise?
• In the manifests that we use, we have currently added a maxlag setting (to know which maxlag= is recommended when making edits). I don't know if that is worth including in your version or if it is actually always going to be 5 in all deployments…

@Samantha_Alipio_WMDE that sounds perfect!

I realized today that I have uptime statistics for this service since I have been monitoring it for a few years (with downnotifier.com).

Lowering the priority since the outage seems to have disappeared. Looking at the #wikimedia-cloud freenode channel this was likely due to a failure to reach the Kubernetes cluster due to a DNS misconfiguration.

Probably!

Another outage today: T262553 (Toolforge-wide ticket: T262550).

It looks like the services can be reached again now, at least for me.

There is also the simpler idea of hosting the existing wrapper as an official Wikidata service (just like query.wikidata.org is an official Wikidata service, even though Blazegraph is not part of the MediaWiki instance, nor is maintained by WMDE). Comparing my tiny wrapper to Blazegraph is perhaps a bit bold (in terms of maturity), but I am just trying to explore all the ways in which Wikidata could officially offer a reconciliation service.

Thanks @Lydia_Pintscher and sorry for forgetting to tag the issue for Wikidata.

Excited to see the Wikibase team picking up this issue. I am tagging this ticket with the corresponding project, hoping that this is appropriate?

About traffic, 17.6 million reconciliation queries were processed by the Toollabs endpoint in 2019 (that excludes other calls such as preview, suggest and extension). I don't know if that counts as "significant" in comparison to what the Wikidata API is exposed to, but I agree it would deserve to live closer to Wikibase. I will try to follow up with a more detailed analysis of the usage of the service soon.

You could have a look at OpenRefine's logs:

• on windows, they are in the black terminal window that launches when you start OpenRefine
• on mac, you need to start OpenRefine differently: https://github.com/OpenRefine/OpenRefine/wiki/Installation-Instructions#obtaining-server-logs-on-mac
• on linux, they show up in the terminal where you launch OpenRefine

If you could paste here what you see in these logs, it would probably be helpful!

I am not sure! Perhaps you could ask on T243701, people might have identified patterns?

@Ciell looking at your screenshot, you have successfully logged in. OpenRefine is not making any edits because Wikidata is overloaded - that is independent from OpenRefine itself.
See T243701.

Thanks a lot @Lydia_Pintscher !

I am unable to maintain the reconciliation endpoint hosted on toolforge (affected by T257405, which I do not know how to solve).

I am unable to mitigate this, so I am migrating the project out of the toolforge infrastructure now. The new service is at https://wikidata.reconci.link/.

Thanks for investigating this @bd808. This problem is still occurring and I have no idea what I can do about it. It seems that this problem appeared when the last steps of the toolforge redirection were put in place - is there any config change on your side that could potentially be linked to that?

UWSI logs are attached (they do not contain personal information)

It is likely that this is a new behaviour (2 user reports about this over the past 12 hours).

(just removing myself as assignee as I am not expected to work on this task, I was just available to help during the event)

The appropriate redirects have been put in place.

Closing this since this does not seem to be an issue with the main client, OpenRefine. The endpoint is used by other clients but not to an extent that this should block the Toolforge migration.

After all it seems that OpenRefine actually follows HTTP redirects even for POST requests, which goes against the standards, but that is pretty lucky in this situation! So a countermeasure is not required, it seems. For users, updating the URL of the reconciliation service to the new one is still worth doing to avoid the cost of redirects (especially for auto-suggest or preview endpoints which are more time-sensitive).

The Android Commons App seems to be using it if I understand this issue correctly:
https://github.com/commons-app/apps-android-commons/issues/3614

Ah interesting! I thought the MediaWiki API could be used to do 2FA log in:
https://www.wikidata.org/w/api.php?action=help&modules=clientlogin
but if not that simplifies our life even more :)

We would be interested in supporting 2FA login directly in OpenRefine, for this we need to improve Wikidata-Toolkit first:
https://github.com/Wikidata/Wikidata-Toolkit/issues/412

@bd808's solution for this wins my vote, since it is transparent for users by preserving the existing syntax for these interwiki links. It is crucial to maintain the possibility to deep-link to tools.

Will be fully resolved once the new consumer request is accepted.

Will be fully resolved once the new consumer request is accepted.

OAuth consumer requested.

I created a new consumer request.

A new OAuth consumer must be requested according to https://wikitech.wikimedia.org/wiki/News/Toolforge.org#Oauth_considerations

A new OAuth consumer must be requested according to https://wikitech.wikimedia.org/wiki/News/Toolforge.org#Oauth_considerations

Thanks a million, this is very kind of you!

Switching on headers capitalization would be absolutely fantastic.

@valerio.bozzolan I suggest you move this comment to T249680 instead, as this ticket is about something else.

Sure, it's your call. I am sure you have more important things to do than this sort of hack.

Thank you very much for the investigation.

This seems related to T249526.

The fix for this issue might also be the cause for T249680, Wikidata-Toolkit not being able to log in anymore.

So, if we can track down which MediaWiki commit started to introduce "set-cookie" headers (instead of "Set-Cookie"), we could hopefully submit a patch to capitalize it.

The feedback was submitted, let's consider this closes the task.

The service is down at the moment because of T247501. We should consider hosting the service outside Toollabs, as we have had a range of similar issues in the past (unavailability due to devops bugs outside our control).

Concerning the choice of language to make it easier to maintain / deploy in a Wikimedia context:

• PHP seems like a pretty widespread choice, and is mandatory if the API is to be implemented as a MediaWiki extension . My understanding from our meeting with @Lokal_Profil is that it would generally be helpful to other Wikimedia organizations who are familiar with this stack, even if the service is not directly integrated in MediaWiki;
• @Mvolz told me that Node.js is also used to run services at WMF (but Parsoid is moving from Node.js to PHP-only, perhaps a sign that Node.js is not a good long-term choice).

Have you tried with a more recent version (such as 5.0 and above)? 3.3.11 is quite old and might not support INCRBYFLOAT, which is used to compute usage statistics for the endpoint.

This does not have anything to do with you indeed! I was just trying to explain that I stopped trying to help solve this issue (therefore unsubscribing from this).

I don't know - I stopped working on this task and T240369 since T240374 was declined. I don't think I can contribute to solving this problem in the current state of affairs, sorry!