Created a corresponding PR here: https://github.com/Wikidata/editgroups/pull/172

Thanks a lot @fnegri for the fix.
@Arcstur I think this new index should be added to the models with a migration. The migration could then be marked as being applied (since the index is already created).
This would help create the same index on the commons instance (this time, applying the migration for real).

Done by @DaxServer. Thanks a lot!

Let me ping them explicitly here:

I imagine some current members will want to remove themselves from this new subproject after it is created and might want to add themselves back to the existing project. Will that be possible or should we plan to create another subproject for them?

I would ask for advice on IRC, #wikimedia-gitlab on libera.chat. I'm sure there are people there who could tell you if what you're trying to do can fit in the CI offering in this gitlab instance.
As a temporary measure to get things unblocked (since not having a working CI will prevent contributions from being reviewed), we could set up a basic CI which just runs the Python tests, without building a Docker image. I imagine it would remain useful even once the Docker issues are resolved, since building the Docker image probably doesn't run any tests anyway, so it would be complementary, no?

Thanks for this fantastic bug report! Since that has to do with OpenRefine and the Wikibase extension, I think it would be more likely to be acted upon if you could file it as a GitHub issue here: https://github.com/OpenRefine/OpenRefine/issues. I am not sure if any active OpenRefine developer tracks Phabricator tickets.

Fantastic! I've added you to the org.

@Pxxlhxslxn @DaxServer I managed to add both of you to the CloudVPS project on Horizon.

Wikibase reconciliation service
Onboarding meeting for new maintainers, 2026-05-17
Participants: Paul, Dax, Antonin

Great! Should we try this Sunday 17th May, 10:00 CEST/UTC+2 (6pm Sydney time)?
I'll send an invite by email. If anyone else wants to join, let me know.

I see a couple of tasks open:

• fix the CI of the new repository
• set up deployment from the repo to the Cloud VPS instance
• change https://openrefine-wikibase.readthedocs.io/en/latest/ so that it gets deployed from this new repo (or archive those docs if people don't find them useful)
• inspect differences between the repo and https://github.com/judaicadh/wikibaseopenrefine to see if there are things worth porting
• set up some governance, to define permissions in the project, guide how people join/leave, and so on

I have filed a request to create a group on Wikimedia's Gitlab instance: T387062.

I'll follow up with @DaxServer and @Pxxlhxslxn to give them access to the deployment of the Wikidata service.
I think it would probably be better to host the repository for the source code of the service on https://gitlab.wikimedia.org/, as it would ease the involvement of the Wikimedia community for the maintenance of the service. It would be ideal to have a form of auto-deployment from the repository to the Wikidata service, I would say.

Ah, I should have thought about CORS. Thanks for the heads up, it should
be fixed now.

During the Wikibase Stakeholder Group Meeting last week, @Loz.ross expressed her openness to using the reconciliation server codebase hosted by NFDI4Culture at https://gitlab.com/nfdi4culture/openrefine-reconciliation-services/openrefine-wikibase, including the issue tracker and accepting pull/merge requests (PRs). Given that this setup is already in place, should we keep this as the reference repository?

That's a nice initiative! Given that there are multiple entities and various kinds of requests flying between them, I thought it would be clearer with a diagram (attached).
From this diagram there are four kinds of requests that can reach Wikidata when someone uses OpenRefine:

• Number 1 on the diagram (from the web frontend to MediaWiki): the User-Agent is set by the user's browser, there is no control over this as things stand (but they could be proxied through OpenRefine's server)
• Number 3 on the diagram, from OpenRefine's server to MediaWiki: the OpenRefine team has full control over what user agent they want to send there
• Number 4 and 5 on the diagram, from the reconciliation service to Wikidata (either MediaWiki or WDQS): the user-agent is set in code of the reconciliation service, which can be changed but will not have impacts on rate-limiting since the recon service is run on a Wikimedia Cloud VPS.

The repo of the code deployed is still https://github.com/wetneb/openrefine-wikibase, with deployment documented here: https://openrefine-wikibase.readthedocs.io/en/latest/install.html#deploying-in-production.

I still run the https://wikidata.reconci.link/ service. It is hosted on a personal server, serving as a load balancer which forwards the queries to https://wikidata-reconciliation.wmcloud.org/ by default, and to its own local instance of the reconciliation when https://wikidata-reconciliation.wmcloud.org/ is not reachable. The reasoning behind this set up was that https://wikidata-reconciliation.wmcloud.org/ is generally pretty fast, being hosted on a Cloud VPS server that sits close to Wikidata's production servers (both for the Wikidata actions API and the query service). But in the past, this Cloud VPS hosting regularly suffered from devops issues beyond my control, hence the introduction of this fallback server to take over when the Cloud VPS has a hiccup. This setup could be reconsidered - perhaps the Cloud VPS infrastructure is now reliable enough that this additional front node brings more instability on its own.

At the moment it seems to be functional, but I do observe long periods where no queries were processed in the past 2 days, despite the monitoring script not being able to detect that the service is unavailable:

Thanks @Tarrow for working on this!

I didn't know about the https://github.com/toolforge org, I wonder about its scope and governance. Would it make sense to add all standards committee members there, perhaps?

The Tool sweep is an initiative initiated by @Legoktm which attempted to identify license issues in toolforge tools (among other issues). I wonder if there is an easy way to get an overview of how many reviewed tools were found not to comply?

The page no longer exists, so it can't be imported anymore.

The property now has more than 3,000 uses so I think we can consider this dataset as imported.

As this is rather old, I am going to tentatively close this. If the problem persists, I would recommend opening a ticket on the bug tracker of the reconciliation service instead:
https://gitlab.com/nfdi4culture/openrefine-reconciliation-services/openrefine-wikibase/-/issues

This is already quite some years old, so I don't think it makes sense to keep this open for other tools. If more tools are affected, we can open fresh tickets for them.

I'm closing this as I don't think this coordination can or should happen. People make different formats for representing edits, which correspond to different use cases, and there's no problem with that.

I agree!

Fantastic, thank you so much!

I have replicated this issue by creating a new Wikitech account "PintochTestDeleteMe", associated it with a fresh SSH key as instructed in the tutorial and attempted the login with that test account, unsuccessfully.
I am still able to log in via my normal account ("Pintoch").

Ah sorry, I hadn't registered the difference between the two types of sessions. Friday 3pm looks ideal!

Thanks! Maybe some time on Saturday would be nice? But happy with anything, really.

I could hardly agree more, and it has indeed been suggested in T203557 shortly after the launch of this tool in 2018.

Likely caused by https://github.com/OpenRefine/OpenRefine/issues/7231.

Hello @Tarrow, I was happy to hear you on the Between the Brackets podcast, where you mentioned EditGroups, which reminded me of this ticket :)

Maybe that's a good occasion to say (in a personal capacity) that it was a pleasure working with WMSE on this. I'm also very grateful to Sunil Natraj who tackled a lot of the priority issues identified on the roadmap created by WMSE.

This is fantastic, thank you so much for offering your help! Is it okay if I follow up by email to coordinate your onboarding? We could have a quick call to walk through the codebase together, work through the process of making a small change to the tool and make sure you're able to deploy it.

I've signed the NDA a few days ago, let me know if more is needed

That's a move I have already done, and it did help with making the service more stable indeed.
The remaining issues I am aware of:

• Celery workers getting killed, probably because of memory usage, meaning that revert tasks get interrupted. I have played a bit with the concurrency and memory allocation settings to make this happen less often, but solving the root cause of the problem would be better
• A need to migrate out of EventStreams to recent changes polling (I guess?) to make it possible to deploy the tool on other wikis (for instance Wikibase.Cloud)
• Need for ongoing care for the tool to make sure it stays in sync with the wikis (if it stays out of sync for longer than what the EventStream covers, we miss edits)
• Various other issues as recorded on the issue tracker: https://github.com/Wikidata/editgroups/issues

The latest version of Django (5.1.1) only supports MariaDB 10.5 or higher, and ToolsDB is currently running 10.4.29, so that's a hurdle I encountered while keeping EditGroup's dependencies up to date. If others encounter the same issue, the latest version of Django to work with MariaDB 10.4.29 seems to be 5.0.8.
It would be great to upgrade :)

Sorry about that! I thought it could be interesting to get a better overview of the urgency of tasks in this board by trying to get a sense of how useful and relied on the mentioned tools are, but if that's not the right way of doing things, I'm happy to step back.

Quoting the Q2-Q3 usability research :

I'm tagging this as low priority given that there are alternatives such as https://tools.wmflabs.org/event-streams as mentioned above.

@waldyrious hello! I'm looking at the backlog of tools needing maintainers in https://phabricator.wikimedia.org/project/view/2952/ and found this.
I don't know what the original tool did, but given that there is since a well-established query builder at https://query.wikidata.org/, I am not sure it is still worth trying to resurrect the old tool just for the sake of providing input for UX research (but maybe it had great design ideas…).
Given that you removed your assignment, I would be inclined to close this task, no?

The OpenRefine developer in question was me ^^ (surprise surprise!)

It would be interesting to have such an API, for instance for mass uploading tools like OpenRefine. This could potentially enable us to check for close duplicates before upload.
It would be ideal if the hashing algorithm used could be sort of standardized, meaning that it could also be implemented client side. But I understand if this is too much of a stability guarantee to offer.

There will also be a meetup in the Belgrade room on Friday, 18:00-19:00:
https://wikimania.wikimedia.org/wiki/2024:Meetups/OpenRefine

Ok, I wonder if it could be linked to the fact that I tried creating the redis job earlier, back when the quota didn't allow for it? I'm not sure.

I have tried the approach described at https://wikitech.wikimedia.org/wiki/Tool:Containers#Redis_container and it works! The editgroups tool is functional again thanks to that. High five!

@Jason.nlw I believe @Gnoeee is right: you should be able to create the tags yourself as administrator of the Wikibase.Cloud instance.

@fnegri I access the logs like this:

become editgroups
kubectl get pods # find the pod id that starts with `editgroups.celery.sh`
kubectl logs editgroups.celery.sh-58496b6654-gp5pj

In the current logs there seems to be a single occurrence of this error (the others such as "Worker exited prematurely" are likely unrelated, but not sure).

Quick update on this: undoing edit batches on Wikidata via the current EditGroups has been broken for about two months now and the few hours I have spent trying to fix it have not been sufficient so far.

I can also observe this in the editgroups tool, which uses Celery:

[2024-05-22 13:39:42,965: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
  File "/data/project/editgroups/www/python/venv/lib/python3.11/site-packages/redis/connection.py", line 706, in send_packed_command
    sendall(self._sock, item)
  File "/data/project/editgroups/www/python/venv/lib/python3.11/site-packages/redis/_compat.py", line 9, in sendall
    return sock.sendall(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: [Errno 110] Connection timed out

I added some specific instructions to setup Django to use a user database:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Django_OAuth_tool#Using_MariaDB_instead_of_SQLite
Those are also documented independently here (which I improved a little):
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Python:_Django

Ok, I think I still don't understand it fully, but I trust you do and I won't stand in the way ^^

For OpenRefine(and other tools) the benefit would be that RDF ontologies can be extended very easily so that tools can define their own (namespaced) properties.

Thanks for reviving this thread I had forgotten about ^^

@Jason.nlw it is possible that the issue is on the OpenRefine side and not on the Wikibase.Cloud (not sure - I haven't checked!).

What is the preferred way to listen to the updates of a Wikibase.Cloud instance? (For instance, how is the Query Service updated?)
Currently EditGroups only supports listening to updates via the Wikimedia Event Service, which is obviously only available for Wikimedia projects and not for Wikibase.Cloud. Porting it to other Wikibase instances would likely involve adding support for updating via Recent Changes polling.
There is a ticket about this:
https://github.com/Wikidata/editgroups/issues/5

Thank you @DannyS712! Let's say this patch solves the problem :)

Yes, if it was unclear from my comments I can try to clarify again here. From an API user perspective my preferences are (from most preferred to least preferred):

1. wikibase-entityid datavalue type, for consistency with the targeted user experience (EntitySchemas being entities themselves)
2. string datavalue type, which is inconsistent with the UX but has the benefit of being an established datavalue type, which any existing API client library is bound to support already (Wikidata-Toolkit, Wikibase-SDK, pywikibot, …)
3. A new datavalue type, which will likely require some light changes in most API client libraries, and perhaps lead to failures of various severities until those changes are made (for those which have not anticipated the introduction of new datavalue types)

@Addshore wrote a blog post summarizing the options around this problem and I think it's a very worthy read:
https://addshore.com/2023/07/wikibase-and-reconciliation/