Thanks @Tarrow for working on this!

I didn't know about the https://github.com/toolforge org, I wonder about its scope and governance. Would it make sense to add all standards committee members there, perhaps?

The Tool sweep is an initiative initiated by @Legoktm which attempted to identify license issues in toolforge tools (among other issues). I wonder if there is an easy way to get an overview of how many reviewed tools were found not to comply?

The page no longer exists, so it can't be imported anymore.

The property now has more than 3,000 uses so I think we can consider this dataset as imported.

As this is rather old, I am going to tentatively close this. If the problem persists, I would recommend opening a ticket on the bug tracker of the reconciliation service instead:
https://gitlab.com/nfdi4culture/openrefine-reconciliation-services/openrefine-wikibase/-/issues

This is already quite some years old, so I don't think it makes sense to keep this open for other tools. If more tools are affected, we can open fresh tickets for them.

I'm closing this as I don't think this coordination can or should happen. People make different formats for representing edits, which correspond to different use cases, and there's no problem with that.

I agree!

Fantastic, thank you so much!

I have replicated this issue by creating a new Wikitech account "PintochTestDeleteMe", associated it with a fresh SSH key as instructed in the tutorial and attempted the login with that test account, unsuccessfully.
I am still able to log in via my normal account ("Pintoch").

Ah sorry, I hadn't registered the difference between the two types of sessions. Friday 3pm looks ideal!

Thanks! Maybe some time on Saturday would be nice? But happy with anything, really.

I could hardly agree more, and it has indeed been suggested in T203557 shortly after the launch of this tool in 2018.

Likely caused by https://github.com/OpenRefine/OpenRefine/issues/7231.

Hello @Tarrow, I was happy to hear you on the Between the Brackets podcast, where you mentioned EditGroups, which reminded me of this ticket :)

Maybe that's a good occasion to say (in a personal capacity) that it was a pleasure working with WMSE on this. I'm also very grateful to Sunil Natraj who tackled a lot of the priority issues identified on the roadmap created by WMSE.

This is fantastic, thank you so much for offering your help! Is it okay if I follow up by email to coordinate your onboarding? We could have a quick call to walk through the codebase together, work through the process of making a small change to the tool and make sure you're able to deploy it.

I've signed the NDA a few days ago, let me know if more is needed

That's a move I have already done, and it did help with making the service more stable indeed.
The remaining issues I am aware of:

• Celery workers getting killed, probably because of memory usage, meaning that revert tasks get interrupted. I have played a bit with the concurrency and memory allocation settings to make this happen less often, but solving the root cause of the problem would be better
• A need to migrate out of EventStreams to recent changes polling (I guess?) to make it possible to deploy the tool on other wikis (for instance Wikibase.Cloud)
• Need for ongoing care for the tool to make sure it stays in sync with the wikis (if it stays out of sync for longer than what the EventStream covers, we miss edits)
• Various other issues as recorded on the issue tracker: https://github.com/Wikidata/editgroups/issues

The latest version of Django (5.1.1) only supports MariaDB 10.5 or higher, and ToolsDB is currently running 10.4.29, so that's a hurdle I encountered while keeping EditGroup's dependencies up to date. If others encounter the same issue, the latest version of Django to work with MariaDB 10.4.29 seems to be 5.0.8.
It would be great to upgrade :)

Sorry about that! I thought it could be interesting to get a better overview of the urgency of tasks in this board by trying to get a sense of how useful and relied on the mentioned tools are, but if that's not the right way of doing things, I'm happy to step back.

Quoting the Q2-Q3 usability research :

I'm tagging this as low priority given that there are alternatives such as https://tools.wmflabs.org/event-streams as mentioned above.

@waldyrious hello! I'm looking at the backlog of tools needing maintainers in https://phabricator.wikimedia.org/project/view/2952/ and found this.
I don't know what the original tool did, but given that there is since a well-established query builder at https://query.wikidata.org/, I am not sure it is still worth trying to resurrect the old tool just for the sake of providing input for UX research (but maybe it had great design ideas…).
Given that you removed your assignment, I would be inclined to close this task, no?

The OpenRefine developer in question was me ^^ (surprise surprise!)

It would be interesting to have such an API, for instance for mass uploading tools like OpenRefine. This could potentially enable us to check for close duplicates before upload.
It would be ideal if the hashing algorithm used could be sort of standardized, meaning that it could also be implemented client side. But I understand if this is too much of a stability guarantee to offer.

There will also be a meetup in the Belgrade room on Friday, 18:00-19:00:
https://wikimania.wikimedia.org/wiki/2024:Meetups/OpenRefine

Ok, I wonder if it could be linked to the fact that I tried creating the redis job earlier, back when the quota didn't allow for it? I'm not sure.

I have tried the approach described at https://wikitech.wikimedia.org/wiki/Tool:Containers#Redis_container and it works! The editgroups tool is functional again thanks to that. High five!

@Jason.nlw I believe @Gnoeee is right: you should be able to create the tags yourself as administrator of the Wikibase.Cloud instance.

@fnegri I access the logs like this:

become editgroups
kubectl get pods # find the pod id that starts with `editgroups.celery.sh`
kubectl logs editgroups.celery.sh-58496b6654-gp5pj

In the current logs there seems to be a single occurrence of this error (the others such as "Worker exited prematurely" are likely unrelated, but not sure).

Quick update on this: undoing edit batches on Wikidata via the current EditGroups has been broken for about two months now and the few hours I have spent trying to fix it have not been sufficient so far.

I can also observe this in the editgroups tool, which uses Celery:

[2024-05-22 13:39:42,965: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
  File "/data/project/editgroups/www/python/venv/lib/python3.11/site-packages/redis/connection.py", line 706, in send_packed_command
    sendall(self._sock, item)
  File "/data/project/editgroups/www/python/venv/lib/python3.11/site-packages/redis/_compat.py", line 9, in sendall
    return sock.sendall(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: [Errno 110] Connection timed out

I added some specific instructions to setup Django to use a user database:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Django_OAuth_tool#Using_MariaDB_instead_of_SQLite
Those are also documented independently here (which I improved a little):
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Python:_Django

Ok, I think I still don't understand it fully, but I trust you do and I won't stand in the way ^^

For OpenRefine(and other tools) the benefit would be that RDF ontologies can be extended very easily so that tools can define their own (namespaced) properties.

Thanks for reviving this thread I had forgotten about ^^

@Jason.nlw it is possible that the issue is on the OpenRefine side and not on the Wikibase.Cloud (not sure - I haven't checked!).

What is the preferred way to listen to the updates of a Wikibase.Cloud instance? (For instance, how is the Query Service updated?)
Currently EditGroups only supports listening to updates via the Wikimedia Event Service, which is obviously only available for Wikimedia projects and not for Wikibase.Cloud. Porting it to other Wikibase instances would likely involve adding support for updating via Recent Changes polling.
There is a ticket about this:
https://github.com/Wikidata/editgroups/issues/5

Thank you @DannyS712! Let's say this patch solves the problem :)

Yes, if it was unclear from my comments I can try to clarify again here. From an API user perspective my preferences are (from most preferred to least preferred):

1. wikibase-entityid datavalue type, for consistency with the targeted user experience (EntitySchemas being entities themselves)
2. string datavalue type, which is inconsistent with the UX but has the benefit of being an established datavalue type, which any existing API client library is bound to support already (Wikidata-Toolkit, Wikibase-SDK, pywikibot, …)
3. A new datavalue type, which will likely require some light changes in most API client libraries, and perhaps lead to failures of various severities until those changes are made (for those which have not anticipated the introduction of new datavalue types)

@Addshore wrote a blog post summarizing the options around this problem and I think it's a very worthy read:
https://addshore.com/2023/07/wikibase-and-reconciliation/

It's not just useful for testing purposes. For applications like OpenRefine, which normally run on the user's machine directly and are not meant to be hosted, it is important that the callback can be a localhost URL, therefore using HTTP. OpenRefine itself runs as a locally hosted web app (typically at http://localhost:3333/).

If it's not too difficult, it would be helpful to have that behavior in list=recentchanges too, typically for tools which do recent changes polling.
(EditGroups sadly uses the EventStreams service and not RC polling directly - not sure if changing list=recentchanges would also impact the EventStreams as well?)

Thank you both, this should simplify the EditGroups tool quite a bit! Here is an example of page where I had to render entity links manually, with some Javascript post-processing code:
https://editgroups.toolforge.org/b/CB/3683873dde8d/. (I guess I will keep this logic for a while, for all the revisions fetched already, but it could be dropped in some years)

Thanks for the heads up, I have opened an issue about it on OpenRefine's side: https://github.com/OpenRefine/OpenRefine/issues/5658

Agreed, I would say PAWS replaces this task.

Here is the current status of this issue:

A feature request came in: handling changes of username on the wiki. I suspect this is a feature that would likely come "for free" in any reimplementation of the current tool as a MediaWiki extension, because it would rely on the existing SQL tables in MediaWiki to represent users.

This works fine when using OpenRefine locally, so it probably has something to do with the deployment indeed. I have not checked, but I suspect the frontend code redirects the user to the project page using some relative URL, which fails in the PAWS context.
I do not know if this can easily be done, but I would assume one radical way to ensure that this does not happen would be to host OpenRefine at the root URL of some automatically-generated domain (instead of using a subpath).
If there are sensible fixes that can be applied on OpenRefine's side to make this easier, we can totally consider them.

hello all, it's just caused by the change of format for our download URL. I have added a suggested change to the pull request that should hopefully fix it:
https://github.com/toolforge/paws/pull/187#pullrequestreview-1074315167

Sounds good! I think @Spinster also intends to write a few follow-up tickets.

Should we close this as done?

This bug was mentioned today at LSWT'22 by @SebastianHellmann as a blocker for integrating Wikidata in linked data applications.

Thanks to @matthiasmullie we now know that this API response cannot return the pageid, because it is not known when the API response is sent back. The pageid is allocated asynchronously afterwards.

However, wouldn't it be even simpler for the user to not even need the page id, and be able to add claims with a Commons file name?

One helpful step in that direction would be to return the page id in the JSON response of a successful file upload. That should be fairly straightforward and would bring the number of requests required from three to two.

I have not checked but I trust you!