FYI, OpenRefine will likely implement a SPARQL importer in the upcoming time (May-August 2022) through an Outreachy internship. Many OpenRefine users have requested to make it possible to start OpenRefine projects from a SPARQL query.

We now have documentation, including a short tutorial video.

The meetup took place, with more than 40 attendees! \o/

This task is completed! \o/ Thank you!

Yay, this is done! \o/ Thank you!

File captions on Commons are not properties. They are technically the equivalent of Labels on Wikidata.

Tested, and this works! Yay! Thank you :-)

I think we covered all datatypes! Closing this ticket as resolved. Thank you so much @Eugene233 :-D

Tested, and this works!! Thank you \o/

Tested, and this works! Thank you 😎

This problem showed up 'live' in our community meetup / demo yesterday as well. I've used the reconciliation service more often now, and I do notice this tends to happen with larger datasets and when larger amounts of data (often, but not always, multiple columns at once) are retrieved via data extension.

This issue was moved to the Commons Extension repository - this feature will be developed as part of the Commons Extension in OpenRefine: https://github.com/OpenRefine/CommonsExtension/issues/3

Downnotifier is set up (with my private email address, so that I can check it easily).

ISA is now also visible in Hay's Directory, with some help from Hay Himself :-) https://hay.toolforge.org/directory/#/search/campaigns

We will work on this issue, it's important and super handy! However, the code that addresses this will not be part of the reconciliation service, but will be made available in OpenRefine's Commons Extension.

As good progress is being made, we can include this task in the current sprint (Jan 24-Feb 11).

Small fix for whenever this is convenient to work on :-)

As discussed in our last team meeting (Jan 25): we will upload new files to Beta Commons while testing OpenRefine's upload functionality, so uploading a batch of files there now is not needed.

To finish/test during current sprint (Jan 24-Feb 11).

I think this task is done! Unless further testing is needed?

To finish/test during current sprint (Jan 24-Feb 11).

To finish/test during current sprint (Jan 24-Feb 11).

Just two more datatypes to enable in the current (Jan 24-Feb 11) sprint:

Last fixes (URL encoded file paths and bare M-ids, see comment above) to be finished during current sprint :-)

Is https://www.downnotifier.com a trusted/good service? I see that it has a premium plan for $14.95 per year. Should we research alternatives too? If it helps, I can pay for such a premium service out of our budget. Let me know if you think that would be helpful @Eugene233 @Pintoch

Let's indeed set up a downtime notifier during this sprint (Jan 24-Feb 11) so that we can track the reliability of the reconciliation service.

Info about the meetup at https://commons.wikimedia.org/wiki/Commons:OpenRefine/Community_meetup_22_February_2022 👈

Can the dialog also suggest things that are not strictly properties (i.e. Wikitext, captions..)?

And I found another dataset with similar behavior, in this case the Creator (P170) property not producing proper data extension for each row (while all 100 files in this dataset do have a P170 statement):

We're looking at Tuesday, February 22, 16:00 till 18:00 CET. More info to come!

Yay, great progress on this task!! Thank you :-)

I have added a selection of files that should cover all supported file types on Wikimedia Commons. In addition, I chose files that will be interesting for testing the preview function per T292526: Add preview service to reconciliation API and inserted a variety of formats of the file path itself so that we can stress-test T290088: The Structured Data on Commons reconciliation service recognizes the most widely used Commons file name formats.

I have collected examples of files pointing to all relevant datatypes in their SDC.

I'm collecting a diverse (and hopefully exhaustive) set of example files for testing in T299135: Assemble various sets of interesting Commons files for testing SDC features in OpenRefine.

I'm collecting files in this spreadsheet for now: https://docs.google.com/spreadsheets/d/1SIqjRdDh7G4KRBi47kz5WV3Ejl6YISiUrq3GhERnBSU/edit#gid=0

I'll take care of organizing this and hosting most of it! Will probably be a session in the last week of February or first week of March 2022, starting at 16.00 CET.

Interviews are finished! 🥳 It was very enlightening; many thanks to everyone who generously provided time and valuable input.

I just tested with this dataset again, and it's still the same weird behavior ... :(

I'll (finally) make a start during this month's sprint. Will also be useful for our midpoint report.

Something for the current sprint (January 2021) :-)

Let's finish current (basic) work on this task for now - more work needs to be done later, but we can stall this after merging the current patch.

In T296059#7517503, @GFontenelle_WMF wrote:

@Spinster, I asked around the Structured Data team and their suggestion was to use Beta, as it is also the one they use. As long as you're only uploading files and not changing any code, it should be fine.

However, there's also test Commons and, apparently, was supposed to be turned off in 2019 and it hasn't so far. So even though there's a risk there, we don't think it's that big if you really wanna use it.

(Answering with my WMF account, not @GiFontenelle, but I think it's fine! 😅).

We think this task is done \o/ !! (Feel free to reopen if there are still issues...)

In T298183#7591180, @Majavah wrote:

In T298183#7591172, @Spinster wrote:

When I load OpenRefine on PAWS, it still asks to upgrade to 3.5.1 so I have the impression that this task is not resolved yet.

You need to restart your server for the updated singleuser image to take effect.

When I load OpenRefine on PAWS, it still asks to upgrade to 3.5.1 so I have the impression that this task is not resolved yet.

A few example files to play around with:

In T222291#7569368, @NavinoEvans wrote:

That's amazing news! Thanks so much for all the help @abi_, you're a star :)