Page MenuHomePhabricator

Deploy the Commons recon service on a test instance of Commons
Closed, ResolvedPublic

Description

To test our editing features developed in the OpenRefine-SDC project, it would be useful to use a test instance of Commons rather than directly make edits to the official Wikimedia Commons instance.

@Spinster, do you know which one you should use? If I remember correctly, there are multiple ones around. For instance I found https://commons.wikimedia.beta.wmflabs.org/wiki/Main_Page but I am not sure this one is appropriate for us to do test uploads.

Once we know which instance we should use, could @Eugene233 deploy an instance of the reconciliation service for that Commons instance? (For instance with a dedicated Toolforge project).

Event Timeline

There are indeed various 'test Commonses' and I'm also a bit confused about them.

There's https://test-commons.wikimedia.org/wiki/Main_Page which says it would be deleted in 2019 ;-) but there are still recent uploads. Files have structured data but no preview thumbnails.

There's also https://commons.wikimedia.beta.wmflabs.org/wiki/Main_Page (the one you point to). It also has structured data. According to an old alert I just saw, and the GWtoolset documentation, this was the test instance for the GLAMwiki Toolset. I see no file preview thumbnails here either but I guess this is not necessary for us.

image.png (104×506 px, 21 KB)

I see that @Ladsgroup did some recent uploads on both so perhaps he knows more, or @GiFontenelle could perhaps ask for us with the StructuredDataOnCommons team?

Since I'm pinged :)

Depends on what you exactly want. I try to describe both test instances and their differences so it might be helpful for you to choose. The beta commons is in the cloud while test commons is in production. So access to the data or config changes are a bit more stricter in test commons. Giving admin rights is not that strict though in both. The test commons uses basically the same code as production while beta cluster is on master which means it's pretty unstable and can break much more often (also due to being in beta cluster and beta cluster being underloved: T215217). Beta cluster also has less capacity, I'm not sure how much you want to upload but even for test commons, you might bring down actual production wikis if you're not careful (beta cluster goes down a lot easier but at least it doesn't bring down actual wikis with it) and I'm sure beta cluster's swift storage can get filled up really quickly.

HTH

@Ladsgroup thank you so much, that is massively helpful!

I do not think storage will be an issue for us: we will need to upload a lot of data in our testing phase. For the next few months we will only be editing wikitext or structured data without uploading new files via the tool. When we get to the part where we upload new files, we can be careful about sending small batches of small files.

So I would say let us go for test commons, https://test-commons.wikimedia.org/wiki/Main_Page, perhaps? I really cannot imagine how our usage would bring anything down. :) Admin access would be nice, but we should also be able to do without initially.

@Spinster, I asked around the Structured Data team and their suggestion was to use Beta, as it is also the one they use. As long as you're only uploading files and not changing any code, it should be fine.

However, there's also test Commons and, apparently, was supposed to be turned off in 2019 and it hasn't so far. So even though there's a risk there, we don't think it's that big if you really wanna use it.

(Answering with my WMF account, not @GiFontenelle, but I think it's fine! 😅).

Whatever you like to use, give me the user and I make it an admin.

@Spinster, I asked around the Structured Data team and their suggestion was to use Beta, as it is also the one they use. As long as you're only uploading files and not changing any code, it should be fine.

However, there's also test Commons and, apparently, was supposed to be turned off in 2019 and it hasn't so far. So even though there's a risk there, we don't think it's that big if you really wanna use it.

(Answering with my WMF account, not @GiFontenelle, but I think it's fine! 😅).

Thanks both for your input! We want to go for the service that is also used by the Structured Data team, namely https://commons.wikimedia.beta.wmflabs.org/wiki/Main_Page - also because it seems to better support structured data with federation to (a) beta Wikidata. I have uploaded a test file https://commons.wikimedia.beta.wmflabs.org/wiki/File:Otto_Puks.jpg which looks very usable. @Ladsgroup or someone else, could you perhaps give me (User:Spinster) admin rights there?

Spinster triaged this task as High priority.Jan 5 2022, 3:54 PM

Test service with chosen test commons instance created. Could be thoroughly tested using base url

Sample tested example with file https://commons.wikimedia.beta.wmflabs.org/wiki/File:Otto_Puks.jpg could be accessed using this link

I think this task is done! Unless further testing is needed?