In T279637#7906343, @MatthewVernon wrote:

Broadly, there should be little impact (and our monitoring suggests error rates within expected ranges); I hope any errors were infrequent and transient.

Unfortunately, a number of the reimages are not proceeding smoothly, which I think is more likely to produce upticks in errors.

Is this upgrade supposed to be without impact? Does it impact the performance of the Swift cluster? I've got quite a few of these for the last couple of days:

In T270911#7900654, @Bawolff wrote:

Not saying this to be mean - I think everyone involved really did try there best, and communication failures, especially on the WMF side, set things up for failure. However, i think a lot of interesting lessons could be learned from GWToolset if anyone ever wants to try again with something similar. Which would be cool. So far it seems like WM-DE (unless i am misinformed) is really the only chapter that has successfully collaborated on mediawiki development. I think Wikimedia would be a lot healthier if more groups participated on the technical side.

Structured data is supposed to be the replacement of wikitext editing and the future. It's very broken for months now. Is nobody responsible for Commons anymore these days?

Thanks for implementing this. I finally got around to update https://www.wikidata.org/wiki/Module:Constraints so that pages like https://www.wikidata.org/wiki/Property_talk:P650 are not throwing a horrible alert anymore. I also filed T303670 to make the same constraint for descriptions.

In T301048#7690148, @Cparle wrote:

@Multichill is the bot just using wbsetclaim then a null edit? Are you getting this with any of your other bots?

Got a ping from @Aklapper about having tasks assigned to my name with no activity for a long time. This task doesn't ring a bell.

In T301048#7686118, @LucasWerkmeister wrote:

I was able to reproduce this on https://commons.wikimedia.org/wiki/File:Pheasant_in_the_stubble_-_geograph.org.uk_-_3185682.jpg, and a purge fixed it. Sounds like T299356: Depicts statements are shown in WCQS but not in SDC is more widespread than it might have seemed at first.

@TheDJ is currently actively deprecating the service, all tiles look like https://tiles.wmflabs.org/osm/18/136090/86311.png now

In T291771#7591567, @Xqt wrote:

The "AttributeError: module 'pywikibot' has no attribute 'Site' error is fixed with T298384. Is anything left to do here?

Phabricator task to remove authentication: T297995

No, as a tool developer I don't want to authenticate, see https://commons.wikimedia.org/wiki/Commons_talk:SPARQL_query_service/Upcoming_General_Availability_release#Mandatory_authentication_considered_harmful . Filed T297995 to remove it.

In T280806#7398805, @Pchelolo wrote:

I did the same analysis as @Legoktm did last week, and the numbers didn't really change over the week 80k reqs/day upgraded out of 238k req/day.

I seems like the upgrades are not progressing any longer. There's been so many announcements related to this project already, that it's unlikely one more announcement will change anything. Let's try to roll it forward - I will not backport the revert just yet.

In T291771#7378841, @JJMC89 wrote:

If you are not using the shared install, then you are required to install the dependencies in a virtual environment.

(Toolforge is not an appropriate tag. See its description.)

Not sure why the Toolforge project was removed by @JJMC89 . A package is missing on the generic infrastructure affecting multiple tools. Also no mention of this on https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Clone_pywikibot_git_repo

Three groups of people are involved in this:

• MediaWiki API developers
• Pywikibot framework maintainers
• Pywikibot framework users

This task is tagged as priority low. It would be completely ridiculous to push this through now knowing what amount of fallout this will cause.

In T280806#7370239, @Legoktm wrote:

Pywikibot 6.6.1 was released yesterday, I'd want to push this one more week so we have time to announce the release and give people time to upgrade.

In T209697#7294371, @Zache wrote:

@JeanFred I didn't create new ticket yet, but would it be possible to update the Mirador and Universal viewer to latest versions? Current ones are old and least Mirador latest has lot better mobile support now. Also latest Tify viewer would be nice addition for the viewers.

In T277665#7275279, @Seddon wrote:

Joining between Wikimedia Commons and Wikidata via P18 in the WCQS can be done using schema:url going forward. Both are using the same encoding and should match.

<schema:url rdf:resource="http://commons.wikimedia.org/wiki/Special:FilePath/Stilleven%2C%20SK-C-1561.jpg"/>
<ps:P18 rdf:resource="http://commons.wikimedia.org/wiki/Special:FilePath/Stilleven%2C%20SK-C-1561.jpg"/>

In T195178#7335946, @Ladsgroup wrote:

In T195178#7335945, @Esc3300 wrote:

Test onwiki: constraint addition, sample statement

It's not deployed yet. It'll be deployed next week (no train this week)

Added you to the group, see https://commons.wikimedia.beta.wmflabs.org/wiki/Special:ListUsers?username=&group=interface-admin&wpsubmit=&wpFormIdentifier=mw-listusers-form&limit=50
Good luck testing.

I expect encoding issues. How are you handling spaces and underscores for example? https://commons.wikimedia.org/wiki/File:Stilleven,_SK-C-1561.jpg gives <schema:contentUrl rdf:resource="https://upload.wikimedia.org/wikipedia/commons/f/f4/Stilleven%2C_SK-C-1561.jpg"/> in https://commons.wikimedia.org/wiki/Special:EntityData/M83614923.rdf and <ps:P18 rdf:resource="http://commons.wikimedia.org/wiki/Special:FilePath/Stilleven%2C%20SK-C-1561.jpg"/> in https://www.wikidata.org/wiki/Special:EntityData/Q18608269.rdf .

I just noticed https://www.wikidata.org/wiki/Wikidata:Requests_for_comment/Non-free_content . Setting this one to declined because clearly no community consensus exists at this moment in time.

In T281854#7184875, @Harej wrote:

In T281854#7184854, @Multichill wrote:

This is not the place to discus if these items should be moved out or not.

This is a confusing statement seeing as the task is explicitly about "splitting scholarly articles from Wikidata". If this is just from a backend perspective the task should be clarified as such.

Hi folks, please stick to the Phabricator etiquette as described at https://www.mediawiki.org/wiki/Bug_management/Phabricator_etiquette . This is not the place to discus if these items should be moved out or not. @MPhamWMF don't see these comments as any indicator of the community view.

I would like to have "If several languages are provided in the constraint then the constraint is satisfied if at least one of the languages has a label added" changed to "If several languages are provided in the constraint then the constraint is satisfied if all of the listed languages have a label added" (so not OR, but AND)

In T118641#7127096, @Ladsgroup wrote:

Tentatively calling this done given that we now have mailman3, reopen if you think mailman3 doesn't satisfy AAA

In T195178#7122976, @abian wrote:

My devil's advocate questions:
<knip>

This new constraint would get plenty of usage, see https://www.wikidata.org/wiki/Help:Property_constraints_portal/Label_language

@dcausse thanks for the pointers. Might be worth switch structured data on Commons first to the new approach. It is much less integrated in all sorts of processes and it uses a ton of blank nodes or was Commons already switched? See for example P170 (creator) on https://commons.wikimedia.org/entity/M106076433.rdf for how blank nodes are currently used on Commons.

I see the right concept uri on https://commons.wikimedia.org/entity/M105912167 now (and still the right one on http://www.wikidata.org/entity/Q106874575). Thanks for fixing.

In T259801#7100788, @Zache wrote:

This affects the query service which uses the formatter URL to render the URI.

When you say Wikibase do you mean Wikidata, Structured data on Commons or completely Wikibase in general? Judging from the context it's SDC so I would scope it on that. For the monuments database we define source fields, how they map to the destination and what kind of conversion to apply. You could do something like that.

I didn't notice this task before. Where can I read the feedback you got? From who did you get feedback?

@Cparle @John_Cummings : Why are you discussion references in this task instead of in T230315 ? This task is about about serialization of the data and that the fact that we use two different keys (claims vs statements) for the same thing.

In T282222#7069792, @dcausse wrote:

My bad, the depool command I ran this morning did not work due to an error running the command (I ran sudo depool wdqs1012 instead of sudo depool so it depooled the service wdqs1012 which is nonexistent).

For the record I ran the proper command now:

dcausse@wdqs1012:~$ sudo depool
Depooling all services on wdqs1012.eqiad.wmnet
eqiad/wdqs/wdqs/wdqs1012.eqiad.wmnet: pooled changed yes => no
eqiad/wdqs/wdqs-heavy-queries/wdqs1012.eqiad.wmnet: pooled changed yes => no
eqiad/wdqs/wdqs-ssl/wdqs1012.eqiad.wmnet: pooled changed yes => no

This server should stop receiving traffic now.

As a work around on pages like https://www.wikidata.org/wiki/Wikidata:WikiProject_sum_of_all_paintings/Old_European_art_missing_genre/Sweden where most thumbnails don't work:
wget -E -H -k -K -p https://www.wikidata.org/wiki/Wikidata:WikiProject_sum_of_all_paintings/Old_European_art_missing_genre/Sweden

File names are bad URI's. Files get renamed all the time (see https://commons.wikimedia.org/w/index.php?title=Special:Log&offset=&limit=500&type=move ) causing all sorts of breakage. The pageid stays the same so the mediaid also stays the same. That's a much more stable identifier.

In T258776#6921752, @CBogen wrote:

I'm going to close this as a duplicate of T277665 as that is what we see as the practical solution to this problem. Please re-open if you don't think that will solve your use case.

In T258590#6363261, @CBogen wrote:

Note that the SD team work to change the Concept URIs in Commons is estimated to be a small.

In T166094#6878436, @Ainali wrote:

In T166094#6878327, @Multichill wrote:

No we don't, captions are just labels, see https://commons.wikimedia.org/wiki/Special:EntityData/M96527.json . Description is currently unused in SDOC.

Well, perhaps the variable is called label in the code, but in the Wikipedia app, people are being asked to fill these fields as image captions. So it's a bit of a conflicting usage of that specific field right now (descriptions on Commons, captions in the Wikipedia app), but neither of them are for the purpose of alt text. Which means it would be a huge work cleaning up before it would be useable even if we started to designate that field for it. Better start fresh in my opinion (and if the label/description/caption field is redundant in your opinion, then suggest remove it in a separate ticket).

In T166094#6877045, @Ainali wrote:

Well, then we need a new field for the caption. They are not meant to solve the same problem and should almost always be different from each other. So for every image it would be very useful with two separate text fields.

Why are we not using the description field for this? Seems more sensible to me than creating new properties

The URI for the image is https://commons.wikimedia.org/entity/M6919529 (yes, https, not http, that got messed up, see T258590). You can see an RDF representation of that at https://commons.wikimedia.org/entity/M6919529.rdf . We should someone include this URI on Wikidata too. The royal way would be to update the image data type which accepts the Mediainfo ID and does all the logic like the current image data type.

In T275787#6871707, @CBogen wrote:

In T275787#6866509, @Multichill wrote:

@CBogen why did you remove the search projects? This looks very search to me.

The Discovery-Search project is mainly for backend search work; this is a front-end project in MediaSearch on Commons so it would be part of the Structured Data team's purview.

We now have more than 1 million files with location of creation on Commons, see https://commons.wikimedia.org/w/index.php?search=haswbstatement%3AP1071&title=Special%3ASearch

If it aint' broken, don't fix it? Let's just see what happens and if anything explodes, than focus on fixing that.

Wrote https://www.wikidata.org/wiki/Wikidata:WikiProject_sum_of_all_paintings/Automated_image_uploads and shared the link by email some time ago.

In T271105#6814507, @Lucas_Werkmeister_WMDE wrote:

In T271105#6812288, @Multichill wrote:

This change is subject to the https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy . Please complete the steps listed there.

Can you clarify which part of the policy you think hasn’t been followed so far?

This change is subject to the https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy . Please complete the steps listed there.

I obviously like getting more structured data and better usage, but I see some issues with this task: The scope seems to be very broad. Is this intentional? What kind of time investment is expected? Might have a higher chance of making a difference when it's more tightly scoped. We have several steps in the process and possible tasks related to it:

• Convert existing data from wikitext to structured data. This is already happening on a small subset, but on a large scale. Not controversial as long as you don't remove any data. Still a ton of work to do here, but quite a few things are not clear yet on how to model things. Building some kind of workflow to extract knowledge from the category tree and show it to users for approval would be extremely useful.
• Show the structured data in a pretty way. You already mentioned some examples, https://commons.wikimedia.org/wiki/Module:Artwork and https://commons.wikimedia.org/wiki/Module:Information . The current approach here is to do incremental improvements without changing the look and view from what it used to be. You can also take a radical different approach like skins: You make another implementation for the same data with a completely different look and feel. With some logic logged-in users can enable this new template skin. That way you can experiment quite freely without disturbing the Monobook people. This task would be fun for someone who is on the edge of development and design.

Reasoning why the convention is suddenly enforced are missing. Instructions on how to fix are also missing.

In T267175#6754527, @dcausse wrote:

Checked a couple of these inconsistencies and they appear to all be out of order in the kafka topics. I suggest to disable async imports as I believe it might be possible cause of these inconsistencies.

@RKemper so what's the status of this? I see a lot of cases where the last edit didn't get processed so the data in SPARQL is not consistent. See https://w.wiki/ugf for some examples.

In T180113#6747475, @Elitre wrote:

@CBogen Hi, can you help me figure out if this is needed, otherwise I'd like to resolve. Thanks.

Just had it again:
pywikibot.data.api.APIMWException: internal_api_error_JobQueueError: [X-VEtQpAIDkAAHaGUqkAAADW] Caught exception of type JobQueueError
[servedby: mw1345;
errorclass: JobQueueError]

Ha, right after I posted that my bot crashed twice. Now with internal API errors:

One of my robots ran non-stop for the last week so looks like it's not happening at the moment. You got to love intermittent problems.....

https://commons.wikimedia.org/wiki/Commons:Bureaucrats%27_noticeboard#Remove_users_from_gwtoolset_group_who_haven%27t_used_it_in_the_last_two_years

Thanks Sam, can you do a per user per year breakdown? I expect a small number of users with a lot of uploads.