Copying over the report from the other ticket, in case that helps resolve this:

Oops, I'd forgotten about the previous ticket, thanks for merging. Also apologies for accidentally changing subscribers, restoring those.

In T209310#9715995, @Nardog wrote:

Not sure what still needs to be done about this technically. The deprecation of the old module can be done by volunteers on wiki.

In T209310#9715656, @Taylor wrote:

Working link: https://www.mediawiki.org/w/index.php?title=Extension:Scribunto/Lua_reference_manual&diff=prev&oldid=6470970

Indeed the global deprecation of "Template:=" "Template:!" and "Module:No globals" should be stronger.

The discussion at Commons is indeed silly. How complicated can it be to remove a blatantly outdated comment line from a module?

After being abruptly told to make an edit on Commons to remove Module:No_Globals from WikidataIB (https://commons.wikimedia.org/w/index.php?title=User_talk:Mike_Peel&diff=prev&oldid=868602169), I finally traced the logic back to this ticket. I've added a link to this from the documentation [https://www.mediawiki.org/w/index.php?title=Extension:Scribunto/Lua_reference_manual&diff=prev&oldid=6470970] to help others find this ticket, and will make the edit, but there needs to be way better outreach to migrate all the uses described at https://www.wikidata.org/wiki/Q16748603 to this new solution.

In T360360#9675009, @Bugreporter wrote:

Note if (in the future) one day we add chapter wikis to Wikidata, this will be embarrassing since they are not "multilingual" sites.

Thanks for working on this @Amire80! It looks like a good way forward. Maybe good to subscribe those following T102417 to this follow-up ticket, to keep people in the loop?

Migrated.

Many thanks @taavi - I managed to figure it all out today, particularly using that last link. It seems it's basically a complete change from the old system. I'll keep an eye on it, but hopefully it will run daily using the new system from now on.

The setup works fine with jsub, how do I just use the same environment with this new setup?

Can't I use my local environment, which loads when I log in?

Ah, well spotted, thanks! That moves the problem on a stage, where it now says:

So I must be missing something, but I was running this at /data/project/pibot/wikicode:

Oh, me again. On it...

I'm the operator of Pi bot, I'll check into this as soon as I can.

In T345783#9333011, @Legoktm wrote:

Switching to path-based input is something that tool developers would need to support in each tool

I'm (personally) not sure this is something that would only be solved with a research paper, it would probably also need the development of a generally available tool to enable future analytics in this area - although those probably aren't decoupled things.

Thanks @Legoktm - that seems to work for Scholia. However, it doesn't work for Resonator - try https://iw.toolforge.org/reasonator/Q1513315 vs. https://reasonator.toolforge.org/?q=Q1513315 . And then there are more complex cases like https://glamtools.toolforge.org/glamorgan.html?&category=South_Pole_Telescope&depth=1&month=last or https://wikishootme.toolforge.org/#q=Q1513315&main_commons_category=South_Pole_Telescope etc. (and other links at the bottom of https://commons.wikimedia.org/wiki/Category:South_Pole_Telescope - or the infobox in most commons categories).

In T14019#9299831, @Umherirrender wrote:

Some base work in core has started and is linked with T268526: Use a dedicated mechanism to track page dependencies
But without support for Special:WhatLinksHere, the data are internal to mediawiki.

Please do make sure to submit your final application via the Outreachy website before the deadline! That is the only way we can formally receive applications to consider for the project.

Just to be clear - make sure to submit your contributions and final application on the Outreachy website before the deadline! It's fine to still include contributions there if they haven't been reviewed or if you've made changes since you last got feedback on them. But do make sure you submit your final application before the deadline!

Just to be clear - make sure to submit your contributions and final application on the Outreachy website before the deadline! It's fine to still include contributions there if they haven't been reviewed or if you've made changes since you last got feedback on them. But do make sure you submit your final application before the deadline!

OK, I think linking is fine, I just wanted to avoid a situation where you submitted something here but didn't fill in a final application on Outreachy, which would be bad. :-) Normally I've just seen written timelines posted on Outreachy (along the lines of 'Weeks 1-2, X, Y, Z activities', with weeks on different lines), I guess the Outreachy system isn't so friendly for tables.

Hi @Shashi.coding Please make sure to post this on the Outreachy website directly, since we can only accept applicants through that process. You must make a final application on the Outreachy website. I'm closing this phabricator task as invalid.

Hi @Anne you need to submit this on the Outreachy website, not here. Closing the task as invalid.

In T313467#9271763, @ItamarWMDE wrote:

We can definitely release it a day or two later. This will give us a chance to review the release before it goes out.

In T313467#9265250, @Arian_Bozorg wrote:

Thanks for the feedback Mike! Looks like we will keep the type column in the last position and it will remain optional for now.

This will be released on 1 November, so should work for your first half of the month schedule :)

If you're going to be supporting mismatches on labels, descriptions, aliases (which would be good!), then making type required might make sense - and making changes like that while the tool is still in early stages is good (or at least, better than later!). There are ~3.6m mismatched English descriptions at https://en.wikipedia.org/wiki/Category:Short_description_is_different_from_Wikidata that I could start automatically uploading when that becomes possible.

Is this the breaking change? I'm currently using 'item_id,statement_guid,property_id,wikidata_value,meta_wikidata_value,external_value,external_url'. I'm not sure what a guid is? Will 'type' be optional - in which case it is better at the end rather than the start?

With "kind of experience you are looking for", it's anything that demonstrates that you are well suited to do the project, which could be coding/documentation/etc. that you can link to as examples; involvement in the Wikimedia projects, other open source projects, etc. - it's an open question that is worth spending time answering to describe the background that you're coming from.

I'm doing a bulk upload from LIghtroom this eve and am running into "The MediaWiki error backend-fail-internal occured: An unknown error occurred in storage backend "local-swift-eqiad"." again.

Thanks so much for working on this @Arlolra ! Let's see how this goes. :-) If the different filename formats do continue to cause problems, I could probably update the code for QICbot to correct them on a daily basis, if that might be useful.

In T347259#9250484, @Jcxiang wrote:

As many of us found out, we cannot access the Slack channel to communicate, for myself, I also tried Zulip and email to contact you but there was no response. I am wondering if we can build some possible communication to discuss after completing these two microtasks, like to discuss the project deadline for the project application.

Apologies for the confusion about Slack - have found and corrected the bad link (hopefully!). We're not using Slack for this part of the process, you should join Zulip instead using this link: https://wikimedia.zulipchat.com/#narrow/stream/399895-outreachy27 (also given at the top of this page).

In T313467#9241353, @ItamarWMDE wrote:

It might have just taken a while for the job to complete. As far as I can see the import succeeded

Re-run this evening, it seems to be stuck at 'Pending'.

Reran today, same error according to https://mismatch-finder.toolforge.org/store/imports

In T313467#9215948, @ItamarWMDE wrote:

@Mike_Peel @Lydia_Pintscher The system was reverted to the stable version, apologies for the mix-up. Could you attempt the upload again, please?

Also seeing this, ongoing issue? E.g., at https://www.wikidata.org/w/index.php?title=Q34623&curid=37349&diff=1985491049&oldid=1959113485 , https://www.wikidata.org/w/index.php?title=Q19599427&oldid=prev&diff=1984920937

Thanks @TheDJ. I'm happy to claim responsibility for this - it's because the Wikidata Infobox improves multilingual search engine optimisation in Commons categories that it's used in by including all language Wikidata labels. I don't think the changes to search helped things here (from my personal point of view, they made things worse by hiding category search results). I've been talking about this for several years now as an improvement to multilingual search on Commons, so I hope Discovery-Search have seen it and that it hasn't unexpectedly affected their research. I would be very reticent to turn this functionality off before knowing that multilingual search has indeed improved beyond this being necessary.

In T343131#9147972, @LucasWerkmeister wrote:

In T343131#9147662, @Mike_Peel wrote:

For links to toolforge, https://wikitech.wikimedia.org/wiki/Tool:Iw exists to provide interwiki links - but it doesn't seem to pass on parameters. For example, https://scholia.toolforge.org/Q1513315 works, as does https://tools.wmflabs.org/scholia - but e.g., https://tools.wmflabs.org/scholia?qid=Q1513315 does not. Some sort of service like this might mean we could change ~10x4.5m external links into interwiki links.

I made a task for adding this capability to the iw tool, which seems reasonable to me because it’s already intended for interwiki links: T345783: iw.toolforge.org does not support URL-encoded query parameters ([[toolforge:foo?bar]])

Alternatively (e.g. if that task doesn’t happen), we could also create dedicated interwikis for highly-used tools (e.g. scholia: for https://tools.wmflabs.org/scholia?qid=$1).

It's stopped working again...

Thanks @Ladsgroup for the chat at Wikimania. Following up on our conversation (sorry for the delay), from the perspective of the Wikidata Infobox on Commons:

Thanks, just to confirm, it's working nicely again now.

Explaining it to the user would definitely be an improvement. Respecting blocks is important though, although it's not obvious when it's still allowed to have access despite the VPN block and then not be able to do certain things without explaination.

In T329631#9096554, @Sarai-WMDE wrote:

I'd advise for us to find a way to notify users (/uploaders) about which of the mismatches were ignored and why (either because they already exist or it has been previously reviewed). Displaying this in the upload summary, if possible, would be a good idea.

(Edited task description accordingly, I hope that helps.)

In T344056#9086692, @taavi wrote:

In T344056#9086675, @Mike_Peel wrote:

Although, I can thank people on enwiki (and just did so), so maybe the VPN isn't the only cause... Odd!

You're an enwiki admin, which would let you bypass IP blocks there.

Although, I can thank people on enwiki (and just did so), so maybe the VPN isn't the only cause... Odd!

Also not there with another account with clean settings, in a private window. I definitely suspect my VPN as being the cause now, though!

Ah - I'm connected with NordVPN at the moment (traveling), perhaps it's auto-disabled on that?

Yup. Will check with another login...

In T343131#9058592, @Ladsgroup wrote:

• To be able to respond to millions of query per second, dbs hold as much as possible in memory, it's called buffer pool and it's 350GB. If the data grows with this speed, dbs will start to read from disk more and more (which is much slower) and that will lead to reads to pile up and generally everything go down together. We really can't buy bigger memories. We have ~250 DB hosts (in total everywhere) and each one have 512GB memory.

I've replied at https://commons.wikimedia.org/wiki/Template_talk:Wikidata_Infobox#Category%3AUses_of_Wikidata_Infobox - of all possible aspects of increased server load caused by the Infobox, category inclusion was the least of my worries! It can be reconfigured if needed, though.

I'm not sure what's changed, but it seems to be working again now?

In T341736#9031844, @Arlolra wrote:

@Mike_Peel Is this still an issue for you? Can you try a hard refresh of the page? CTRL-Shift-R

Hmm, or not, it stopped again. Very odd.

Hmm, perhaps that was it? It seems to be working OK again now.

It seems to have stopped working again in the last hour, but at a different point of the process. You can enter votes, and you see "1 pending review confirmations" - but 'Confirm reviews' doesn't work. Did something change in the last hour?

Thank you so much, that seems to have fixed it!

This possibly broke QICvote on Commons - please have a look at T341736.

Sorry for the delay, 412 photos now uploaded at https://commons.wikimedia.org/wiki/Category:Wikimedia_Hackathon_Athens_2023 - let me know if there's any problems with any of them!

Still happening. "The MediaWiki error backend-fail-internal occured: An unknown error occurred in storage backend "local-swift-eqiad"."

Thanks, the behaviour does seem to have changed to be correct now.

Thanks for publishing it! :-)

Adding Edward, chatting to him at the Hackathon he may have useful input here. :-)

Do you (or could you) support Cite Q through this tool? Also see T289287.

Sorry for the belated response. The process of getting the geoshape is described at https://www.mediawiki.org/wiki/Help:Extension:Kartographer#Via_Wikidata_ID - it works in reverse from the usual setup, you have the Wikidata QID on the OSM feature, and Kartographer somehow manages to query for that and pull the shape over to display it. (It's not stored on Wikidata since OSM IDs aren't stable.)

I plan on taking photos during the event and uploading them to Commons, would be happy to also take photos on request.

Not sure if it's related, but I keep getting this error when uploading new version of files.

@MatthewVernon Thanks for looking into this. Just to note that it does go back a bit longer, this ticket was started in early February.

Just to note that I've now gotten to the bottom of this in the pywikibot code. If after getting the property claim for an external-id, I request 'clm.getTarget()' - then I get lower case. If, however, I do 'clm.getTarget().title()' then the first letter is capitalised. To generalise the code, I've added a check for if a claim is an external-id or not, and use the appropriate call accordingly. This diff: https://github.com/mpeel/wikicode/commit/3f2c18ef8c559a02d04b81aa330ad9a843fb3a52

Copying over my comment from the task that was merged in (I didn't see this one, wasn't expecting Yann to have created two on the same issue):

I'm also getting this error surprisingly often at the moment (after <5 minutes of starting a bulk upload). From Lightroom uploading: "The MediaWiki error backend-fail-internal occured: An unknown error occurred in storage backend "local-swift-codfw"." I've also had the same error when uploading a new version of a file.

The items were created in 2022, so probably not a current problem?

Sorry, it works out too expensive for me in the end. :-( Hope to join in with a future Wikisafari instead!

It looks much better now, thank you for working on this!

I think this happens when the article is shorter than the sidebars, going from a random selection of articles.

Hi, it's still on my to-do list, sorry. :-( Will try to get to it as soon as I can.

I'm potentially interested, but I could only participate if it's before the end of March. The current suggestion of 11 February might work.

I'm just wondering - if the system is checking and knows that the link is to a redirect already, is there a way to highlight or auto-select the sitelink to redirect options?

@Azaya89 @Aklapper The Slack link was mentioned on the Outreachy website, I've fixed it. At this point in the process we use Zulip - please see https://www.mediawiki.org/wiki/Outreach_programs/Zulip for how to connect.

Ah, I see, thanks. Looking a bit more, there are examples where they are noindex but probably they do want to be in Wikidata, e.g., https://en.wikipedia.org/wiki/Category:Serial_murders_in_France - so excluding noindex probably wouldn't work well anyway. Will have to look into this further before deciding how the bot should approach this now.

@Ladsgroup The specific case I came across was AfC categories on enwiki, like https://en.wikipedia.org/wiki/Category:AfC_submissions_by_date/07_January_2021 - that should be excluded, but weren't.

Hi, using the new database structure, I've run into a problem: I think it's not excluding one or both of noindex and hiddencat - which normally should not be connected to Wikidata. Please could someone check this?