@Uzume Thank you for all of this investigation! I really could not figure out what the logic was that was causing this, and makes more sense noticing the sitelink, and not that it was coming from a random other property (that just happens to be the same as the sitelink's value).

Thanks for making this. I started to try to make a metadata template as well, that could be fully powered by its own structured data, and allow a media file to require no plain text metadata in the wiki page if all required properties are present, but I ran into several pieces that are impossible using only the existing {{#property}} parser function. Here are some examples:

In T230315#7303039, @mwilliams wrote:

Thanks for the great feedback. @Dominicbm you are correct that this needs to be a collection of grouped claims and that my mockups didn't communicate that functionality.

Here is a quick mockup of how that might work, keeping each grouped claim in some sort of gray/shaded box. I'd love to keep the functionality as close as possible to Wikidata, while still mapping to the visual design choices that have already been made with SDoC.

Most of the design decisions that shifted away from exactly replicating Wikidata pre-date my involvement on this work, I'll try and track down some context around that. I'm sure we could revisit those decisions if needed but wouldn't want to conflate that work with this new feature.

In T230315#7302885, @John_Cummings wrote:

Thanks very much for explaining this, is what you would like basically the same way Wikidata does references? I've been trying to think of a use case that wouldn't work for the existing Wikidata approach for adding references and I can't think of one.

@mwilliams So, in terms of the interface, I think it's going to be a little more complicated than this mockup has captured so far. A reference does more than a qualifier does, so it's not just a matter of adding a new sub-claim with a property and value when a user selects "add reference". If you check how this works on Wikidata, clicking "add reference" creates a shaded box, within which the user can add any number of claims that are all grouped collectively as that one reference. If you click "add reference" again, you would actually be starting a new shaded box with a different group of claims for the second reference.

In T188684#7106038, @Chicocvenancio wrote:

The timeout is now 24 hours. This will be an experiment and we might need to reduce it, but the intention is to support more long running tasks in PAWS.

In T258122#6317543, @Urbanecm wrote:

I'm afraid this is hard to solve... IIRC it's impossible to whitelist something smaller than a domain, and we're definitely not going to whitelist Amazon - that would be equivalent to whitelisting anything.

Update: using the default command, the generator takes a 2-3 hours to complete for this large category. Putting step at 1 and it was still unfinished 2 days later when I killed it, so that wasn't really feasible. I can experiment with other lower increments between 1 and 50, but I already know enough to know the only values low enough to prevent truncation and lost pages will make the operation take too long to complete, since 50 was already very slow and still getting truncated.

In T253591#6164399, @Xqt wrote:

The problem is that, for my use case, I really just want page titles, but I guess since Pywikibot wants to generate all the page objects using all the metadata, there is no way around this error currently.

CategorizedPageGenerator does not load the content by default. To get it you have to use content=False as parameter or use an explicit cat.get() later.

In attempting to upload to Wikimedia Commons using chunked upload, I receive a whole series of warnings and errors, including
(1) "WARNING: Unexpected offset.", (2) a large traceback from a read timeout (T253236), (3) series of internal_api_error_DBQueryError and retries, and then finally (4) stashfailed error that fails the upload attempt.

You can upload at a faster rate with a bot account. See https://commons.wikimedia.org/wiki/Commons:Bots

I think it happened for many hours, or maybe more than a day, but is not an ongoing issue.

As @WilliamGraham notes, this is actually breaking some categorization on a large scale, which may take a long time to repopulate once they have been depopulated. It's not just about infoboxes on categories, though. If I am understanding right, this is presumably affecting millions of pages on Wikimedia Commons, considering how widespread the use of templates like Template:Artwork, Template:Institution, Template:Creator in image metadata. It might be used a dozen times in a single file page alone, such as for an artist name, genre, medium, institution, and even the labels for basic terms like "height", "width", "collection", "genre", etc. displayed by the template are pulled in from Wikidata.

It's a bit hard to estimate total size in bytes, especially because the total is a moving target, based on how many partners sign on. @SandraF_WMF I recently gave a webinar on DPLA's work (https://youtu.be/0BSoKSYBcBI). Basically, we are now actively doing outreach to our partners (essentially, and US-based GLAM), and doing uploads on-request for partners that have appropriate rights and can give us the media. There are 37 million items in DPLA's dataset, of which 1.7 million are currently licensed appropriately for Commons upload. An item can have any number of media files, and we don't really know the aggregate number (or byte size), because we don't host them—we're just working with the partners to download the assets they host and reupload them to Commons.

Hi, this is me! 😳If it's easier, I can get on Telegram or IRC to chat with you about my project. Obviously, I've been going at a high rate, but I don't really want to break Wikimedia!

UploadWizard aside, this functionality would be also useful in particular for users relying on the API to do bulk uploads, since it would allow users to upload a file with structured data attached—in the same way as the unstructured file description is currently included in the same request as the media upload. This would reduce the number of API requests necessary to achieve the task of adding structured data statements for new uploads.

I would like to give this a bump. If all of the rationales provided in https://www.mediawiki.org/wiki/PAWS#Why%3F are to believed, then simply telling users to use Toolforge for any task that might need to run for more than an hour isn't the solution. I have made hundreds of thousands of edits via PAWS, using PWB, and it would be an incredible improvement to experience, as a user, to not have to worry about keeping a browser session connected.

@Jdforrester-WMF Since macOS Mojave, Shift-Command-5 is now used for an additional screenshot option, which conflicts with strikethrough. Is it possible to reconfigure? https://support.apple.com/en-us/HT201236

Thanks @Dereckson. I linked to the catalog record, which does have tiled images for dynamic zoom in the UI. However, the download link (and media URLs in API outputs) contains a regular media file (e.g. https://catalog.archives.gov/OpaAPI/media/299685/content/arcmedia/media/images/40/7/40-0678a.gif).