Percent-encoding is a basic part of URLs as specified in rfc 1738, section 2.2. URL Character Encoding Issues, so it should work everywhere.

Since I commented above, https://www.wikidata.org/wiki/Help:Dates has been edited and now says that "Any date in range 1801-1900 with precision 7 is interpreted as 19th century". Does this resolve the inconsistency?

https://www.wikidata.org/wiki/Property:P2000 is another one that's affected by the incorrect encoding of spaces to '+'. Given that these values are strings like "Cantigas de Santa Maria", which generates a failing link of http://www1.cpdl.org/wiki/index.php/Cantigas+de+Santa+Maria, it probably fails more often that it works. It should be encoded as %20, which would work everywhere.

Thanks. There are plenty of other examples in Commons, such as other subcategories in https://commons.wikimedia.org/wiki/Category:Hidden_categories and many in https://commons.wikimedia.org/wiki/Category:Pages_with_local_coordinates_and_similar_Wikidata_coordinates. It seems to effect only categories that have names starting with numbers, but not every such category.

Finding quite a few lately, such as

I still see them occasionally, such as: these two:

Is this the same thing? https://commons.wikimedia.org/wiki/Commons:Village_pump#Wrong_link_in_globale_usage

It's a bit hard to find information about this problem online, since it affects browsers that became obsolete in 2009 when a newer version of IE was released. Perhaps Microsoft even patched the bug in the old versions, if they supported them for several years afterwards?

Editing Exif without potentially breaking something is difficult. A lot of extensions have been added to JPEG over the years, and some of them contain pointers to data outside their own segments. Things like maker notes (proprietary and not fully documented) and Multi-Picture Format (which allows putting multiple images in a single JPEG file). They all need to be unpacked, adjusted, and repacked. The only practical method is probably to use ExifTool, and even that isn't perfect.

I suppose I misinterpreted the comment about disabling logins. The idea would be that somebody would craft a JPEG (or other file type, presumably) file with some embedded Javascript, and some other user would come along with some old version of IE that would bizarrely ignore the MIME type when displaying the full-size file, execute the Javascript, and do some stupid thing under their account?

At what point does this IE security issue manifest itself, during the upload process? The file is being uploaded, not being sent to the browser to be displayed.

I also noticed this recently. I attempted the work-around of creating the link from Wikipedia instead, but that also fails because there doesn't seem to be any way of selecting Commons as the link target. Maybe that should be another issue raised in Phabricator.

A small batch of 6:

It may be caused by T198350, in this latest case.

Here's another recent mini-surge:

The actual bug is that some software (such as the Wikidata user interface) doesn't seem to be following the definition of the "precision" field in a date.

Yeah, I was wrong about the values getting set to zero, the insignificant digits are retained internally.

I agree with Jc3s5h. Wikidata dates are represented in JSON with ISO 8601 strings with associated precisions, and a "century" precision value looks like:

And now dead again. Affects www.wikipedia.org, commons, wikidata, wiktionary.

Just started working again.

Well, phabricator is fine.

They still don't load for me. I think this is about April 10, not April 11.

It's working OK for me today. I'm not sure if somebody changed something or if there's some other factor.

There are more, I think at least 400 in total, but I'm not going to list them all here.

Here's another batch of 29 from the same day, a few minutes later. The ordering may be random.

I just found 24 of these files, all uploaded between 2018-02-21T20:27:36 and 2018-02-21T20:28:13 UTC, by various uploaders. Here are links, ordered by upload time:

Wikidata above should read en.wiktionary.org.

It may need to be done as part of the Structured Data project. Part of that project could involve copying useful values from Exif fields into a file's data page, and there's not much point in doing that until the corrupt values are fixed.

Maybe the non-existing category isn't significant. The file in the merged duplicate T178555 was uploaded with a valid category created in 2012.

I never saw anything like this prior to 1 Nov. Did something change around that time, a new MediaWiki version or changes to the UploadWIzard?

It can, because the first entry in the history is when the page was created by Guanaco a copule of days after the original upload. Better yet, take a look at https://commons.wikimedia.org/wiki/File:Ahora_nos_toca_a_nosotros,_ovaci%C3%B3n.jpg which doesn't have a proper page and doesn't have a history link at all.

The same problem of notability will turn up if the subjects of images are going to be linked to Wikidata items, since there are plenty of images of things like buildings that may not be notable for Wikidata. At present, there's no problem having a category for the building. The X# solution does seem preferable to having Q# items hosted on both Commons and Wikidata, and you could prevent Q# items from linking to X# items.

Technically, I suspect this would be easiest if Wikidata was allowed to hold all the items that Commons needs to store information about. Maybe there could be a way to restrict particular Wikidata items to a limited subset of properties, so that URLs etc., could be stored without giving people a place to store arbitrary unverifiable information about themselves or others. A few bits of information are relevant for determining copyright expiration: whether an author is a person or a company (my previous comments neglected to mention the concept of "corporate authors", although they seem to be usually treated the same way as anonymous authors), and date of death.

Maybe it's not even right to consider Department of Immigration and Citizenship to be the copyright holder in that example, and it's a file with Australian Government crown copyright. In that case, it's not clear what role DIAC is filling, if not the author or copyright holder.

On Commons, we currently tend to confuse author and copyright holder. It's the copyright holder who can release a file with a free license, and in practice chooses the attribution. We have a lot of files like https://commons.wikimedia.org/wiki/File:Christmas_Island_(5774557225).jpg where the author (photographer) is unknown but the copyright holder is presumably the Australian government Department of Immigration and Citizenship. This is also a good example of a file with a bad URI, since the department has been reorganised and renamed and the Flickr account is gone.

I had the idea from the demo system that such records would exist (http://structured-commons.wmflabs.org/wiki/Q2). Oh well. With a system of credit string + URI it will be problematic in some cases to find all the works by a particular author. Credit string could be all over the place (G Bush on one photo, G.W.Bush on another, George Bush on a third etc., not to mention having multiple people named G Bush) so it's not very helpful. With other people you may not be able to find a URI at all, e.g., non-notable 19th century photographers, or US government employees whose work is published on their department's Flickr account.

I'm not sure why you'd want a URI to an external site in a field like this. Is it not possible to assume that every person referred to will have an item either on Wikidata or on Commons? Then this field only needs to link to one of those places, where links to Flickr profiles etc., can be collected. When somebody is uploading a new file, you'd need to ask for some kind of link like that that identifies the person and allows linking to the right Wikidata/Commons record, creating a new one if needed. External links also go bad sometimes when people delete their Flickr accounts etc. There will be other complexities, like multiple people sharing a single Flickr account (e.g., if the Flickr account is for an organization and they have images from various authors), or people who have multiple Flickr accounts.

Those are also fixed by making sitelinks. The only relevant Wikidata policy that I know of is Wikidata:Notability, which says "an item with only a sitelink to a category page in Wikimedia Commons is not allowed on main article items", which I believe means you can't create new Wikidata item which is linked only to Commons. That won't be an issue for most of the biological categories.

Actually in this case there's a category item on Wikidata, so the Commons category can link to that. It makes it slightly harder to set up, because it needs a template on the category to pull in the links from the main Wikidata item.

I made a sitelink on Commons Category:Anastigmat and now the links are fine. There has been some discussion on Wikidata in the past about whether this is the right way to do it, but no consensus was reached. However the software is set up to support it ("Add Links" in the sidebar) and it's the only way to make all the sidebar links appear on the Commons side, including the link to the Wikidata item.

Did you set a default license in Preferences / Upload Wizard?

Thanks, that fixes it for me. I guess the load order was changing between the working and non-working versions, maybe depending on whether the files were cached.

I have more luck reproducing it with "fresh" files. Refreshing the page will fix it, and then it stays fixed when refreshed several more times. There's nothing special about that particular file I posted, it's quite easy for me to reproduce on a random file. Adding ?debug=true seems to fix it too.

Disabling the CropTool gadget in preferences seems to help.

One thing that stands out is:

Thanks for fixing that file, but there are many others in https://commons.wikimedia.org/wiki/Category:Invalid_equipment_in_Exif (some are genuinely invalid, others are probably caused by this bug), and more are found all the time.

I've reopened this, since T140419 is supposedly fixed since August. I've tried purging the image description page, which according to https://www.mediawiki.org/wiki/Manual:File_metadata_handling is supposed to read the metadata again, but the metadata is still corrupt.

Hmm, maybe that fix isn't on Commons yet: T140419

I think my problems may go away if I request commonmetadata instead of metadata.

It seems that the entire text of a document may be embedded in djvu or pdf metadata. Apart from the problem of requests failing due to the limit, it's wasteful to need to request so much data if you are interested in a few specific fields.

Is this 12MB+ of metadata in any way useful? The file page in Commons doesn't show any trace of it: https://commons.wikimedia.org/wiki/File:Boiste_-_Dictionnaire_universel,_1851.djvu Maybe its corruption either in the original file or when it's imported into Commons?

Thanks, I only save the initial cookies from the login. I'm curious to know why they change now.

OK, I created T130091. I'm not using pywikibot, so that's not the issue.

I also have a problem that seems related. I save the cookies from a login request and use them in subsequent runs of a bot. Previously, it would work fine for months on end. But in recent weeks/months the bot is no longer able to edit pages after a few hours of inactivity. According to an assert=user query, it's still logged in. But fetching a new csrf token and attempting to edit a page always fails with the error: badtoken: Invalid token. It works again after a new login.