Why would you want to do that anyway? It doesn't make the URL prettier.
https://nl.wikipedia.org/w/index.php?title=Arjen_Robben&diff=next&oldid=49465457
Description
Description
Event Timeline
Comment Actions
It's not about pretty in many cases. It's about ensuring functionality by making sure the proper encoding is being sent. This is a normal, properly conforming, URL.
Comment Actions
It's not about pretty in many cases. It's about ensuring functionality by making sure the proper encoding is being sent. This is a normal, properly conforming, URL.
Comment Actions
Encoding
https://google.be?query=test
to
https://google.be?query%3Dtest
https://web.archive.org/web/20100724024636/http://www.waarschuwingsdienst.nl/Risicos/Virussen+en+malware/Keyloggers+houden+uw+toetsaanslagen+in+de+gaten.html
to
https://web.archive.org/web/20100724024636/http://www.waarschuwingsdienst.nl/Risicos/Virussen%2Ben%2Bmalware/Keyloggers%2Bhouden%2Buw%2Btoetsaanslagen%2Bin%2Bde%2Bgaten.html
is very ugly. I do not consider this an improvement, so edits like this don't make sense: https://nl.wikipedia.org/w/index.php?title=Keylogger&diff=prev&oldid=49591421
= and + will always work, I don't see the need to encode these characters. Can you please exclude these from being encoded? Your bot makes URLs unreadable otherwise.
Comment Actions
The first URL doesn't get encoded that way. = is only encoded in the path. This is done because = isn't actually legal in the URL and is only used to define values to parameters in the queries, based on RFC standards.
The second URL is being normalized to ensure operation of the archive.
Comment Actions
OK
The second URL is being normalized to ensure operation of the archive.
Why? The archive link already works.
Comment Actions
The biggest reason is to keep the DB of IABot from cluttering with duplicates. The second reason is if the archive conforms to proper standards, it makes it easier for the Wayback Machine to load it, and should a change that breaks support for improper encoding, this URL is less likely to break from said hypothetical change.
Comment Actions
I would prefer to keep the original URL behind the https://wayback.archive.org/web/XXXXXXXX/ part for obvious reasons:
- knowing what the exact original URL was
- it works - don't change it unless needed
- it's more readable
Comment Actions
The problem is it's hard coded into the analysis routine. It's sanitized and then saved into the DB. Not sanitizing it begins the cluttering process of the DB. I wouldn't have any idea how that can be "turned off" for certain wikis while retaining the sanitized URL in the DB.
Comment Actions
When reading the URLs, you santize the URL to store it in the database. You write that sanitized URL back to Wikipedia, but with a few exceptions:
- use + instead of its encoded variant
- use = instead of its encoded variant (unless you really want to comply with RFC standards)
Does this sound difficult to you?
Comment Actions
Encoding + is also complying with standards. And yes it is, since the library of sanitization uses standard encoding protocols from an external library.
Comment Actions
You could afterprocess it, when it comes out of the external library. I will ask others for opinions on this matter.
Comment Actions
That's the next problem. Bloating the code. This also in the global parser which means it will also change on the other 5 wikis it's being run on.
Comment Actions
We chatted about this, and we came up with an idea: if we can disable converting the encoding for existing archive.org URLs, I can live with this solution. I propose to call this "convert_archives_encoding=0" but feel free to change the parameter name.