imagerotate sounds oddly specific (also, this supports video, right?). Could we use something like filetransform with transform=rotate? (Easier to rename befor it starts getting used in production...)

In T35186#1457957, @brion wrote:

(Note that at the hackathon I chatted with Trevor about ideas for non-destructive image editing that I think would be better long-term. Will file issues in phab in a bit...)

Keep in mind that jpegtran makes 90-degree rotations, flips, crops, etc. lossless. It's probably still destructive in absolute terms, since doing one action and then the opposite might not yield a file identical to the initial one, but does that matter? SVG, PNG, TIFF, GIF, XCF are all lossless.

I think that only leaves DjVu and PDF as formats that can be lossy. But I'm pretty sure that lossy PDFs can be rotated losslessly (probably storing the page orientation as metadata), not sure about DjVu.

DjVu and PDF

Ambitious as they allow multiple pages: User Sim Plex follows a link (&page=2) to the file description page showing page two of InconsistentOrientation.pdf, which deserves rotation. Sim Plex issues a rotation request. User Power Admin recognizes that, sees page one, which is correctly orientated and blocks Sim Plex for vandalism.

In T35186#1458884, @Rillke wrote:

DjVu and PDF

Ambitious as they allow multiple pages: User Sim Plex follows a link (&page=2) to the file description page showing page two of InconsistentOrientation.pdf, which deserves rotation. Sim Plex issues a rotation request. User Power Admin recognizes that, sees page one, which is correctly orientated and blocks Sim Plex for vandalism.

Well I'm not familiar with the workflow for PDF editing on wikis, but that sounds like a solvable issue in general. Since it's a paged format, edit actions that modify a specific page should convey that information clearly to people who will review those editing actions. If that's not how it works right now, that's what needs to be fixed, it's not an issue with the idea of rotating a page inside a PDF per-se.

https://gerrit.wikimedia.org/r/225097 - updated patch with partial infrastructure for tracking the jobs, and expandability to add cropping and trimming in the future. Still not quite done, the jobs get lost in my test instance after I queue them, probably doing something wrong with the jobs. :)

Need to at least add the ability to query the status; not sure if should delete queue entries when they're done or keep them around for future status-checking...

About the "find out whether there is already a rotation pending" problem:

What about the API response of action=imagerotate tells you if the file could be added to the queue and if it could not, give an appropriate message?

rotation pending" problem
What about the API response of action=imagerotate tells you if the file could be added to the queue and if it could not, give an appropriate message

Two considerations:

• Users like to know about the status before they're requesting an action, everything else constitutes bad UX and is not necessarily much easier implemented
• It could make sense removing the old job from the queue if the newly requested job is different, and enqueue the new job (and return success + details about the deleted job) or to throw an error or to have a parameter telling imagerotate what to do in such cases

Keep in mind that jpegtran makes 90-degree rotations, flips, crops, etc. lossless. It's probably still destructive in absolute terms, since doing one action and then the opposite might not yield a file identical to the initial one, but does that matter? SVG, PNG, TIFF, GIF, XCF are all lossless.

I think that only leaves DjVu and PDF as formats that can be lossy. But I'm pretty sure that lossy PDFs can be rotated losslessly (probably storing the page orientation as metadata), not sure about DjVu.

For a start, if it makes the implementation faster, we could do with only pictures (JPEG, GIF, PNG, TIFF), and leave the rest (SVG, PDF, DJVU, videos, etc.) for later.

In T35186#1459807, @McZusatz wrote:

About the "find out whether there is already a rotation pending" problem:

What about the API response of action=imagerotate tells you if the file could be added to the queue and if it could not, give an appropriate message?

So as of the current patch, this req to rotate a video file (which does not yet support rotation):

(new mw.Api()).postWithToken('edit', {
	action: 'filetransform',
	titles: 'File:Pumpjack.webm',
	rotation: '90'
}).done(function(data) {
	console.log('done', JSON.stringify(data));
}).fail(function(data) {
	console.log('fail', JSON.stringify(data));
});

yields:

{"batchcomplete":"","filetransform":[{"id":10,"ns":6,"title":"File:Pumpjack.webm","result":"Failure","errormessage":"<file_transform_rotate_cannot_rotate>"}]}

The localization is incomplete, so you see a message key in angle brackets there instead of the explanatory message. :)

On a successful queuing you get something like:

{"batchcomplete":"","filetransform":[{"id":23,"ns":6,"title":"File:Copyrightpirates.jpg","result":"Queued","ftid":4}]}

so you know it's queued, and what the file_transform.ft_id key value is to look it up for status later.

(new mw.Api()).get({
	action: 'query',
	titles: 'File:Copyrightpirates.jpg',
	prop: 'filetransformstatus',
	ftid: 4
}).done(function(data) {
	console.log('done', JSON.stringify(data));
}).fail(function(data) {
	console.log('fail', JSON.stringify(data));
});

which gives you some info:

"done" "{"batchcomplete":"","query":{"pages":{"23":{"pageid":23,"ns":6,"title":"File:Copyrightpirates.jpg","filetransform":[{"ftid":4,"userid":"1","timestamp":"2015-07-18T06:59:31Z","startedTimestamp":"2015-07-18T06:59:32Z","completedTimestamp":"2015-07-18T06:59:33Z","success":""}]}}}}"

In T35186#1461372, @Yann wrote:

Keep in mind that jpegtran makes 90-degree rotations, flips, crops, etc. lossless. It's probably still destructive in absolute terms, since doing one action and then the opposite might not yield a file identical to the initial one, but does that matter? SVG, PNG, TIFF, GIF, XCF are all lossless.

I think that only leaves DjVu and PDF as formats that can be lossy. But I'm pretty sure that lossy PDFs can be rotated losslessly (probably storing the page orientation as metadata), not sure about DjVu.

For a start, if it makes the implementation faster, we could do with only pictures (JPEG, GIF, PNG, TIFF), and leave the rest (SVG, PDF, DJVU, videos, etc.) for later.

Agree. JPEG and PNG are the file with the most rotation requests. (The current broken bot does not support other file types.)

Why is a queue needed here? Do we have a queue for file uploads? This problem seems basically like it's solved by a reupload, without the need for an external HTTP request and inter-network data transfer. A reupload neatly preserves file history, allows for trivial reverting, and there's infrastructure in place already to regenerate local and foreign thumbs, of course.

Are we really expecting rotation requests to be burdensome enough that a queue is needed? If so, perhaps that points to a problem during initial upload...

In T35186#1462034, @MZMcBride wrote:

Why is a queue needed here? Do we have a queue for file uploads? This problem seems basically like it's solved by a reupload, without the need for an external HTTP request and inter-network data transfer. A reupload neatly preserves file history, allows for trivial reverting, and there's infrastructure in place already to regenerate local and foreign thumbs, of course.

Are we really expecting rotation requests to be burdensome enough that a queue is needed? If so, perhaps that points to a problem during initial upload...

• it *is* done as a reupload, done on the server side.
• the queue is for the actual file modification operation, which can be expensive on large files.

In T35186#1462034, @MZMcBride wrote:

Why is a queue needed here? Do we have a queue for file uploads? This problem seems basically like it's solved by a reupload, without the need for an external HTTP request and inter-network data transfer. A reupload neatly preserves file history, allows for trivial reverting, and there's infrastructure in place already to regenerate local and foreign thumbs, of course.

Are we really expecting rotation requests to be burdensome enough that a queue is needed? If so, perhaps that points to a problem during initial upload...

I think that what is wanted, is for the rotation to happen on a different server (As it could be I/O heavy, so they don't want it to be done by the servers answering the main web request). Having a queue is one way to accomplish that.

In T35186#1462035, @brion wrote:

• the queue is for the actual file modification operation, which can be expensive on large files.

Large files are annoying. :-(

I split out a discussion about the general workflow problem to T106216.

AFAIK, we have a publishing queue from personal stashing area to the public area, which can be optionally used through an async parameter of action=upload. It tends to be robust, especially for large uploads which were uploaded in multiple chunks, at least this was my experience when authoring bigChunkedUpload.js

AIUI this is supposed to extend to videos eventually, and that definitely needs a queue.

@Rillke @Tgr : for now it is important to create a functional file rotation (of jpeg files) to restore status quo :). The vide rotation stuff is not so important for now.

Can I make a request that this ONLY be done for 90, 180, or 270° rotations? Anything that needs a crop or transparency should require actual human thought.

In T35186#1636134, @AdamCuerden wrote:

Can I make a request that this ONLY be done for 90, 180, or 270° rotations? Anything that needs a crop or transparency should require actual human thought.

Hi @AdamCuerden, this task is for the implementation in MediaWiki. What is finally enabled on Commons/ Wikipedia/ ... will be subject to another discussion. Also I think it would be a nice feature if you could somehow edit your files before uploading, don't you think?

@Rillke

That's what's image editing programs are for. I think it's actually kind of dangerous: Each change degrades an image slightly, and arbitrary rotation - in the absence of things like guides and such - are likely to be too approximate. And heaven help us if we allow arbitrary cropping on upload - we already have a problem with engravings and such being inappropriately cropped in ways that kills a lot of their reusability. (e.g. removing the edges, meaning they can no longer be reproduced in their original form; removing text that was a part of the engraving (and often not documenting it) etc.

And that's not even getting into the horrors that could be done with colour editing. You generally only have one shot to get colours right. Get them wrong, and if you don't have the original, you can often have burnt out too much of the image.

The ability to edit before uploading exists. GIMP is freely available on every major platform save mobiles - which would be unable to do any serious image editing anyway - , and free. On uploading, where people are unlikely to spend more than a few minutes on it, we should only offer the basics: rotation as a multiple of 90°. Anything else should only ever be offered after uploading.

It rather seems like the scope of this is growing into something the community doesn't want or need, when all we really need is the ability to correct simple 90° rotations. Hell, even just the ability to set an orientation code in the EXIF might be enough.

See http://www.daveperrett.com/articles/2012/07/28/exif-orientation-handling-is-a-ghetto/ - it's not perfectly supported by every website, but Wikipedia handles it quite well, and it's not like we haven't led things before. OGG support was almost non-existent before Wikipedia.

In T35186#1641997, @AdamCuerden wrote:

It rather seems like the scope of this is growing into something the community doesn't want or need, when all we really need is the ability to correct simple 90° rotations. Hell, even just the ability to set an orientation code in the EXIF might be enough.

Agree.
This bug is only for the rotate function itself - to replace this function with the currently broken bot. The bot will be switched off soon (it is just fixed with some ugly hacks) (currently there are thousands of requests to rotate files every month). :-)

In T35186#1641997, @AdamCuerden wrote:

It rather seems like the scope of this is growing into something the community doesn't want or need, when all we really need is the ability to correct simple 90° rotations. Hell, even just the ability to set an orientation code in the EXIF might be enough.

As stated earlier in the comment thread, 90 degree rotations can be performed losslessly on many formats (the vast majority of our originals, when you look at the distribution of file types), including JPEG without having to use EXIF rotation. jpegtran is what allows lossless EXIF-free rotation of JPEGs.

In T35186#1642142, @Gilles wrote:

As stated earlier in the comment thread, 90 degree rotations can be performed losslessly on many formats (the vast majority of our originals, when you look at the distribution of file types), including JPEG without having to use EXIF rotation. jpegtran is what allows lossless EXIF-free rotation of JPEGs.

Agreed that jpegtran and the equivalents for other formats are better; just pointing out the EXIF option as a possible fallback if that's unfeasable.

@brion: May i ask what is the status of this? :-)

Change 250291 had a related patch set uploaded (by Ori.livneh):
include mediawiki::multimedia on all application servers

https://gerrit.wikimedia.org/r/250291

Since the migration to HHVM, several things have changed:

• Request resource limits are now identical across all application servers.
• CPU utilization on the API cluster is a third of what it was a year ago.

If we simply install multimedia packages on all application servers (as I'm proposing we do in Ib5dee0fce), the end result would be a world in which every application server is in principle capable of serving every kind of request. I think that this would make our software stack substantially simpler to manage and would pave the way for elastic load-balancing and SLAs.

Once Ib5dee0fce is merged, I think the next logical step is to simply enable the imagerotate API (in its current, synchronous form) while monitoring load.

There is a risk that the load generated by image rotation requests will be too high and that we will need to disable image rotation once again, which would no doubt be frustrating to users. But I think there is a very good chance that we will handle the load just fine, and that it is worth a shot.

@ori, just for me getting things right, it is okay now to do multimedia operations synchronously on the application servers? No queue required because the resource limits are now identical across all application servers and there is less load on them and not desired because it would complicate managing the software stack substantially?

That aside, I like Brion V.'s suggested interface "filetransform". It's not so narrow in scope.

From 105877, the imagerotate API module is disabled on beta just like production:

In T105877#1466397, @hashar wrote:

CommonSettings.php has:

// T35186: turn off incomplete feature action=imagerotate
$wgAPIModules['imagerotate'] = 'ApiDisabled';

Disabled because of T35186: Provide a well-performing API to rotate an image.

If one want to enable it for testing purpose, I guess it is all about adding the setting in a -labs.php file.

Because of the recent changes in session handling/tokens the bot is broken. I am unable, even after multiple hours of work (i don't have time to spend moor hours for this), to detect the problem. Which mean rotatebot is no longer rotating files and the backlog will become bigger every day: https://commons.wikimedia.org/wiki/Category:Images_requiring_rotation_by_bot

If the bot is getting logged out, it's probably not handling cookies right. (Use a standard library, don't try to do it by hand.) If it's not getting logged out but getting token errors, it's probably not urlencoding request parameters properly.

Unfortunately, i don't have time to re-write it / porting to a standard library. And i think it is time now to include this function in mw, especially because it is highly used.

I attempted to fix up how the rotation bot does logins. https://bitbucket.org/Steinsplitter/rotatebot/pull-requests/2

Thanks but unfortunately it does not work. The whole code needs a re-write. The best would be to port it to MW (the function in MW api exist yet, but not compatible with WMF cluster yet).

Hmm, I see your bot uploaded rotated versions of a few files today – have you gotten it to work after all?

In T35186#2033230, @matmarex wrote:

Hmm, I see your bot uploaded rotated versions of a few files today – have you gotten it to work after all?

I am current merging the bot to a stable library (https://en.wikipedia.org/wiki/Wikipedia:Peachy).

Update: Bot is back again. Now rotation of bigger files and .tif(f) is supported.

Update2: But of course this bug should be fixed contemporary.

Change 250291 abandoned by Ori.livneh:
include mediawiki::multimedia on all application servers

https://gerrit.wikimedia.org/r/250291

FYI: https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey/Categories/Commons#Allow_file_rotation_on_commons

The biggest problem with this patch is that it came out of a request to
allow rotations in multiples of 90°. Commons does not need or want anything
besides that, because you don't want eyeballed rotations, and 90° multiples
are about the only ones you can do eyeballing that are worth doing.
Anything else you want people taking the image to proper image editing
software, with guidelines and care.

PLEASE DON'T give us something that you think would be better, when,
practically, all it does is encourage slow degredation of image as people
go "is 1°' enough... no... try another degree..."

Image curation and a free rotation tool are not compatible.

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
Virus-free.
www.avast.com
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Any updates on this?