As a preliminary result, I just uploaded a few (read: 10-20) files at a time, both smaller files and larger ones, and I didn't see any significant spike on cluster load during the timeframe of my uploads and first thumbnails. Also, all of the thumbnails generated on the first file page load, regardless of file size, up to about 22MB. I am fairly satisfied that we can handle any increased load due to this new filetype, but if we need additional testing via mass uploads through the API, then we can do so! I just need to find a bigger source of test files, since Thingiverse is getting incredibly tedious to use. (I have some thoughts on this)

@dr0ptp4kt thoughts?

Yeah, let's do the mass uploads through the API technique, too. I imagine you could taint some unimportant part of the files to basically upload "different" files.

Yeah, I'm running a script right now to generate a bunch of ~400kb files, but I don't know how much more I can do.

I'm assuming you'll be storing the result in swift beta, should be ok in terms of space (i.e. around 400MB + thumbs)

Update for those following along: we're hopeful to validate this approximately this current week.

OK, now with thumbnails working and my mass-upload complete, looking at the performance dashboards for beta labs, we have a performance hit that isn't too terrible, even with 1000 image uploads over the course of 30-45 minutes. However, imagescaler-01 looks a little ominous:

I don't think this will be a problem, because once we move towards the live cluster, we'll have load balancing on a better scale, and it seems pretty unlikely that we'll have a huge hit like that (1000 files is pretty huge, we might get a donation of a few thousand, but it won't be sustained)

In any case, it looks fine to me, and I'll let Performance-Team consider whether they want to take umbrage with the graphs included here, or disagree with my conclusion.

@MarkTraceur @MBinder_WMF if we're waiting feedback from Performance shouldn't this go into "tracking" (still appearing in "code review")

@Cparle It's a good question, maybe worth raising in retro. My gut says that this task is not a Tracking task if the MM team still needs to do work on it. Tracking is commonly a place where we put tasks we want to keep an eye on but don't actually have work for.

We don't have any further work to do on this unless someone comes back and asks us to do more analysis, so I think tracking would maybe be appropriate, but it also might be fine to straight up resolve this task (as we've done our work, and despite being called out, Performance has not chimed in)

The correct tag to summon us is Performance-Team, I'm not subscribed to that more generic Performance-Team tag. I'm not worried, this isn't the only expensive type of thumbnail. That being said, I'm going to add it to the list of expensive ones, that get their own dedicated throttle, POOLCOUNTER_CONFIG_EXPENSIVE in Thumbor's configuration. Essentially that means no more than 16 STL thumbnails will be rendered at once. The same policy is currently applied to XCF, DJVU, PDF and TIFF files.

Change 381045 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/puppet@production] Thumbor: apply expensive type throttle to STL

https://gerrit.wikimedia.org/r/381045

Change 381045 merged by Filippo Giunchedi:
[operations/puppet@production] Thumbor: apply expensive type throttle to STL

https://gerrit.wikimedia.org/r/381045

Mentioned in SAL (#wikimedia-operations) [2017-10-02T13:42:32Z] <godog> roll restart thumbor to apply https://gerrit.wikimedia.org/r/#/c/381045/ - T166699

So @MarkTraceur can we close this then?