Thu, Feb 21
Feb 7 2019
I don't understand the preference for sampling Swift requests rather than Varnish requests. You'd have greater resilience to overload (for the reasons I cited above), and you'd have looser coupling and better fault tolerance by building on top of kafka.
Feb 6 2019
Feb 5 2019
The worry I had was this: a thumbnail that is requested once a minute on average probably has an approximately similar varnish cache hit rate to a thumbnail that is requested a hundred times per second. If that's true, then both thumbnails would be retrieved from Swift about as often, and they would be equally likely to have their expiry renewed before they're vacuumed up. This increases the risk of overload in case of a varnish cold restart or failure.
(It might have to be X% of access on varnish, since I assume the most oft-requested thumbnails enjoy a very high varnish cache hit rate. You could do this asynchronously by having a daemon that samples thumbnail requests from the varnishkafka stream.)
It seems that Swift has built-in support for object expiration, which can be requested by setting a header (either X-Delete-After or X-Delete-At).
It also looks like the expiry can be re-set, either by first removing it via X-Remove-Delete-At, and then setting it anew, or by updating the metadata in-place.
Is there a reason why these mechanisms are not under consideration?
What orders of magnitude are we talking about with respect to: total number of thumbnails in Swift, and number of thumbnail requests per second?
Oct 3 2018
Tim's rationale for the callback interface makes sense to me. Thanks for the explanation.
Sep 24 2018
I wonder if a callback is the right choice, particularly for the periodic mode. Presumably you'd want the processing of samples to occur after response data has been flushed, in a context that is healthy (e.g., you can make use of storage and logging facilities, and you are not on the cusp of hitting the timeout). But if the callback can be called because the sample buffer is full, then the call could come at any time, no? Wouldn't it be better to leave it to PHP code to call $excimer->getLog(), and have that return null if the log is empty?
Aug 27 2018
Aug 23 2018
Aug 15 2018
@Krinkle OK, that makes sense to me. As a lead-up to that, would it make sense to run a brief study in which we log an event when an event fails validation on the client? The code for logging of the 'validation failed' event could use an alternate, light-weight code-path so we don't end up wondering how many 'validation failed' events failed validation.
Aug 7 2018
I think that Nuria was right to press for an evidence-based rationale, and I haven't seen one.
Aug 2 2018
Jul 23 2018
It'd be nifty if you got backtraces from Lua, too. Luasandbox is about ~7% of wall time.
Jul 22 2018
According to https://www.mediawiki.org/wiki/Manual:Developing_libraries#Bootstrapping_a_new_library generator-wikimedia-php-library is deprecated in favor of cookiecutter-library.
Tom, are you sure you meant to send this to me? I am not involved with the maintenance of the PageForms extension. Maybe you meant @Yaron_Koren ?
I'm not working on this, and haven't kept up with the relevant web standards, so I'm unlikely to be of much use here.
AFAICT, the base VM for MediaWiki-Vagrant is Debian Stretch, which has pip 9: https://packages.debian.org/stretch/python-pip
Jul 17 2018
Jun 26 2018
May 23 2018
Using gdb will likely have a much smaller overhead.
May 11 2018
Mar 8 2018
This is really impressive, Krinkle. Kudos.
Dec 7 2017
Nov 16 2017
Nov 9 2017
Oct 23 2017
There is only a single entry in autoloader.log for the past three days:
Oct 20 2017
Oct 19 2017
Oct 12 2017
Jul 20 2017
Jul 10 2017
Jun 13 2017
I know you are not 100% done, but still: congratulations, Sirs! It's really impressive to see a long arc of design and implementation work and operational planning come to fruition. Kudos!
Jun 2 2017
Since this code is hot, I assume AutoLoader.php runs before the Composer autoloader, and that the case manipulation happens for every class that lives in an external depedency. If that is true, then it might be good idea to relegate the code that does the case manipulation to a discrete legacy autoloader that runs last.