Gilles (Gilles Dubuc)
Senior Performance Engineer, WMF

Projects (18)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 7 2014, 8:24 AM (128 w, 6 d)
Availability
Available
IRC Nick
gilles
LDAP User
Gilles
MediaWiki User
Unknown

Recent Activity

Today

Gilles added a project to T161531: Track time spent in Thumbor subprocesses with header(s): Patch-For-Review.
Mon, Mar 27, 8:40 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles added a revision to T161531: Track time spent in Thumbor subprocesses with header(s): D607: Report duration of all subprocess commands as debug headers.
Mon, Mar 27, 8:40 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles created D607: Report duration of all subprocess commands as debug headers.
Mon, Mar 27, 8:40 PM
Gilles triaged T161531: Track time spent in Thumbor subprocesses with header(s) as "Normal" priority.
Mon, Mar 27, 8:40 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

One should bear in mind that passing a request to the next server is only possible if nothing has been sent to a client yet. That is, if an error or timeout occurs in the middle of the transferring of a response, fixing this is impossible.

Mon, Mar 27, 7:46 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

Ah, there is a default for that directive:

Mon, Mar 27, 7:38 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

So, in theory our nginx config retries on the next upstream:

Mon, Mar 27, 7:30 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles renamed T150746: Nginx timeouts on Thumbor from "Thumbor inexplicably 504s intermittently on files that render fine later" to "Nginx timeouts on Thumbor".
Mon, Mar 27, 7:26 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles added a revision to T161535: Track nginx request time in Thumbor debug headers: D606: Include request date header given by nginx in the response.
Mon, Mar 27, 5:00 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles created D606: Include request date header given by nginx in the response.
Mon, Mar 27, 5:00 PM
Gilles removed a project from T161535: Track nginx request time in Thumbor debug headers: Operations.
Mon, Mar 27, 4:56 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles added a subtask for T150746: Nginx timeouts on Thumbor: T161531: Track time spent in Thumbor subprocesses with header(s).
Mon, Mar 27, 4:56 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles added a parent task for T161531: Track time spent in Thumbor subprocesses with header(s): T150746: Nginx timeouts on Thumbor.
Mon, Mar 27, 4:56 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles created T161535: Track nginx request time in Thumbor debug headers.
Mon, Mar 27, 4:56 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

To verify my theory, I would have to be able to log requests to Thumbor when they come in. Since Thumbor is single-threaded I doubt that it's itself capable of such logging. When Thumbor gets to pick up the request, it's probably been waiting for some time in Tornado. I wonder if nginx can add a header to its request before passing it to the upstream? That would be a foolproof way to take into account any queueing delay experienced at the Thumbor level.

Mon, Mar 27, 4:54 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

Looking at the nginx logs on thumbor1001, I notice that some of the timeouts are for files that don't exist. Example: https://upload.wikimedia.org/wikipedia/en/thumb/5/5a/Premier_League.svg/125px-Premier_League.svg.png

Mon, Mar 27, 4:38 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles created T161531: Track time spent in Thumbor subprocesses with header(s).
Mon, Mar 27, 4:32 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

Things don't seem to have improved :(

Mon, Mar 27, 3:45 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles closed T153041: Alerts using WebPageTest as "Resolved".

Closing this as I think we're done with these, only the alert emailing remains, which is tracked on the Nagios task.

Mon, Mar 27, 2:01 PM · Patch-For-Review, Performance-Team, WebPageTest
Gilles closed T153041: Alerts using WebPageTest , a subtask of T153166: Set up Grafana alerts for Web Performance metrics, as "Resolved".
Mon, Mar 27, 2:01 PM · Epic, Performance-Team
Gilles removed a project from T151553: Add request URL to thumbor errors: Patch-For-Review.
Mon, Mar 27, 2:00 PM · Operations, Performance-Team, Thumbor
Gilles moved T151553: Add request URL to thumbor errors from Blocked to Doing on the Performance-Team board.
Mon, Mar 27, 2:00 PM · Operations, Performance-Team, Thumbor
Gilles moved T160670: Point Thumbor to https Swift from Blocked to Doing on the Performance-Team board.
Mon, Mar 27, 2:00 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles moved T156245: Create Nagios Grafana alert checks from Blocked to Doing on the Performance-Team board.
Mon, Mar 27, 2:00 PM · Performance-Team
Gilles removed a project from T156245: Create Nagios Grafana alert checks: Patch-For-Review.
Mon, Mar 27, 1:59 PM · Performance-Team
Gilles added a comment to T156245: Create Nagios Grafana alert checks.

Looking at https://grafana.wikimedia.org/dashboard/db/webpagetest-alerts?refresh=5m&panelId=49&fullscreen&orgId=1&from=now-7d&to=now we can indeed see that the dashboard was alerting at the time the puppet change to forward alerts was merged and that it recovered exactly when the recovery message made it to IRC. So, it works! Left a comment on the changeset about the mailing list not working: https://gerrit.wikimedia.org/r/#/c/342431/

Mon, Mar 27, 1:59 PM · Performance-Team
Gilles added a comment to T156245: Create Nagios Grafana alert checks.

This is set up now. The IRC component works on #wikimedia-perf-bots:

Mon, Mar 27, 1:50 PM · Performance-Team
Gilles committed rTHMBREXTb76fce3e2cee: Add request URL context to logging (authored by Gilles).
Add request URL context to logging
Mon, Mar 27, 1:43 PM
Gilles added a commit to T151553: Add request URL to thumbor errors: rTHMBREXTb76fce3e2cee: Add request URL context to logging.
Mon, Mar 27, 1:43 PM · Operations, Performance-Team, Thumbor
Gilles closed D603: Add request URL context to logging by committing rTHMBREXTb76fce3e2cee: Add request URL context to logging.
Mon, Mar 27, 1:43 PM
Gilles added a comment to D603: Add request URL context to logging.

And adding it to the config before the custom formatter with is fallback is there causes exceptions.

Mon, Mar 27, 1:42 PM
Gilles added a comment to D603: Add request URL context to logging.

Yeah, that's added through config.

Mon, Mar 27, 1:38 PM
Gilles added a comment to T72932: MediaViewer does not support IE8.

If you abandon a task long enough... ;)

Mon, Mar 27, 12:49 PM · Patch-For-Review, MediaWiki-extensions-MultimediaViewer
Gilles added a comment to T70861: Performance review #2 of Hovercards (Popups extension).

I take issue with the idea of delaying display of data once it's available, through animation or otherwise. When you load an article page, we don't make the article bounce into place for aesthetics. Unless proven otherwise artificial delays and animations have no justification. You find that research cost is too high? Fine, reduce the artificial delays and remove the animation without research. That what we're requesting, as "guardians" of site performance. The burden of proving that extra delays and animations have a measurable benefit is on you. It's not for us to prove the opposite, because there's extensive public research about the fact that hundreds of milliseconds in content visibility matter a great deal in terms of user engagement. It's always possible that for a specific feature might perform better with an artificial delay, but that goes against what is known about performance, and this requires to be demonstrated, not based on a designer's hunch.

Mon, Mar 27, 12:39 PM · Page-Previews (2016-17-Q3-Goal), Performance, Traffic, Operations, Reading-Web-Backlog, Performance-Team, Wikimedia-Extension-setup

Fri, Mar 24

Gilles added a comment to T70861: Performance review #2 of Hovercards (Popups extension).

Accidental UI popups are major point of frustration for some.

Fri, Mar 24, 9:38 AM · Page-Previews (2016-17-Q3-Goal), Performance, Traffic, Operations, Reading-Web-Backlog, Performance-Team, Wikimedia-Extension-setup
Gilles removed a project from T150746: Nginx timeouts on Thumbor: Patch-For-Review.
Fri, Mar 24, 9:19 AM · Patch-For-Review, Operations, Performance-Team, Thumbor

Thu, Mar 23

Gilles moved T160678: Reduce ChronologyProtector init cost for load.php from Inbox to Next-up on the Performance-Team board.
Thu, Mar 23, 9:08 PM · Patch-For-Review, MediaWiki-extensions-WikibaseClient, Wikidata, Performance-Team, MediaWiki-ResourceLoader, MediaWiki-Database
Gilles moved T160685: Increase $wgExpensiveParserFunctionLimit on nowiki from Inbox to Radar on the Performance-Team board.
Thu, Mar 23, 9:05 PM · Wikidata, Performance-Team, Operations, Performance, Wikimedia-Site-requests
Gilles moved T160834: Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG from Inbox to Radar on the Performance-Team board.
Thu, Mar 23, 9:01 PM · Upstream, Performance-Team, Wikimedia-General-or-Unknown, Commons, MediaWiki-File-management, Multimedia
Gilles moved T160953: Adopt direct use of ES5 methods in core from Inbox to Blocked on the Performance-Team board.
Thu, Mar 23, 9:00 PM · Patch-For-Review, Performance-Team, JavaScript, MediaWiki-General-or-Unknown
Gilles closed T110361: Weekly speed reports for external stakeholders as "Declined".
Thu, Mar 23, 8:58 PM · Performance-Team
Gilles moved T147569: Evaluate/Deploy TCP BBR when available (kernel 4.9+) from Inbox to Radar on the Performance-Team board.
Thu, Mar 23, 8:54 PM · Performance-Team, Operations, Traffic
Gilles moved T161192: Use the sampling profiler for JS call stacks from Inbox to Radar on the Performance-Team board.
Thu, Mar 23, 8:52 PM · Performance-Team, WebPageTest, Upstream
Gilles moved T161193: Upgrade to Grafana 4.2.0 from Inbox to Radar on the Performance-Team board.
Thu, Mar 23, 8:51 PM · User-fgiunchedi, Performance-Team, Operations
Gilles moved T161195: Write upstream test for Thumbor's 500 on loader failures PR from Inbox to Next-up on the Performance-Team board.
Thu, Mar 23, 8:51 PM · Performance-Team, Thumbor
Gilles moved T161196: tungsten is out of space on /srv from Inbox to Next-up on the Performance-Team board.
Thu, Mar 23, 8:50 PM · Performance-Team
Gilles moved T151553: Add request URL to thumbor errors from Backlog to Blocked on the Performance-Team board.
Thu, Mar 23, 2:39 PM · Operations, Performance-Team, Thumbor
Gilles added a project to T151553: Add request URL to thumbor errors: Patch-For-Review.
Thu, Mar 23, 2:39 PM · Operations, Performance-Team, Thumbor
Gilles created D603: Add request URL context to logging.
Thu, Mar 23, 2:39 PM
Gilles added a revision to T151553: Add request URL to thumbor errors: D603: Add request URL context to logging.
Thu, Mar 23, 2:39 PM · Operations, Performance-Team, Thumbor
Gilles added a project to T151456: Thumbor original file download limit should be 4GB: Patch-For-Review.
Thu, Mar 23, 12:23 PM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles removed a project from T159358: Get rid of lingering temp files following ImageMagick subprocess update of Thumbor: Patch-For-Review.
Thu, Mar 23, 12:04 PM · Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

Actually, nginx was already at 90s, but I think we can double that and see the effect, because it might not be enough as-is to accomodate the worst case scenarios.

Thu, Mar 23, 11:54 AM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles added a comment to T150746: Nginx timeouts on Thumbor.

It seems like Mediawiki 200ing while Thumbor 504s has about halved:

Thu, Mar 23, 11:01 AM · Patch-For-Review, Operations, Performance-Team, Thumbor
Gilles closed T149328: Generate systematic traffic to test Thumbor as "Resolved".

Ran the script above for 10 minutes and compared it to a baseline of organic traffic.

Thu, Mar 23, 10:51 AM · Performance-Team, Thumbor
Gilles closed T114518: Investigate PNG metadata in thumbnails as "Resolved".

We figured out what's filtered and what could be, I think that "going the extra mile", if it's possible, should be a separate task.

Thu, Mar 23, 9:22 AM · Commons, MediaWiki-File-management, Multimedia, Performance-Team
Gilles closed T114518: Investigate PNG metadata in thumbnails, a subtask of T111633: Optimize thumbs right after creation, as "Resolved".
Thu, Mar 23, 9:22 AM · Commons, MediaWiki-File-management, Multimedia
Gilles edited the description of T161195: Write upstream test for Thumbor's 500 on loader failures PR.
Thu, Mar 23, 9:20 AM · Performance-Team, Thumbor
Gilles triaged T161195: Write upstream test for Thumbor's 500 on loader failures PR as "Low" priority.
Thu, Mar 23, 9:19 AM · Performance-Team, Thumbor
Gilles created T161195: Write upstream test for Thumbor's 500 on loader failures PR.
Thu, Mar 23, 9:19 AM · Performance-Team, Thumbor
Gilles added a comment to T153048: Write blog post about the "hidden browser tabs incident".

Chrome 57 was released earlier this month: https://developers.google.com/web/updates/2017/03/background_tabs

Thu, Mar 23, 9:13 AM · Performance-Team

Tue, Mar 21

Gilles added a project to T147569: Evaluate/Deploy TCP BBR when available (kernel 4.9+): Performance-Team.
Tue, Mar 21, 5:00 PM · Performance-Team, Operations, Traffic
Gilles added a comment to T147569: Evaluate/Deploy TCP BBR when available (kernel 4.9+).

On the tech-mgmt meeting you mentioned this was underway, is there another phab task for it?

Tue, Mar 21, 4:34 PM · Performance-Team, Operations, Traffic
Gilles added a comment to T149328: Generate systematic traffic to test Thumbor.

Wrote this dumb little thing: P5097 which seems to do the job.

Tue, Mar 21, 2:14 PM · Performance-Team, Thumbor
Gilles created P5097 thumb-hammer.py.
Tue, Mar 21, 2:13 PM · Performance-Team, Thumbor
Gilles moved T149328: Generate systematic traffic to test Thumbor from Backlog to Doing on the Performance-Team board.
Tue, Mar 21, 2:12 PM · Performance-Team, Thumbor
Gilles removed a project from T121388: Service-based thumbnailing re-architecture in production with Thumbor: Patch-For-Review.
Tue, Mar 21, 10:23 AM · Performance-Team, Thumbor
Gilles added a project to T156245: Create Nagios Grafana alert checks: Patch-For-Review.
Tue, Mar 21, 10:22 AM · Performance-Team
Gilles added a project to T150734: Make Thumbor logs available in ELK: Patch-For-Review.
Tue, Mar 21, 10:21 AM · User-fgiunchedi, Patch-For-Review, Performance-Team, Thumbor
Gilles moved T150734: Make Thumbor logs available in ELK from Doing to Blocked on the Performance-Team board.
Tue, Mar 21, 10:20 AM · User-fgiunchedi, Patch-For-Review, Performance-Team, Thumbor
Gilles added a comment to T150734: Make Thumbor logs available in ELK.

Debian package ready at https://github.com/gi11es/thumbor-debian/tree/master/python-logstash @fgiunchedi please review and add to our production apt repos when you can.

Tue, Mar 21, 10:20 AM · User-fgiunchedi, Patch-For-Review, Performance-Team, Thumbor
Gilles added a comment to T157111: [8 hours] Create a Page Previews performance dashboard.

Looking at the API response time graph, I would advise to separate out the p95 into a separate one. Putting the median and the 95 on the same scale means that you have to mouse over to get a sense of whether the median is moving, because it's so small it's hidden by the p95. Similarly, the median seems to have high variance. I understand that you're going to increase the sampling rate. I think you should also use a moving average, which makes for a more readable graph when there's quite a bit of variance in the measure. Also something that we do in places is graph the amount of samples a given data point is based on.

Tue, Mar 21, 9:35 AM · Patch-For-Review, Reading-Web-Sprint-94, MW-1.29-release (WMF-deploy-2017-03-14_(1.29.0-wmf.16)), Reading-Web-Sprint-93-🔍🔍🔍🔍🔍, Page-Previews (2016-17-Q3-Goal), Reading-Web-Sprint-92-🍜, Reading-Web-Backlog
Gilles added a comment to T160834: Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG.

I don't think there's ever a case where it's useful for us to serve 4:4:4 jpegs. Even if Guetzli makes them "less expensive", at equal quality in my small test above it's still 9% bigger for absolutely no objective higher quality than a 4:2:0 quality equivalent. And that's at 95 quality in Guetzli on the reference image they provide, which is their advertised objective. So even for what it intends to do, it falls short compared to 4:2:0 images on objectively-measured visual differences (which includes color).

Tue, Mar 21, 8:30 AM · Upstream, Performance-Team, Wikimedia-General-or-Unknown, Commons, MediaWiki-File-management, Multimedia

Mon, Mar 20

Gilles added a comment to T160563: Provide infrastructure for embedding styles in the content HTML with deduplication.

Good point, so you're talking about taking those styles statements that are in the head solely because of our no-JS support, and wouldn't need to be in the head if it wasn't for that, and bringing them inline inside the page?

Mon, Mar 20, 1:42 PM · MobileFrontend, Mobile-Content-Service, Performance, Parsoid, MediaWiki-Parser, MediaWiki-ResourceLoader
Gilles added a comment to T160834: Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG.

We've talked about doing a later more expensive pass to optimize images further for other file formats (PNG, I believe), but it's a project of its own. And I think we have yet to stumble into gains big enough that would apply to a large enough chunk of our content that it would be something that gets picked up as a project.

Mon, Mar 20, 1:36 PM · Upstream, Performance-Team, Wikimedia-General-or-Unknown, Commons, MediaWiki-File-management, Multimedia
Gilles added a comment to T160834: Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG.

Quick test using https://github.com/jterrace/pyssim and the "sample high quality image" Guetzli linked to on their github.

Mon, Mar 20, 1:30 PM · Upstream, Performance-Team, Wikimedia-General-or-Unknown, Commons, MediaWiki-File-management, Multimedia
Gilles added a comment to T160738: Increase Grafana user rights for Performance team members.

Possibly an upstream bug, then, that the auth level is so high for that feature?

Mon, Mar 20, 12:22 PM · Performance-Team, Operations, Graphite
Gilles added a comment to T160834: Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG.

Also, from their github page:

Mon, Mar 20, 12:07 PM · Upstream, Performance-Team, Wikimedia-General-or-Unknown, Commons, MediaWiki-File-management, Multimedia
Gilles added a comment to T160738: Increase Grafana user rights for Performance team members.

I appear to be able to delete one I've just created. @Peter which way are you trying to delete the alert and failing to do so? Can you take screenshots?

Mon, Mar 20, 11:52 AM · Performance-Team, Operations, Graphite
Gilles edited projects for T160834: Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG, added: Performance-Team; removed Performance.
Mon, Mar 20, 10:08 AM · Upstream, Performance-Team, Wikimedia-General-or-Unknown, Commons, MediaWiki-File-management, Multimedia
Gilles added a comment to T160834: Investigate deploying Google's Guetzli to further reduce thumbnail size of JPEG.

I'm initially skeptical of these claims. MozJPEG had similar ones that turned out to be false when put to the test with our compression parameters.

Mon, Mar 20, 10:06 AM · Upstream, Performance-Team, Wikimedia-General-or-Unknown, Commons, MediaWiki-File-management, Multimedia
Gilles added a comment to T160563: Provide infrastructure for embedding styles in the content HTML with deduplication.

I agree with @Krinkle's position. You took my recommendation that was about TemplateStyles out of context. We never said styles embedding should be a solution for everything. User-generated styles and extension styles are different. I expect extension styles to be able to modify anything on the page, including the chrome, while template styles should only affect the template contents. This is the main reason why the technical compromise for TemplateStyles can't be generalized to both situations. Their scope is fundamentally different. I can't speak for each extension listed because I don't know them all, but you're trying to solve something that isn't broken. Extensions should already only include with addModuleStyles things that *have to* be in the head to avoid FOUC, otherwise they should use addModule. There's no need to change that and the benefits of cross-page caching for using RL is high for extensions, that tend to apply (and need) their styles to whole namespaces or even all pages. Whereas TemplateStyles will create a much more scattered usage pattern of unknown distribution.

Mon, Mar 20, 9:05 AM · MobileFrontend, Mobile-Content-Service, Performance, Parsoid, MediaWiki-Parser, MediaWiki-ResourceLoader

Fri, Mar 17

Gilles added a comment to T160738: Increase Grafana user rights for Performance team members.

I can't find where that's defined in Puppet.

Fri, Mar 17, 9:49 AM · Performance-Team, Operations, Graphite
Gilles created T160738: Increase Grafana user rights for Performance team members.
Fri, Mar 17, 9:45 AM · Performance-Team, Operations, Graphite
Gilles created T160736: Requesting access to perf-roots for gilles.
Fri, Mar 17, 9:38 AM · Operations, Ops-Access-Requests
Gilles renamed T160670: Point Thumbor to https Swift from "Point to Thumbor to https Swift" to "Point Thumbor to https Swift".
Fri, Mar 17, 9:31 AM · Patch-For-Review, Performance-Team, Thumbor
Gilles moved T160670: Point Thumbor to https Swift from Inbox to Blocked on the Performance-Team board.
Fri, Mar 17, 9:23 AM · Patch-For-Review, Performance-Team, Thumbor
Gilles triaged T160670: Point Thumbor to https Swift as "Normal" priority.
Fri, Mar 17, 9:23 AM · Patch-For-Review, Performance-Team, Thumbor
Gilles added a comment to T153169: Grafana alerts for Navigation Timing and related.

You can create alerts but not delete them?

Fri, Mar 17, 9:07 AM · Performance-Team
Gilles committed rTHMBREXTc446428f34d1: Add comment about limitation recently fixed upstream (authored by Gilles).
Add comment about limitation recently fixed upstream
Fri, Mar 17, 8:46 AM
Gilles added a comment to T153169: Grafana alerts for Navigation Timing and related.

Yeah, that sounds good to me

Fri, Mar 17, 7:32 AM · Performance-Team

Thu, Mar 16

Gilles added a comment to T98831: Honor DNT header for access logs & varnish logs.

Aggregate counts isn't problematic, but the data we store is. If it's recorded, it can be compromised. I know we have retention policies, etc. but they're of no use if we someone gets access to our data. 60 days is a lot.

Thu, Mar 16, 9:02 PM · WMF-Legal, Analytics, Operations, Privacy
Gilles created T160670: Point Thumbor to https Swift.
Thu, Mar 16, 5:38 PM · Patch-For-Review, Performance-Team, Thumbor
Gilles added a comment to T154906: Jessie branch fails to run apt, puppet after VM is reloaded.

Added that snippet to ~/.vagrant.d/Vagrantfile and it worked, thanks!

Thu, Mar 16, 3:43 PM · MediaWiki-Vagrant
Gilles added a comment to T154906: Jessie branch fails to run apt, puppet after VM is reloaded.

Fresh VM without roles, then:

Thu, Mar 16, 2:00 PM · MediaWiki-Vagrant
Gilles reopened T154906: Jessie branch fails to run apt, puppet after VM is reloaded as "Open".

This seems to be back. Repro steps using the Varnish role still "work".

Thu, Mar 16, 1:56 PM · MediaWiki-Vagrant
Gilles reopened T154906: Jessie branch fails to run apt, puppet after VM is reloaded, a subtask of T136429: [EPIC] Migrate base image to Debian Jessie, as "Open".
Thu, Mar 16, 1:56 PM · Patch-For-Review, Operations, Release-Engineering-Team, Epic, MediaWiki-Vagrant
Gilles committed rTHMBREXT3fc3ead39d39: The engine is responsible for deleting the source temp file no matter what (authored by Gilles).
The engine is responsible for deleting the source temp file no matter what
Thu, Mar 16, 10:03 AM
Gilles added a commit to T159358: Get rid of lingering temp files following ImageMagick subprocess update of Thumbor: rTHMBREXT3fc3ead39d39: The engine is responsible for deleting the source temp file no matter what.
Thu, Mar 16, 10:03 AM · Performance-Team, Thumbor
Gilles closed D599: The engine is responsible for deleting the source temp file no matter what by committing rTHMBREXT3fc3ead39d39: The engine is responsible for deleting the source temp file no matter what.
Thu, Mar 16, 10:03 AM