hhvm-staging.hhvm.eqiad.wmflabs has high system and user CPU. Most probably there is a wild process on it ?
Would be great to clean up and reclaim some CPU capacity :)
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Andrew | T179378 some labvirt servers are at full CPU capacity | |||
| Resolved | bd808 | T179380 hhvm-staging.hhvm.eqiad.wmflabs has high user/system CPU |
Event Timeline
Comment Actions
I never used that VM and we don't have any HHVM hosts in production apart from silver, so from my PoV this can be dropped.
Comment Actions
The root partition here was full. That looks to have been caused by the hhvm-alt upstart service being in an infinite restart loop. That in turn seems to be caused by permissions with the /run directory. I cleaned up a bunch of logs and then shutdown the hhvm-alt process with service hhvm-alt stop. The load avg has dropped to 0.02.
I think that @MoritzMuehlenhoff probably meant that we don't have (many) Trusty HHVM hosts (?).
hhvm-staging.hhvm.eqiad.wmflabs VM was built by @Joe on 2016-02-29. It looks like he and Ori are the only folks who have ever logged in there as non-root users. The latest artifacts there look to be from around 2017-01-20.
Comment Actions
That's what I meant; all HHVM hosts in production are running jessie for a while now, the only exception still on trusty is silver at this point (with a forthcoming replacement as labweb*).
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2017-11-16T19:03:06Z] <bd808> Joined project as admin to cleanup vm (T179380)
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2017-11-16T19:06:33Z] <bd808> Deleted hhvm-staging.hhvm.eqiad.wmflabs (T179380)