Page MenuHomePhabricator

Investigate which xhgui pages are expensive and block them from public access
Closed, InvalidPublic

Description

Bots are probably responsible for hits on expensive pages that are making mongodb quite busy. We should either block those urls or put them behind login.

  • XHGui: Loading of profiling history is not working. Regular profiling details work, and the overview page of all profiles also works. But the history of one url seems to fail. Might be a pre-existing issue, though. (e.g. when loading https://performance.wikimedia.org/xhgui/url/view?url=%2F%2FWK33-ApAIHwAAC7E6gQAAAAQ%2Fw%2Fload.php - It claims to be caused by either MongoDB credentials being wrong, MonoDB not running, or the cache directory not being writeable.)

[..] I'll assume that never worked. It's not an important aspect of XHGui anyway. I initially thought that that error happened on all XHGui, which would've been a regression, but other queries work fine. Probably the db is getting too big for generic queries like on that page.

Event Timeline

Gilles created this task.Mar 1 2017, 5:45 PM
Krinkle triaged this task as Low priority.Mar 9 2017, 9:54 PM
Krinkle updated the task description. (Show Details)Mar 9 2017, 10:31 PM
Krinkle closed this task as Invalid.Mar 25 2017, 9:26 PM

Per T161196, it turns out all these views work quite well as long we do don't pump it full with too large of a data sets. I've disabled the production sampling for now and cleared the database per T161196: tungsten is out of space on /srv.

We should be able to keep manual XMD profiles for a near-indefinite period of time given the 1TB space available. If we do want to enabled production sampling again, we should clear these non-XMD profiles within a set time period (e.g. a nightly cron that deletes all non-XMD entries in mongo.xhprof.results more than 7 days old or something like that).

Given the db now performs properly we don't need to disable any urls or restrict access to avoid overload.