Change Details

In Wikimedia production, we profile MediaWiki in two ways: * Request profiling for debugging – see T206152. * Sampling profiles for production - this task. This is currently collected from all production servers/requests via the internal HHVM sampling profiler ("Xenon") and periodically collected from HHVM's parent process. Links: * <https://github.com/facebook/hhvm/wiki/Profiling#xenon> * <https://github.com/brendangregg/FlameGraph> * <https://performance.wikimedia.org/xenon/> * Instrumentation code: ** [wmf-config/profiler.php](https://github.com/wikimedia/operations-mediawiki-config/blob/7ebd5b15a1ec690100eb1f2d3106acf4bf6ffa7d/wmf-config/profiler.php#L175-L236). ** [hhvm/server.ini#hhvm.xenon.period](https://github.com/wikimedia/puppet/blob/94c4f5a41d97a8931d22843192717f01b414bf42/modules/mediawiki/manifests/hhvm.pp#L24-L33) The Xenon sampling profiler is enabled by default for us in production for all requests. There is no run-time toggle. During request shutdown, we ask HHVM for data. The data is internally buffered by HHVM which means it usually doesn't return anything yet, and then at certain intervals it yields a buffer of what it's got since the last flush, which we send to a Redis queue from which we then produce aggregated log files and flame graphs, published at <https://performance.wikimedia.org/. High-level tasks: * [ ] Find or create a sampling profiler for PHP 7. – T205059 * [ ] Install the profiler in production. * [x] Generalise the arc-lamp ingestion pipeline to support a second queue. – T195312 * [ ] Set up the second consumer queue. * [ ] Enable capturing and flushing of PHP 7 samples to this queue.

In Wikimedia production, we profile MediaWiki in two ways: * Request profiling for debugging – see T206152. * Sampling profiles for production - this task. This is currently collected from all production servers/requests via the internal HHVM sampling profiler ("Xenon") and periodically collected from HHVM's parent process. Links: * <https://github.com/facebook/hhvm/wiki/Profiling#xenon> * <https://github.com/brendangregg/FlameGraph> * <https://performance.wikimedia.org/xenon/> * Instrumentation code: ** [wmf-config/profiler.php](https://github.com/wikimedia/operations-mediawiki-config/blob/7ebd5b15a1ec690100eb1f2d3106acf4bf6ffa7d/wmf-config/profiler.php#L175-L236). ** [hhvm/server.ini#hhvm.xenon.period](https://github.com/wikimedia/puppet/blob/94c4f5a41d97a8931d22843192717f01b414bf42/modules/mediawiki/manifests/hhvm.pp#L24-L33) The Xenon sampling profiler is enabled by default for us in production for all requests. There is no run-time toggle. During request shutdown, we ask HHVM for data. The data is internally buffered by HHVM which means it usually doesn't return anything yet, and then at certain intervals it yields a buffer of what it's got since the last flush, which we send to a Redis queue from which we then produce aggregated log files and flame graphs, published at <https://performance.wikimedia.org/. High-level tasks: * [x] Find or create a sampling profiler for PHP 7. – T205059 * [x] Install the profiler in production. * [x] Generalise the arc-lamp ingestion pipeline to support a second queue. – T195312 * [ ] Set up the second consumer queue. * [ ] Enable capturing and flushing of PHP 7 samples to this queue.

In Wikimedia production, we profile MediaWiki in two ways: * Request profiling for debugging – see T206152. * Sampling profiles for production - this task. This is currently collected from all production servers/requests via the internal HHVM sampling profiler ("Xenon") and periodically collected from HHVM's parent process. Links: * <https://github.com/facebook/hhvm/wiki/Profiling#xenon> * <https://github.com/brendangregg/FlameGraph> * <https://performance.wikimedia.org/xenon/> * Instrumentation code: ** [wmf-config/profiler.php](https://github.com/wikimedia/operations-mediawiki-config/blob/7ebd5b15a1ec690100eb1f2d3106acf4bf6ffa7d/wmf-config/profiler.php#L175-L236). ** [hhvm/server.ini#hhvm.xenon.period](https://github.com/wikimedia/puppet/blob/94c4f5a41d97a8931d22843192717f01b414bf42/modules/mediawiki/manifests/hhvm.pp#L24-L33) The Xenon sampling profiler is enabled by default for us in production for all requests. There is no run-time toggle. During request shutdown, we ask HHVM for data. The data is internally buffered by HHVM which means it usually doesn't return anything yet, and then at certain intervals it yields a buffer of what it's got since the last flush, which we send to a Redis queue from which we then produce aggregated log files and flame graphs, published at <https://performance.wikimedia.org/. High-level tasks: * [ * [x] Find or create a sampling profiler for PHP 7. – T205059 * [ * [x] Install the profiler in production. * [x] Generalise the arc-lamp ingestion pipeline to support a second queue. – T195312 * [ ] Set up the second consumer queue. * [ ] Enable capturing and flushing of PHP 7 samples to this queue.