Change Details

The webperf node in Beta Cluster would be be similar to the webperf1001 and webperf1002 nodes in production. It would run coal, navtiming, and the performance site. Where coal and navtiming would consume from beta's kafka brokers, and produce to beta's statsd/graphite hosts. That should happen automatically given Hiera configuration, but would be good to confirm. The main thing that'd be useful to be able to test is the performance site. E.g. changes to its Apache configuration and other things that require puppet changes, which we can then cherry-pick ourselves to beta's puppet-master. The navtiming/coal would mostly be dormant, but that's fine. Aside from the performance site, it'd also be a useful place to be able to test upgrades to XHGui, as well as T195312 in the future. * {icon check color=green} performance::site - https://performance-beta.wmflabs.org/ * coal::web ** {icon check color=green} Check the web API is up and working. ** {icon check color=green} Fix the perfsite JS to use the relative local one instead of hardcoding the prod url. * [ ] coal::processor ** Confirm the process is consuming from beta Kafka and producing to beta Graphite. * [ ] webperf::statsv ** Check that /statsv beacons requests to Beta varnishes result in webperf/statsv writing to Beta statsd/graphite. * [ ] webperf::navtiming ** Check that NavigationTiming data from Beta EventLogging is written to Beta graphite/frontend.navtiming. * [ ] coal::processor ** Check that NavigationTiming data from Beta EventLogging is written to Beta graphite/coal. Issues: * [x] `profile::webperf` fails due to `kafka_config('jumbo-eqiad')` being production-specific. – https://gerrit.wikimedia.org/r/436435 * [x] `webperf::statsv` and `webperf::navtiming` failing due to host missing from scap::dsh. – <https://gerrit.wikimedia.org/r/436586> * [x] `webperf::statsv` and `webperf::navtiming` failing due to repos missing from scap::sources on deployment-tin. ** puppet: <https://gerrit.wikimedia.org/r/436601> ** {T196034} * [x] Puppet unable to complete `Package[statsv/statsv]` and others due to circular dependency on restarting the systemd service before it exists. ** statsv: <https://gerrit.wikimedia.org/r/436920> ** navtiming: <https://gerrit.wikimedia.org/r/436908> ** coal: <https://gerrit.wikimedia.org/r/436914> * [ ] ..

The webperf node in Beta Cluster would be be similar to the webperf1001 and webperf1002 nodes in production. It would run coal, navtiming, and the performance site. Where coal and navtiming would consume from beta's kafka brokers, and produce to beta's statsd/graphite hosts. That should happen automatically given Hiera configuration, but would be good to confirm. The main thing that'd be useful to be able to test is the performance site. E.g. changes to its Apache configuration and other things that require puppet changes, which we can then cherry-pick ourselves to beta's puppet-master. The navtiming/coal would mostly be dormant, but that's fine. Aside from the performance site, it'd also be a useful place to be able to test upgrades to XHGui, as well as T195312 in the future. * {icon check color=green} performance::site - https://performance-beta.wmflabs.org/ * coal::web ** {icon check color=green} Check the web API is up and working. ** {icon check color=green} Fix the perfsite JS to use the relative local one instead of hardcoding the prod url. * [ ] webperf::statsv ** Check that /statsv beacons requests to Beta varnishes result in webperf/statsv writing to Beta statsd/graphite. * [ ] webperf::navtiming ** Check that NavigationTiming data from Beta EventLogging is written to Beta graphite/frontend.navtiming. * [ ] coal::processor ** Check that NavigationTiming data from Beta EventLogging is written to Beta graphite/coal. Issues: * [x] `profile::webperf` fails due to `kafka_config('jumbo-eqiad')` being production-specific. – https://gerrit.wikimedia.org/r/436435 * [x] `webperf::statsv` and `webperf::navtiming` failing due to host missing from scap::dsh. – <https://gerrit.wikimedia.org/r/436586> * [x] `webperf::statsv` and `webperf::navtiming` failing due to repos missing from scap::sources on deployment-tin. ** puppet: <https://gerrit.wikimedia.org/r/436601> ** {T196034} * [x] Puppet unable to complete `Package[statsv/statsv]` and others due to circular dependency on restarting the systemd service before it exists. ** statsv: <https://gerrit.wikimedia.org/r/436920> ** navtiming: <https://gerrit.wikimedia.org/r/436908> ** coal: <https://gerrit.wikimedia.org/r/436914> * [ ] ..

The webperf node in Beta Cluster would be be similar to the webperf1001 and webperf1002 nodes in production. It would run coal, navtiming, and the performance site. Where coal and navtiming would consume from beta's kafka brokers, and produce to beta's statsd/graphite hosts. That should happen automatically given Hiera configuration, but would be good to confirm. The main thing that'd be useful to be able to test is the performance site. E.g. changes to its Apache configuration and other things that require puppet changes, which we can then cherry-pick ourselves to beta's puppet-master. The navtiming/coal would mostly be dormant, but that's fine. Aside from the performance site, it'd also be a useful place to be able to test upgrades to XHGui, as well as T195312 in the future. * {icon check color=green} performance::site - https://performance-beta.wmflabs.org/ * coal::web ** {icon check color=green} Check the web API is up and working. ** {icon check color=green} Fix the perfsite JS to use the relative local one instead of hardcoding the prod url. * [ ] coal::processor ** Confirm the process is consuming from beta Kafka and producing to beta Graphite. * [ ] webperf::statsv ** Check that /statsv beacons requests to Beta varnishes result in webperf/statsv writing to Beta statsd/graphite. * [ ] webperf::navtiming ** Check that NavigationTiming data from Beta EventLogging is written to Beta graphite/frontend.navtiming. * [ ] coal::processor ** Check that NavigationTiming data from Beta EventLogging is written to Beta graphite/coal. Issues: * [x] `profile::webperf` fails due to `kafka_config('jumbo-eqiad')` being production-specific. – https://gerrit.wikimedia.org/r/436435 * [x] `webperf::statsv` and `webperf::navtiming` failing due to host missing from scap::dsh. – <https://gerrit.wikimedia.org/r/436586> * [x] `webperf::statsv` and `webperf::navtiming` failing due to repos missing from scap::sources on deployment-tin. ** puppet: <https://gerrit.wikimedia.org/r/436601> ** {T196034} * [x] Puppet unable to complete `Package[statsv/statsv]` and others due to circular dependency on restarting the systemd service before it exists. ** statsv: <https://gerrit.wikimedia.org/r/436920> ** navtiming: <https://gerrit.wikimedia.org/r/436908> ** coal: <https://gerrit.wikimedia.org/r/436914> * [ ] ..