Change 383916 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/puppet@production] Add profiles/roles for stats/ML on Wikimedia Cloud

https://gerrit.wikimedia.org/r/383916

Change 383916 merged by Gehel:
[operations/puppet@production] Add profiles/roles for stats/ML on Wikimedia Cloud

https://gerrit.wikimedia.org/r/383916

@Gehel: getting the following errors. test-bayes, test-learner, test-deep-learner, test-forecaster, and test-allstar-cruncher all use Stretch.

From role::discovery::learner:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type r_land::bioc at /etc/puppet/modules/profile/manifests/discovery_computing/machine_learning.pp:44 on node test-learner.discovery-stats.eqiad.wmflabs
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Not sure what's wrong.

From role::discovery::deep_learner:

Error: Failed to apply catalog: Parameter provider failed on Package[tensorflow]: Invalid package provider 'pip3' at /etc/puppet/modules/profile/manifests/discovery_computing/deep_learning.pp:30

Weird! I guess pip3 is not available as a provider in the outdated version of puppet we use?

From role::discovery::bayes and role::discovery::forecaster:

Error: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install python-pymc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Package python-pymc is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'python-pymc' has no installation candidate
Error: /Stage[main]/Packages::Python_pymc/Package[python-pymc]/ensure: change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install python-pymc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Package python-pymc is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'python-pymc' has no installation candidate

I'll be removing python-pymc because I didn't realize it was only available for Jessie.

Change 386271 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/puppet@production] role::discovery: Fix stats/ML classes

https://gerrit.wikimedia.org/r/386271

Change 386271 merged by Gehel:
[operations/puppet@production] role::discovery: Fix stats/ML classes

https://gerrit.wikimedia.org/r/386271

Change 386536 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/puppet@production] R, Shiny Server, and Discovery Computing fixes

https://gerrit.wikimedia.org/r/386536

Change 386536 merged by Gehel:
[operations/puppet@production] R, Shiny Server, and Discovery Computing fixes

https://gerrit.wikimedia.org/r/386536

Change 386669 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/puppet@production] role::discovery::learner: Remove deep learning profile

https://gerrit.wikimedia.org/r/386669

Change 386669 merged by Gehel:
[operations/puppet@production] role::discovery::learner: Remove deep learning profile

https://gerrit.wikimedia.org/r/386669

• discovery::deep_learner role works (makes TensorFlow, Keras, and Caffe available, including the R wrappers for TF & Keras)
• discovery::learner role works (makes a bunch of ML packages available – in R & Python)
• discovery::forecaster role works (makes a bunch of time series / forecasting R packages available: prophet, bsts, forecast)
• discovery::bayes role mostly works (for some reason it doesn't install rstan but discovery::forecaster – which also includes the discovery_computing::bayesian_statistics profile – does)
• discovery::allstar_cruncher role works (multi-purpose, has ALL of the packages from the other roles, so get your game on, go play)