Page MenuHomePhabricator

Upload shiny-server .deb to our Stretch apt repository
Closed, ResolvedPublic

Description

RStudio only provides Shiny Server as a package for 64-bit Ubuntu 14.04 or later, but it seems that quite a few people use the exact same package for Debian.[citation needed] If we can't use the provided package, we would need to make our own.

Instructions for building Shiny Server from source: https://github.com/rstudio/shiny-server/wiki/Building-Shiny-Server-from-Source

Details

Related Gerrit Patches:
operations/puppet : productionshiny_server: change gfortran/g++ dep

Event Timeline

Gehel created this task.Jun 27 2017, 3:47 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 27 2017, 3:47 PM
Gehel triaged this task as High priority.Jun 27 2017, 3:47 PM
debt lowered the priority of this task from High to Normal.Oct 17 2017, 5:37 PM
debt added a subscriber: debt.

@mpopov - is this still needed?

mpopov lowered the priority of this task from Normal to Lowest.Oct 23 2017, 8:28 PM

Ish? Until this is done, we're limited to using Ubuntu for the VMs that host our dashboards. Since the WM Cloud team (formerly WM Labs) is deprecating Ubuntu Trusty in favor of only offering Debian for VMs, we'll have to file a Phab ticket requesting a Trusty instance if we have to shut one down and launch a replacement. I don't think this task should be declined, but I am gonna adjust the priority to reflect where we are on this.

I can also try contacting RStudio and asking them if they could also offer a Deb package in addition to the Ubuntu one.

Restricted Application added a project: Product-Analytics. · View Herald TranscriptApr 19 2018, 12:25 AM
mpopov moved this task from Triage to Tracking on the Product-Analytics board.Apr 23 2018, 11:08 PM
Gehel removed Gehel as the assignee of this task.Apr 26 2018, 6:51 AM

Removing discovery / search from this ticket, since it is really not related to search.

mpopov updated the task description. (Show Details)Sep 24 2018, 8:43 PM

It looks like the Debian Science team was working on packaging this for Debian (cf https://salsa.debian.org/science-team/shiny-server) but I don't see an actual ITP.

Who made the package for Ubuntu Trusty / where did it come from?

Who made the package for Ubuntu Trusty / where did it come from?

The Ubunty Trusty package was made by RStudio, Inc. (creators of Shiny Server) team itself (see https://www.rstudio.com/products/shiny/download-server/)

mpopov added a subscriber: Krenair.Oct 22 2018, 1:29 PM

Hi all, just to let you know this now has a deadline of 2018-12-18 per https://wikitech.wikimedia.org/wiki/News/Trusty_deprecation#Cloud_VPS_projects
Please get in contact if you need help.

@Gehel: I guess Discovery Dashboards are going offline 2018-12-18 unless this is resolved?

Are these only used in Cloud VPS, not in production? If the former, we don't need to mirror them to our apt.wikimedia.org repository, they could just as well use the upstream apt repository via puppet. Or is this task about testing (and )potentially fixing) the Ubuntu packages on a jessie host?

Currently Shiny Server is available (via its developer, RStudio) as a package only for Ubunty Trusty. This task is about packaging it up ourselves to make it available on VMs running Debian…I guess Stretch at this point. (I'll update the task title & description.)

This comment was removed by Paladox.
mpopov renamed this task from Upload shiny-server .deb to our Jessie apt repository to Upload shiny-server .deb to our Stretch apt repository.Oct 22 2018, 1:57 PM
mpopov updated the task description. (Show Details)
This comment was removed by mpopov.

Ping, this is blocking the parent task about Trusty deprecation.

debt added a comment.Nov 28 2018, 7:20 PM

@aborrero I believe that @mpopov is still working on this issue, and the Discovery-Search team is well aware of the timeline, we're concerned that if we don't get this migrated, we'll lose all our dashboard data in mid December 2018. :(

So @Gehel told me we might actually be fine. I'm testing out this theory but ran into a problem with the current puppet config for shiny_server module:

bearloga@discovery-testing-01:~$ sudo puppet agent -t
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for discovery-testing-01.shiny-r.eqiad.wmflabs
Notice: /Stage[main]/Base::Environment/Tidy[/var/tmp/core]: Tidying 0 files
Info: Applying configuration version '1544452134'
Error: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install g++-4.8' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Package g++-4.8 is not available, but is referred to by another package.

I've made the patch to address that: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/478252/ but now that Mr. G is on parental leave someone else is gonna need to +2 it.

Change 478252 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/puppet@production] shiny_server: change gfortran/g++ dep

https://gerrit.wikimedia.org/r/478252

mpopov claimed this task.Dec 10 2018, 2:33 PM

Change 478252 merged by Giuseppe Lavagetto:
[operations/puppet@production] shiny_server: change gfortran/g++ dep

https://gerrit.wikimedia.org/r/478252

Alright, dependencies for Shiny Server resolved. Now on to the problem of Shiny Server itself:

Error: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install shiny-server' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Package shiny-server is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'shiny-server' has no installation candidate

I don't know if Guillaume uploaded https://download3.rstudio.org/ubuntu-14.04/x86_64/shiny-server-1.5.9.923-amd64.deb to the internal Debian repository or if it's still only in the Ubuntu one (idk how to check). @aborrero can you or someone else on WMCS team please check? Or is it just a single apt repository? (Again, I'm not ops so I don't know.) From shiny_server::init.pp#L93-L94:

# Assuming shiny-server-1.5.3.838-amd64.deb exists in the WMF apt repo...
require_package('shiny-server')

As soon as discovery-testing-01.eqiad.wmflabs (currently set up as Stretch) is up and working and we can use the dashboards on https://discovery-dev.wmflabs.org, we can give the all-clear from our side on shutting down Trusty instances (at least ours).

I don't know if Guillaume uploaded https://download3.rstudio.org/ubuntu-14.04/x86_64/shiny-server-1.5.9.923-amd64.deb to the internal Debian repository or if it's still only in the Ubuntu one (idk how to check). @aborrero can you or someone else on WMCS team please check? Or is it just a single apt repository? (Again, I'm not ops so I don't know.) From shiny_server::init.pp#L93-L94:

# Assuming shiny-server-1.5.3.838-amd64.deb exists in the WMF apt repo...
require_package('shiny-server')

It doesn't seems to be present in our internal WMF apt repository:

aborrero@install1002:~ $ sudo -i reprepro list stretch-wikimedia | grep shiny
[.. nothing ..]

Right now I'm unsure what the intention with this is, if including it in stretch-wikimedia or what. @MoritzMuehlenhoff any idea?
If the plan is to include this .deb package in stretch-wikimedia I can do that if someone confirms.

As soon as discovery-testing-01.eqiad.wmflabs (currently set up as Stretch) is up and working and we can use the dashboards on https://discovery-dev.wmflabs.org, we can give the all-clear from our side on shutting down Trusty instances (at least ours).

Great!

EBjune added a subscriber: EBjune.Dec 13 2018, 12:24 AM

@mpopov are you waiting on anything from Guillaume here? He's on leave atm, and I want to make sure nothing's blocking from our end in getting the shiny server running.

@EBjune I am waiting for someone with access to the internal WMF apt repository to make shiny-server available.

@aborrero Howdy! I wasn't sure if you needed my confirmation (in which case yes please and thank you!) or an approval from a fellow ops person.

Mentioned in SAL (#wikimedia-operations) [2018-12-13T17:31:20Z] <arturo> T168967 added shiny-server .deb to stretch-wikimedia

Please @mpopov try now.

If later is decided that having this package in the WMF repo is not the way to go, someone ping me and I will drop it from there.

mpopov closed this task as Resolved.Dec 17 2018, 10:06 PM

It works! :D Thanks, @aborrero!