Page MenuHomePhabricator

Provide and set up server space for new Monumental
Open, Needs TriagePublic

Description

Work with @Yarl and @Lokal_Profil to find a home for monumental-glam (which needs Node).


Set-up

  • T233656 created the Cloud VPS project finding-glams
  • Create a single instance finding-glams-monumental with source=Debian-10.0-buster, flavour=m1.large, security_groups=Default
  • Create a web proxy mapping port 3000 of finding-glams-monumental to finding-glams.wmflabs.org
  • Create a new security group (web-server) and add the rule:
Rule: Custom TCP Rule
Direction: Ingress
Open Port: Port
Port: 3000
Remote: CIDR
CIDR: 172.16.0.0/21
  • Add the web-server rule to the finding-glams-monumental instance.
  • Install nodjs12
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs

node --version
> v12.10.0
npm --version
> 6.10.3
  • Install yarn
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt-get update && sudo apt-get install yarn

yarn --version
> 1.19.0
  • Create user monumental-agent to run the service: sudo useradd -m monumental-agent -p <password>
  • Change to the user: sudo su -s /bin/bash - monumental-agent
  • As monumental-agent:
    • clone the repo: git clone https://github.com/yarl/finding-glams.git
    • Change to the cloned directory: cd finding-glams
    • Install dependencies: yarn install
    • Build the project: yarn build
    • Start the webservice as a background process: nohup yarn start &
    • Store the PID of the process to be able to kill it (and it's subprocess) later: echo $! > nohup.pid
  • Check that it is up at https://finding-glams.wmflabs.org

Event Timeline

Currently blocked by a clarification from @Yarl about which component is incompatible with ToolForge (needed to determine if Cloud VPS is an alternative or if we need to run with hosting on Bahnhof)

Lokal_Profil moved this task from Backlog to Watching on the User-LokalProfil board.

Discussed it today, if we don't have more specific information by Thursday, we'll initiate work with Cloud VPS.

The finding-glams project has now been set up on Cloud VPS. I gave admin access to @Sebastian_Berlin-WMSE to coordinate around initial setup (drawing on his memory of what we did for Wikispeech).

@Yarl is a single Debian machine enough?

Set up a single instance (finding-glams-monumental with source=Debian-10.0-buster, flavour=m1.large, security_groups=Default).

Added a proxy mapping port 80 of finding-glams-monumental to finding-glams.wmflabs.org

Installed nodjs 12 (per these instructions) - note that apt.wikimedia.org carries only nodjs10

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs

node --version
> v12.10.0
npm --version
> 6.10.3

Installed yarn (per these instructions):

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt-get update && sudo apt-get install yarn

yarn --version
> 1.19.0

updated the proxy to point to port 3000

Lokal_Profil renamed this task from Provide server space for new Monumental to Provide and set up server space for new Monumental.Oct 4 2019, 8:06 AM

Created a separate user to be responsible for running the service and cloned the repo

sudo useradd -m monumental-agent -p <password>
sudo su -s /bin/bash - monumental-agent
git clone https://github.com/yarl/monumental-vue.git
cd monumental-vue

When running yarn build (as monumental-agent) I get

yarn run v1.19.0
$ nuxt build
/bin/sh: 1: nuxt: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Figured I first needed to install the dependencies so I ran yarn install which gave me

yarn install v1.19.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@2.0.7: The platform "linux" is incompatible with this module.
info "fsevents@2.0.7" is an optional dependency and failed compatibility check. Excluding it from installation.
info fsevents@1.2.9: The platform "linux" is incompatible with this module.
info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > bootstrap@4.3.1" has unmet peer dependency "jquery@1.9.1 - 3".
warning " > bootstrap@4.3.1" has unmet peer dependency "popper.js@^1.14.7".
warning "bootstrap-vue > portal-vue@2.1.5" has unmet peer dependency "vue@^2.5.18".
warning "babel-jest > babel-preset-jest > @babel/plugin-syntax-object-rest-spread@7.2.0" has unmet peer dependency "@babel/core@^7.0.0-0".
warning "nuxt-leaflet > vue2-leaflet@2.1.1" has unmet peer dependency "@types/leaflet@^1.2.11".
warning "nuxt-leaflet > vue2-leaflet@2.1.1" has unmet peer dependency "vue@^2.5.17".
warning " > @vue/test-utils@1.0.0-beta.29" has unmet peer dependency "vue@2.x".
warning " > @vue/test-utils@1.0.0-beta.29" has unmet peer dependency "vue-template-compiler@^2.x".
warning " > babel-core@7.0.0-bridge.0" has unmet peer dependency "@babel/core@^7.0.0-0".
warning " > babel-jest@24.8.0" has unmet peer dependency "@babel/core@^7.0.0".
warning "babel-jest > babel-preset-jest@24.6.0" has unmet peer dependency "@babel/core@^7.0.0".
warning " > eslint-config-prettier@4.3.0" has unmet peer dependency "eslint@>=3.14.1".
warning " > eslint-plugin-prettier@3.1.0" has unmet peer dependency "eslint@>= 5.0.0".
warning " > vue-jest@3.0.4" has unmet peer dependency "vue@^2.x".
warning " > vue-jest@3.0.4" has unmet peer dependency "vue-template-compiler@^2.x".
[4/4] Building fresh packages...
warning Your current version of Yarn is out of date. The latest version is "1.19.1", while you're on "1.19.0".
info To upgrade, run the following command:
$ sudo apt-get update && sudo apt-get install yarn
Done in 27.78s.

The unmet peer dependencies should probably get fixed in the repo itself (per this SO post) and it seems as though the fsevents warning can normally be ignored.

Despite the above yarn build works and yarn start starts the webservice succesfully.

Lokal_Profil added a comment.EditedOct 11 2019, 1:43 PM

To make the webservice accessible from the internet.

Create a new security group ("web-server":"Firewall rules for web access") then add the rule:

Rule: Custom TCP Rule
Direction: Ingress
Open Port: Port
Port: 3000
Remote: CIDR
CIDR: 172.16.0.0/21

Then add the "web-server" security group to the finding-glams-monumental instance

Inserted the following into nuxt.config.js to make it listen at the right ip (rather than local host) per FAQ at nuxtjs.org

server: {
    port: 3000,
    host: '0.0.0.0',
  }

And of course (as monumental-agent) start the service with nohup yarn start & to not make it die when you log out.

Lokal_Profil added a comment.EditedOct 11 2019, 7:52 PM

Checked out the right repo instead (github.com/yarl/finding-glams.git) and started that instead.

Also followed the nohup yarn start &command with echo $! > nohup.pid to store the process id to make it easier to kill it later. The various hoops needed to be jumped through also made the need for T235301: Create a update-and-redeploy script clear.

Inserted the following into nuxt.config.js to make it listen at the right ip (rather than local host) per FAQ at nuxtjs.org

server: {
    port: 3000,
    host: '0.0.0.0',
  }

https://github.com/yarl/finding-glams/pull/4

Lokal_Profil updated the task description. (Show Details)Oct 14 2019, 8:51 AM
Lokal_Profil moved this task from This week to Done on the User-LokalProfil board.

Summarised all of the (corrected) steps to make it easier to re-use and understand in the future.

Lokal_Profil moved this task from Backlog to Done on the WMSE-FindingGLAMs-2018 (Campaign) board.

Mentioned in SAL (#wikimedia-cloud) [2019-10-14T09:21:13Z] <wm-bot> <lokal-profil> Deploy latest from Git master: 945ce11 (T232074)

Lokal_Profil updated the task description. (Show Details)Oct 14 2019, 9:47 AM

Needs an additional bit about Plinth

To get Plinth running I did

sudo apt install python-pip

# as monumental-agent
cd plinth
pip install --user virtualenv
python -m virtualenv .venv
pip install -r requirements.txt

python plinth/app.py
> Traceback (most recent call last):
>  File "plinth/app.py", line 240, in <module>
>    app = create_app()
>  File "plinth/app.py", line 185, in create_app
>    routes = [fe_app_route('/', ignore_trailing=False),  # TODO: necessary?
>  File "plinth/app.py", line 183, in fe_app_route
>    return StaticFileRoute(path, target)
>  File "/home/monumental-agent/plinth/.venv/local/lib/python2.7/site-packages/clastic/static.py", line 121, in __init__
>    open(file_path).close()
> IOError: [Errno 2] No such file or directory: '/home/monumental-agent/plinth/static/index.html'

This was after figuring out that I had to run plinth/app.py instead of monumental/server.py as described in the README.

Note that I haven't even touched the config which need updating.