Maniphest T232074

Provide and set up server space for new Monumental
Closed, DeclinedPublic
Actions

Assigned To

Authored By

	Alicia_Fagerving_WMSE
	Sep 5 2019, 7:06 AM

Description

Work with @Yarl and @Lokal_Profil to find a home for monumental-glam (which needs Node).

Set-up

T233656 created the Cloud VPS project finding-glams
Create a single instance finding-glams-monumental with source=Debian-10.0-buster, flavour=m1.large, security_groups=Default
Create a web proxy mapping port 3000 of finding-glams-monumental to finding-glams.wmflabs.org
Create a new security group (web-server) and add the rule:

Rule: Custom TCP Rule
Direction: Ingress
Open Port: Port
Port: 3000
Remote: CIDR
CIDR: 172.16.0.0/21

Add the web-server rule to the finding-glams-monumental instance.
Install nodjs12

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs

node --version
> v12.10.0
npm --version
> 6.10.3

Install yarn

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt-get update && sudo apt-get install yarn

yarn --version
> 1.19.0

Create user monumental-agent to run the service: sudo useradd -m monumental-agent -p <password>
Change to the user: sudo su -s /bin/bash - monumental-agent
As monumental-agent:
- clone the repo: git clone https://github.com/yarl/finding-glams.git
- Change to the cloned directory: cd finding-glams
- Install dependencies: yarn install
- Build the project: yarn build
- Start the webservice as a background process: nohup yarn start &
- Store the PID of the process to be able to kill it (and it's subprocess) later: echo $! > nohup.pid
Check that it is up at https://finding-glams.wmflabs.org

Related Objects
Search...

Status	Assigned	Task
Resolved	Jopparn	T215263 Develop the FindingGLAMs campaign for 2019
Resolved	Jopparn	T214909 Development of Monumental for the FindingGLAMs campaign
Declined	Lokal_Profil	T232074 Provide and set up server space for new Monumental
Resolved	Jopparn	T232326 Launch new Monumental
Declined	None	T232458 Update statistics queries with new change tag
Open	None	T234664 Set up redirect from old address

Event Timeline

Alicia_Fagerving_WMSE created this task.Sep 5 2019, 7:06 AM

Alicia_Fagerving_WMSE added a parent task: T214909: Development of Monumental for the FindingGLAMs campaign.

Alicia_Fagerving_WMSE updated the task description. (Show Details)Sep 5 2019, 7:41 AM

Alicia_Fagerving_WMSE added a subscriber: Sebastian_Berlin-WMSE.Sep 6 2019, 9:14 AM

Alicia_Fagerving_WMSE added a subtask: T232326: Launch new Monumental.Sep 9 2019, 12:00 PM

Currently blocked by a clarification from @Yarl about which component is incompatible with ToolForge (needed to determine if Cloud VPS is an alternative or if we need to run with hosting on Bahnhof)

Lokal_Profil added a project: User-Lokal_Profil.Sep 9 2019, 1:06 PM

Lokal_Profil moved this task from 📥 Backlog to 👁 Watching on the User-Lokal_Profil board.

Discussed it today, if we don't have more specific information by Thursday, we'll initiate work with Cloud VPS.

Alicia_Fagerving_WMSE moved this task from 🗃️ Inbox to ♾️ Watching on the User-Alicia_Fagerving_WMSE board.Sep 20 2019, 9:04 AM

Alicia_Fagerving_WMSE mentioned this in T233607: Decide on schedule for campaign (Oct 2).Sep 23 2019, 2:33 PM

T233656

Lokal_Profil moved this task from 👁 Watching to 📆 This week on the User-Lokal_Profil board.Oct 3 2019, 3:16 PM

The finding-glams project has now been set up on Cloud VPS. I gave admin access to @Sebastian_Berlin-WMSE to coordinate around initial setup (drawing on his memory of what we did for Wikispeech).

@Yarl is a single Debian machine enough?

Sure.

Set up a single instance (finding-glams-monumental with source=Debian-10.0-buster, flavour=m1.large, security_groups=Default).

Added a proxy mapping port 80 of finding-glams-monumental to finding-glams.wmflabs.org

Installed nodjs 12 (per these instructions) - note that apt.wikimedia.org carries only nodjs10

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs

node --version
> v12.10.0
npm --version
> 6.10.3

Installed yarn (per these instructions):

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt-get update && sudo apt-get install yarn

yarn --version
> 1.19.0

updated the proxy to point to port 3000

Lokal_Profil renamed this task from Provide server space for new Monumental to Provide and set up server space for new Monumental.Oct 4 2019, 8:06 AM

Created a separate user to be responsible for running the service and cloned the repo

sudo useradd -m monumental-agent -p <password>
sudo su -s /bin/bash - monumental-agent
git clone https://github.com/yarl/monumental-vue.git
cd monumental-vue

When running yarn build (as monumental-agent) I get

yarn run v1.19.0
$ nuxt build
/bin/sh: 1: nuxt: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Figured I first needed to install the dependencies so I ran yarn install which gave me

yarn install v1.19.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@2.0.7: The platform "linux" is incompatible with this module.
info "fsevents@2.0.7" is an optional dependency and failed compatibility check. Excluding it from installation.
info fsevents@1.2.9: The platform "linux" is incompatible with this module.
info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > bootstrap@4.3.1" has unmet peer dependency "jquery@1.9.1 - 3".
warning " > bootstrap@4.3.1" has unmet peer dependency "popper.js@^1.14.7".
warning "bootstrap-vue > portal-vue@2.1.5" has unmet peer dependency "vue@^2.5.18".
warning "babel-jest > babel-preset-jest > @babel/plugin-syntax-object-rest-spread@7.2.0" has unmet peer dependency "@babel/core@^7.0.0-0".
warning "nuxt-leaflet > vue2-leaflet@2.1.1" has unmet peer dependency "@types/leaflet@^1.2.11".
warning "nuxt-leaflet > vue2-leaflet@2.1.1" has unmet peer dependency "vue@^2.5.17".
warning " > @vue/test-utils@1.0.0-beta.29" has unmet peer dependency "vue@2.x".
warning " > @vue/test-utils@1.0.0-beta.29" has unmet peer dependency "vue-template-compiler@^2.x".
warning " > babel-core@7.0.0-bridge.0" has unmet peer dependency "@babel/core@^7.0.0-0".
warning " > babel-jest@24.8.0" has unmet peer dependency "@babel/core@^7.0.0".
warning "babel-jest > babel-preset-jest@24.6.0" has unmet peer dependency "@babel/core@^7.0.0".
warning " > eslint-config-prettier@4.3.0" has unmet peer dependency "eslint@>=3.14.1".
warning " > eslint-plugin-prettier@3.1.0" has unmet peer dependency "eslint@>= 5.0.0".
warning " > vue-jest@3.0.4" has unmet peer dependency "vue@^2.x".
warning " > vue-jest@3.0.4" has unmet peer dependency "vue-template-compiler@^2.x".
[4/4] Building fresh packages...
warning Your current version of Yarn is out of date. The latest version is "1.19.1", while you're on "1.19.0".
info To upgrade, run the following command:
$ sudo apt-get update && sudo apt-get install yarn
Done in 27.78s.

The unmet peer dependencies should probably get fixed in the repo itself (per this SO post) and it seems as though the fsevents warning can normally be ignored.

Despite the above yarn build works and yarn start starts the webservice succesfully.

To make the webservice accessible from the internet.

Create a new security group ("web-server":"Firewall rules for web access") then add the rule:

Rule: Custom TCP Rule
Direction: Ingress
Open Port: Port
Port: 3000
Remote: CIDR
CIDR: 172.16.0.0/21

Then add the "web-server" security group to the finding-glams-monumental instance

Inserted the following into nuxt.config.js to make it listen at the right ip (rather than local host) per FAQ at nuxtjs.org

server: {
    port: 3000,
    host: '0.0.0.0',
  }

And it's up!

And of course (as monumental-agent) start the service with nohup yarn start & to not make it die when you log out.

Checked out the right repo instead (github.com/yarl/finding-glams.git) and started that instead.

Also followed the nohup yarn start &command with echo $! > nohup.pid to store the process id to make it easier to kill it later. The various hoops needed to be jumped through also made the need for T235301: Create a update-and-redeploy script clear.

In T232074#5566439, @Lokal_Profil wrote:
Inserted the following into nuxt.config.js to make it listen at the right ip (rather than local host) per FAQ at nuxtjs.org
server: {
    port: 3000,
    host: '0.0.0.0',
  }

https://github.com/yarl/finding-glams/pull/4

Summarised all of the (corrected) steps to make it easier to re-use and understand in the future.

Lokal_Profil claimed this task.Oct 14 2019, 8:52 AM

Lokal_Profil moved this task from Backlog to Done on the WMSE-FindingGLAMs-2018 (Campaign) board.

Alicia_Fagerving_WMSE moved this task from ♾️ Watching to ☑️ Done on the User-Alicia_Fagerving_WMSE board.Oct 14 2019, 9:09 AM

Mentioned in SAL (#wikimedia-cloud) [2019-10-14T09:21:13Z] <wm-bot> <lokal-profil> Deploy latest from Git master: 945ce11 (T232074)

Lokal_Profil updated the task description. (Show Details)Oct 14 2019, 9:47 AM

Needs an additional bit about Plinth

To get Plinth running I did

sudo apt install python-pip

# as monumental-agent
cd plinth
pip install --user virtualenv
python -m virtualenv .venv
pip install -r requirements.txt

python plinth/app.py
> Traceback (most recent call last):
>  File "plinth/app.py", line 240, in <module>
>    app = create_app()
>  File "plinth/app.py", line 185, in create_app
>    routes = [fe_app_route('/', ignore_trailing=False),  # TODO: necessary?
>  File "plinth/app.py", line 183, in fe_app_route
>    return StaticFileRoute(path, target)
>  File "/home/monumental-agent/plinth/.venv/local/lib/python2.7/site-packages/clastic/static.py", line 121, in __init__
>    open(file_path).close()
> IOError: [Errno 2] No such file or directory: '/home/monumental-agent/plinth/static/index.html'