Page MenuHomePhabricator

Update OpusMT cloud instance
Closed, ResolvedPublic

Description

Update language team's OpusMT instance (https://opusmt.wmflabs.org/) for:

  • Use systemd service instead of user session+screen.
  • Automate download of new language pair models:
    • Script is now available at: /usr/local/bin/opusmt-model-download.sh and accessible by all users.
  • Remove unused language pairs.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 10 2020, 6:36 AM
KartikMistry updated the task description. (Show Details)Sep 14 2020, 3:38 PM
KartikMistry triaged this task as Medium priority.Sep 17 2020, 12:43 PM
KartikMistry updated the task description. (Show Details)Sep 28 2020, 5:11 AM

systemd service is available for review at: kartik@opusmt.eqiad.wmflabs:/home/kartik/opusmt.service

The file should be in some stable location in case the cloud machine goes poof.

What's ENV_DIR? Do not use /bin/sh. You can use EnvironmentFile= I think.

The file should be in some stable location in case the cloud machine goes poof.

Agree. I'll submit pull request to upstream after we get it working :)

What's ENV_DIR? Do not use /bin/sh. You can use EnvironmentFile= I think.

ENV_DIR is used for Python virtual environment. Used EnvironmentFile= now to tell service to use it from there.

KartikMistry updated the task description. (Show Details)Oct 6 2020, 11:11 AM
KartikMistry updated the task description. (Show Details)Oct 7 2020, 3:22 AM

Unused language models are removed from configuration file, so instance will be faster now.

@Nikerabbit I've moved script as systemd service now, but not yet enabled by default. opusmt still running as script in my user screen :) I'll ping you again once it is ready.

Still breaks service startup, so moved back to 'In progress'.

OpusMT now runs with basic (bit hacky, but better than screen session!) service on cloud instance:

kartik@opusmt:~$ sudo service opusmt status
● opusmt.service - OpusMT service
   Loaded: loaded (/lib/systemd/system/opusmt.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2020-11-03 06:46:46 UTC; 3s ago
     Docs: https://github.com/Helsinki-NLP/Opus-MT
 Main PID: 29828 (opusmt-server)
    Tasks: 14 (limit: 4915)
   Memory: 793.6M
Rileych closed this task as Resolved.Nov 19 2020, 1:55 PM