Page MenuHomePhabricator
Create Task
Maniphest T184296

Convert CloudVPS instances to stretch.
Closed, ResolvedPublic
Actions

Description

ores.wmflabs.org
ores-staging.wmflabs.org

Related Objects
Search...

Event Timeline

Halfak created this task.Jan 5 2018, 8:46 PM

ores-staging-01 is back online as a stretch instance.

Halfak claimed this task.Jan 5 2018, 8:55 PM
Halfak edited projects, added: Machine-Learning-Team (Active Tasks); removed: Machine-Learning-Team.
Halfak added a comment.Jan 5 2018, 10:45 PM

Working on this now :|

halfak@ores-staging-01:~$ virtualenv
-bash: virtualenv: command not found
Halfak added a comment.EditedJan 5 2018, 10:52 PM

So in staging, we import base. And in base we require 'virtualenv'. Hmm... I'm checking on puppet now.

Refs:

Running:

halfak@ores-staging-01:~$ sudo puppet agent -tv
Warning: Setting configtimeout is deprecated. 
   (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1146:in `issue_deprecation_warning')
Notice: Starting Puppet client version 4.8.2
...
Halfak added a comment.Jan 5 2018, 10:57 PM

Looks like I forgot to enable role::labs::ores::staging. Thought I'd clicked that :/

Halfak added a comment.Jan 5 2018, 11:08 PM

Now I'm looking at:

[ores-staging-01.eqiad.wmflabs] sudo: /srv/ores/venv/bin/pip install --use-wheel --no-deps /srv/ores/config/submodules/wheels/pip-*.whl
[ores-staging-01.eqiad.wmflabs] out: Requirement '/srv/ores/config/submodules/wheels/pip-*.whl' looks like a filename, but the file does not exist
[ores-staging-01.eqiad.wmflabs] out: pip-*.whl is not a valid wheel filename.
[ores-staging-01.eqiad.wmflabs] out:

So... what? Looks like we're supposed to include pip in our wheels.

Halfak added a comment.Jan 5 2018, 11:25 PM

And we're online! See https://ores-staging.wmflabs.org/v3/scores/fiwiki/1241

But it's returning an error. It seems like we have the wrong version of celery running.

{
  "fiwiki": {
    "models": {
      "damaging": {
        "version": "0.4.1"
      },
      "goodfaith": {
        "version": "0.4.1"
      }
    },
    "scores": {
      "1241": {
        "damaging": {
          "error": {
            "message": "Never call result.get() within a task!\nSee http://docs.celeryq.org/en/latest/userguide/tasks.html#task-synchronous-subtasks\n",
            "type": "RuntimeError"
          }
        },
        "goodfaith": {
          "error": {
            "message": "Never call result.get() within a task!\nSee http://docs.celeryq.org/en/latest/userguide/tasks.html#task-synchronous-subtasks\n",
            "type": "RuntimeError"
          }
        }
      }
    }
  }
}
Halfak added a comment.Jan 5 2018, 11:56 PM

IT'S ALIVE https://ores-staging.wmflabs.org/v3/scores/fiwiki/1241241

Halfak closed this task as Resolved.Jan 5 2018, 11:56 PM
Halfak reopened this task as Open.
Halfak moved this task from Parked to Completed on the Machine-Learning-Team (Active Tasks) board.
Halfak moved this task from Completed to Parked on the Machine-Learning-Team (Active Tasks) board.

Woops. I didn't do the main cluster. Leaving that for later.

Halfak added a comment.Jan 6 2018, 4:36 PM

OK so I'm thinking of just taking wmflabs cluster down half-way, installing, then doing the other half. Doesn't seem too crazy.

Halfak added a subtask: T184766: Convert ores-misc-01 to stretch.Jan 11 2018, 7:38 PM
Halfak added a comment.Jan 11 2018, 9:16 PM

ores-web-01 is created and configured, but not pooled. It seems to work fine to request scores from this node and it is able to talk to celery as planned.

Halfak added a comment.Jan 17 2018, 9:39 PM

Done! https://ores.wmflabs.org/v3/scores/enwiki/32456784/wp10?features

Halfak moved this task from Parked to Completed on the Machine-Learning-Team (Active Tasks) board.Jan 17 2018, 9:39 PM
awight mentioned this in Blog Post: Status Update (January 30, 2018).Jan 30 2018, 7:03 PM
Halfak closed this task as Resolved.Jan 30 2018, 8:30 PM
Halfak closed subtask T184766: Convert ores-misc-01 to stretch as Resolved.
• 238482n375 removed Halfak as the assignee of this task.Jun 15 2018, 8:01 AM
• 238482n375 triaged this task as Lowest priority.
• 238482n375 moved this task from Next Up to In Code Review on the Analytics-Kanban board.
• 238482n375 added projects: Analytics-Kanban, acl*security, Wikimedia-VE-Campaigns (S2-2018), Scap (Scap3-Adoption-Phase2), AbuseFilter, Data-release, Hashtags, LabsDB-Auditor, Ladies-That-FOSS-MediaWiki, Language-2018-Apr-June, Language-2018-Jan-Mar, HHVM, HAWelcome.
• 238482n375 edited subscribers, added: • 238482n375; removed: Aklapper.

SG9tZVBoYWJyaWNhdG9yCk5vIG1lc3NhZ2VzLiBObyBub3RpZmljYXRpb25zLgoKICAgIFNlYXJjaAoKQ3JlYXRlIFRhc2sKTWFuaXBoZXN0ClQxOTcyODEKRml4IGZhaWxpbmcgd2VicmVxdWVzdCBob3VycyAodXBsb2FkIGFuZCB0ZXh0IDIwMTgtMDYtMTQtMTEpCk9wZW4sIE5lZWRzIFRyaWFnZVB1YmxpYwoKICAgIEVkaXQgVGFzawogICAgRWRpdCBSZWxhdGVkIFRhc2tzLi4uCiAgICBFZGl0IFJlbGF0ZWQgT2JqZWN0cy4uLgogICAgUHJvdGVjdCBhcyBzZWN1cml0eSBpc3N1ZQoKICAgIE11dGUgTm90aWZpY2F0aW9ucwogICAgQXdhcmQgVG9rZW4KICAgIEZsYWcgRm9yIExhdGVyCgpUYWdzCgogICAgQW5hbHl0aWNzLUthbmJhbiAoSW4gUHJvZ3Jlc3MpCgpTdWJzY3JpYmVycwpBa2xhcHBlciwgSkFsbGVtYW5kb3UKQXNzaWduZWQgVG8KSkFsbGVtYW5kb3UKQXV0aG9yZWQgQnkKSkFsbGVtYW5kb3UsIEZyaSwgSnVuIDE1CkRlc2NyaXB0aW9uCgpPb3ppZSBqb2JzIGhhdmUgYmVlbiBmYWlsaW5nIGF0IGxlYXN0IGEgZmV3IHRpbWVzIGVhY2guIE1vcmUgaW52ZXN0aWdhdGlvbiBuZWVkZWQuCkpBbGxlbWFuZG91IGNyZWF0ZWQgdGhpcyB0YXNrLkZyaSwgSnVuIDE1LCA3OjIxIEFNCkhlcmFsZCBhZGRlZCBhIHN1YnNjcmliZXI6IEFrbGFwcGVyLiC3IFZpZXcgSGVyYWxkIFRyYW5zY3JpcHRGcmksIEp1biAxNSwgNzoyMSBBTQpKQWxsZW1hbmRvdSBjbGFpbWVkIHRoaXMgdGFzay5GcmksIEp1biAxNSwgNzoyMiBBTQpKQWxsZW1hbmRvdSB1cGRhdGVkIHRoZSB0YXNrIGRlc2NyaXB0aW9uLiAoU2hvdyBEZXRhaWxzKQpKQWxsZW1hbmRvdSBhZGRlZCBhIHByb2plY3Q6IEFuYWx5dGljcy1LYW5iYW4uCkpBbGxlbWFuZG91IG1vdmVkIHRoaXMgdGFzayBmcm9tIE5leHQgVXAgdG8gSW4gUHJvZ3Jlc3Mgb24gdGhlIEFuYWx5dGljcy1LYW5iYW4gYm9hcmQuCkNoYW5nZSBTdWJzY3JpYmVycwpDaGFuZ2UgUHJpb3JpdHkKQXNzaWduIC8gQ2xhaW0KTW92ZSBvbiBXb3JrYm9hcmQKQ2hhbmdlIFByb2plY3QgVGFncwpBbmFseXRpY3MtS2FuYmFuCtcKU2VjdXJpdHkK1wpXaWtpbWVkaWEtVkUtQ2FtcGFpZ25zIChTMi0yMDE4KQrXClNjYXAK1wpTY2FwIChTY2FwMy1BZG9wdGlvbi1QaGFzZTIpCtcKQWJ1c2VGaWx0ZXIK1wpEYXRhLXJlbGVhc2UK1wpIYXNodGFncwrXCkxhYnNEQi1BdWRpdG9yCtcKTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2kK1wpMYW5ndWFnZS0yMDE4LUFwci1KdW5lCtcKTGFuZ3VhZ2UtMjAxOC1KYW4tTWFyCtcKSEhWTQrXCkhBV2VsY29tZQrXCkJvbGQKSXRhbGljcwpNb25vc3BhY2VkCkxpbmsKQnVsbGV0ZWQgTGlzdApOdW1iZXJlZCBMaXN0CkNvZGUgQmxvY2sKUXVvdGUKVGFibGUKVXBsb2FkIEZpbGUKTWVtZQpQcmV2aWV3CkhlbHAKRnVsbHNjcmVlbiBNb2RlClBpbiBGb3JtIE9uIFNjcmVlbgoyMzg0ODJuMzc1IGFkZGVkIHByb2plY3RzOiBTZWN1cml0eSwgV2lraW1lZGlhLVZFLUNhbXBhaWducyAoUzItMjAxOCksIFNjYXAgKFNjYXAzLUFkb3B0aW9uLVBoYXNlMiksIEFidXNlRmlsdGVyLCBEYXRhLXJlbGVhc2UsIEhhc2h0YWdzLCBMYWJzREItQXVkaXRvciwgTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2ksIExhbmd1YWdlLTIwMTgtQXByLUp1bmUsIExhbmd1YWdlLTIwMTgtSmFuLU1hciwgSEhWTSwgSEFXZWxjb21lLlBSRVZJRVcKMjM4NDgybjM3NSBtb3ZlZCB0aGlzIHRhc2sgZnJvbSBJbiBQcm9ncmVzcyB0byBJbiBDb2RlIFJldmlldyBvbiB0aGUgQW5hbHl0aWNzLUthbmJhbiBib2FyZC4KMjM4NDgybjM3NSByZW1vdmVkIEpBbGxlbWFuZG91IGFzIHRoZSBhc3NpZ25lZSBvZiB0aGlzIHRhc2suCjIzODQ4Mm4zNzUgdHJpYWdlZCB0aGlzIHRhc2sgYXMgTG93ZXN0IHByaW9yaXR5LgoyMzg0ODJuMzc1IHJlbW92ZWQgc3Vic2NyaWJlcnM6IEFrbGFwcGVyLCBKQWxsZW1hbmRvdS4KQ29udGVudCBsaWNlbnNlZCB1bmRlciBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLVNoYXJlQWxpa2UgMy4wIChDQy1CWS1TQSkgdW5sZXNzIG90aGVyd2lzZSBub3RlZDsgY29kZSBsaWNlbnNlZCB1bmRlciBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSAoR1BMKSBvciBvdGhlciBvcGVuIHNvdXJjZSBsaWNlbnNlcy4gQnkgdXNpbmcgdGhpcyBzaXRlLCB5b3UgYWdyZWUgdG8gdGhlIFRlcm1zIG9mIFVzZSwgUHJpdmFjeSBQb2xpY3ksIGFuZCBDb2RlIG9mIENvbmR1Y3QuILcgV2lraW1lZGlhIEZvdW5kYXRpb24gtyBQcml2YWN5IFBvbGljeSC3IENvZGUgb2YgQ29uZHVjdCC3IFRlcm1zIG9mIFVzZSC3IERpc2NsYWltZXIgtyBDQy1CWS1TQSC3IEdQTApZb3VyIGJyb3dzZXIgdGltZXpvbmUgc2V0dGluZyBkaWZmZXJzIGZyb20gdGhlIHRpbWV6b25lIHNldHRpbmcgaW4geW91ciBwcm9maWxlLCBjbGljayB0byByZWNvbmNpbGUu

• 238482n375 set Security to Software security bug.Jun 15 2018, 8:04 AM
• 238482n375 changed the visibility from "Public (No Login Required)" to "Custom Policy".

SG9tZVBoYWJyaWNhdG9yCk5vIG1lc3NhZ2VzLiBObyBub3RpZmljYXRpb25zLgoKICAgIFNlYXJjaAoKQ3JlYXRlIFRhc2sKTWFuaXBoZXN0ClQxOTcyODEKRml4IGZhaWxpbmcgd2VicmVxdWVzdCBob3VycyAodXBsb2FkIGFuZCB0ZXh0IDIwMTgtMDYtMTQtMTEpCk9wZW4sIE5lZWRzIFRyaWFnZVB1YmxpYwoKICAgIEVkaXQgVGFzawogICAgRWRpdCBSZWxhdGVkIFRhc2tzLi4uCiAgICBFZGl0IFJlbGF0ZWQgT2JqZWN0cy4uLgogICAgUHJvdGVjdCBhcyBzZWN1cml0eSBpc3N1ZQoKICAgIE11dGUgTm90aWZpY2F0aW9ucwogICAgQXdhcmQgVG9rZW4KICAgIEZsYWcgRm9yIExhdGVyCgpFVzZSC3IERpc2NsYWltZXIgtyBDQy1CWS1TQSC3IEdQTApZb3VyIGJyb3dzZXIgdGltZXpvbmUgc2V0dGluZyBkaWZmZXJzIGZyb20gdGhlIHRpbWV6b25lIHNldHRpbmcgaW4geW91ciBwcm9maWxlLCBjbGljayB0byByZWNvbmNpbGUu

Halfak claimed this task.Jun 15 2018, 1:39 PM
Halfak removed projects: HAWelcome, HHVM, Language-2018-Jan-Mar, Language-2018-Apr-June, Ladies-That-FOSS-MediaWiki, LabsDB-Auditor, Hashtags, Data-release, AbuseFilter, Scap (Scap3-Adoption-Phase2), Wikimedia-VE-Campaigns (S2-2018), acl*security.
Halfak changed the visibility from "Custom Policy" to "Public (No Login Required)".
Halfak removed a subscriber: • 238482n375.
Restricted Application added a project: acl*security. · View Herald TranscriptJun 15 2018, 1:39 PM
Aklapper raised the priority of this task from Lowest to Needs Triage.Jun 15 2018, 2:25 PM
Aklapper removed projects: acl*security, Analytics-Kanban.
Aklapper subscribed.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits