Page MenuHomePhabricator

"wikistream" Cloud VPS project jessie deprecation
Closed, ResolvedPublic

Description

The end of life of Debian Jessie is approaching in 2020 and we need to move to Debian Buster (or Stretch) before that date.

All instances in the wikistream project need to upgrade as soon as possible. Instances not upgraded by 2019-12-31 may be subject to deletion unless prior arrangements for an extended deadline has been approved by the Cloud VPS administration team.

Remaining Debian Jessie instances (live report):

Listed administrators are:

See also:

More info on current project instances is available via openstack browser

Details

Due Date
Dec 31 2019, 11:59 PM

Event Timeline

StrikerBot created this task.
bd808 added a subscriber: bd808.

Ping @edsu

The Cloud Services team would like to have Debian Jessie systems replaced with Debian Buster (or Stretch if necessary) before 2019-12-31. Please do respond of this task with comments if you know you will not be able to meet that target date or have additional questions about what to do or how to do it. Ideally you will create new instances in your Cloud VPS project, test them, and then migrate any final state data to the new instances before deleting the old Jessie instances. If you need more quota space in your project to create new instances in parallel with your existing instances please create a quota request task describing the increase you need and referencing this deprecation task.

If there are instructions on how to create the new instance and update DNS appropriately so that wikistream.wmflabs.org points to it that would be appreciated. I won't be able to determine whether I can get this done in the next few weeks without more information.

Creating a new instance can be done through horizon and should be fairly
intuitive, you'll want to pick either Debian stretch (older, process is
likely to need repeating sooner) or Debian buster (newer, process shouldn't
need repeating for longer), and an instance size (can use the same as the
existing one shown in openstack-browser/horizon, though if it's currently
larger than required this is a good opportunity to downsize).
Once you've submitted the form in horizon it'll go through the setup
process and after a few minutes you should find yourself able to log into
it. You then configure it probably roughly the same way you did the
existing instance.
In your case it looks like instead of floating IPs and your own DNS you use
the provided HTTP(S) proxy. You'll want to delete the existing entry and
make a new one (if it needs to be done in place leave a note and maybe
someone can have a look to see if that's doable).
If you encounter issues you can request help on the task or maybe on IRC
(#wikimedia-cloud).

@edsu You marked this project as "in use" in the 2019 project purge. Can you provide an estimate of when you or others may be able to address this issue?

I cannot honestly. If you think it is worth preserving please feel free to migrate the service. It really is a minimal node process that requires no database access at all. Otherwise I'll run it myself somewhere else.

As an aside, it is somewhat annoying that employees of WMF can't step up to provision the new instance, vm, container or whatever the latest awesome is. Instead you are asking a volunteer to do this?

Perhaps WMF's idea of "active" and mine are different. I am not actively supporting the service other than responding to people's queries (on twitter, github) when the service is not available because of the occasional hiccup. But as you can see from the access logs it is actively used by other people. If you decide that doesn't meet your definition of active please consider it inactive and turn it off.

As an aside, it is somewhat annoying that employees of WMF can't step up to provision the new instance, vm, container or whatever the latest awesome is. Instead you are asking a volunteer to do this?

Managed instances are not generally a service offered by the Cloud VPS environment or the Wikimedia Cloud Services team, but we can obviously try to provide help when folks need it.

Can you point me to the documentation of how this system is configured and how to test it? I do not see any maintenance instructions at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikistream.

Perhaps WMF's idea of "active" and mine are different. I am not actively supporting the service other than responding to people's queries (on twitter, github) when the service is not available because of the occasional hiccup. But as you can see from the access logs it is actively used by other people. If you decide that doesn't meet your definition of active please consider it inactive and turn it off.

Our definition of "active" is maintained by one or more Wikimedia community members. No one is trying to force you to continue to take care of this tool, just as no one is trying to force you to abandon this tool. I have been pinging you because you are the only person with administrative access to the OpenStack project other than current and former Cloud Services staff.

Perhaps what is really needed here is some call for new maintainers for the tool? It is not clear to me why this is a Cloud VPS project and not a tool on Toolforge. Maybe one "fix" would be to find someone interested in actively maintaining the tool, have them fork the upstream GitHub repo, setup the software as a Toolforge tool, and then once this is known to be working redirect the current wikistream.wmflabs.org HTTP proxy to the new tool.

I seem to remember moving this to run at WMF as Toolforge was transitioning its architecture so it wasn't available at the time. I maintain another service in Toolforge and have found it to be a supportive environment. So I will go ahead and request the new tool and udpate this ticket when it's available there.

@yuvipanda would you be open to me using your wikistream Tools project as a place to move wikistream into? It doesn't look like the space is currently being used? If you add me as a member of the project I think I can take it from there.

@yuvipanda would you be open to me using your wikistream Tools project as a place to move wikistream into? It doesn't look like the space is currently being used? If you add me as a member of the project I think I can take it from there.

It looks like Yuvi was actually trying to run your code on Toolforge there anyway. The $HOME for that tool is a git clone of https://github.com/edsu/wikistream.git from around 2014-10-02. Let's wait a day or two to see if Yuvi responds, but if he does not I think we can add you as a maintainer under the rules of https://wikitech.wikimedia.org/wiki/Help:Toolforge/Abandoned_tool_policy.

@yuvipanda would you be open to me using your wikistream Tools project as a place to move wikistream into? It doesn't look like the space is currently being used? If you add me as a member of the project I think I can take it from there.

https://twitter.com/yuvipanda/status/1213215336444518401

I will do the needful to add @edsu as a co-maintainer of the wikistream tool.

I will do the needful to add @edsu as a co-maintainer of the wikistream tool.

{{Done}}

@edsu Because you are starting fresh, I would like to invite you to be an early adopter of the updated Kubernetes cluster that we are very close to rolling to for everyone. The new cluster is a work-alike for the legacy cluster and there are just a few simple steps needed for you to start using it:

$ ssh login.tools.wmflabs.org
$ become wikistream
$ kubectl config use-context toolforge
$ alias kubectl=/usr/bin/kubectl
$ echo "alias kubectl=/usr/bin/kubectl" >> $HOME/.profile

The kubectl config use-context toolforge command is the real "magic" that will make your interactions with webservice and kubectl talk to the new cluster. The kubectl aliases are a temporary step needed until we remove the older version of kubbectl that is found first in the default search path.

Use webservice --backend=kubernetes node10 start|stop|restart|shell to get a Node v10 runtime environment. See https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#node.js_web_services for more details.

The 2020-04-25 EOL date for Debian Jessie is rapidly approaching. Projects where the status of migration is unknown maybe subject to forced instance shutdown on or before that date. Please reply with a rough timeline/plan for the resolution of this task.

@edsu Any update on this project? Did you manage to get things working in Toolforge? The EOL date for Jessie is 3 weeks out now.

@edsu Any update on this project? Did you manage to get things working in Toolforge? The EOL date for Jessie is 3 weeks out now.

2020-04-25 is now a short 2 weeks away.

I allowed myself to get sucked into keeping this tool alive in the near term. In service of that I have now setup:

The fork contains code changes that were needed to make it easier to run the app using the Toolforge conventions for running Node applications. I also upgraded most (all?) of the npm managed libraries to their most modern versions which required a few code changes to deal with api deprecations.

Next steps are:

Mentioned in SAL (#wikimedia-cloud) [2020-04-30T16:50:31Z] <bd808> Added BryanDavis (self) as project admin (T236551)

Mentioned in SAL (#wikimedia-cloud) [2020-04-30T17:26:09Z] <bd808> Added proxy for wikistream.wmflabs.org (T236551)

Mentioned in SAL (#wikimedia-cloud) [2020-04-30T17:31:05Z] <bd808> Shutdown ws-web.wikistream.eqiad.wmflabs (T236551)

  • Advertise for additional maintainers for the newly updated wikistream tool so that it is more likely to be kept running in the future.

T251555: wikistream.toolforge.org needs new maintainers

Mentioned in SAL (#wikimedia-cloud) [2020-04-30T20:11:59Z] <bd808> Deleted ws-web.wikistream.eqiad.wmflabs (T236551)

Mentioned in SAL (#wikimedia-cloud) [2020-04-30T20:31:09Z] <bd808> Deleting project; service migrated to Toolforge (T236551)

bd808 updated the task description. (Show Details)