Page MenuHomePhabricator

Decom legacy ex-parsoidcache cxserver, citoid, and restbase service hostnames
Open, MediumPublic0 Estimated Story Points

Description

Related: T110472

This is about eventually decommissioning the alternate hostname entrypoints for these services. Going forward, these should all be using /api/rest_v1/ on the wiki domains themselves.

Current Status

Original Explanation
Quoting from the wikimedia.org DNS zonefile, the current list of these entrypoint hostnames is:

;;;; cxserver/citoid/rest(base) below are LEGACY, to be removed eventually
cxserver.eqiad      600 IN DYNA geoip!text-addrs
cxserver            600 IN DYNA geoip!text-addrs
citoid.eqiad        600 IN DYNA geoip!text-addrs
citoid              600 IN DYNA geoip!text-addrs
restbase.eqiad      600 IN DYNA geoip!text-addrs
restbase            600 IN DYNA geoip!text-addrs
rest                600 IN DYNA geoip!text-addrs

There's also some varnish code in support of these on the text cluster.

I'd like to start with pulling the .eqiad. subdomain entries - these should never have been used publicly and probably aren't now, and we can try to confirm that with traffic logging and code searches (aside: perhaps we should do better messaging within the org that any hostname in a datacenter-specific public subdomain is purely for infrastructure tracking/debugging, not for use as an official service hostname).

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
BBlack triaged this task as Medium priority.Apr 19 2016, 1:47 AM
BBlack updated the task description. (Show Details)

citoid.wm.org and cxserver.wm.org are still being actively and officially used, as we haven't put them behind RESTBase yet. We are officially dropping support for rest*.wm.org next Monday.

I'd like to start with pulling the .eqiad. subdomain entries

+1

I saw the deprecation email about RESTBase itself. Are we clear to pull the DNS for rest(base)?.wm.o? I'm going to do some Host: and/or DNS logging about the other .eqiad names to make sure we don't have significant usage volume before pulling them.

@BBlack, if there is no specific reason to pull the DNS soon, I would propose to keep the current RB error / redirect notice up for a little longer. We definitely don't serve any content from rest.wikimedia.org any more, but there are still links from old announcements out there that some people might still follow.

And actually, on the issue of the .eqiad. hostnames: they've been broken for a while due to out TLS redirects and our certs work, so no point leaving them around.

@BBlack, if there is no specific reason to pull the DNS soon, I would propose to keep the current RB error / redirect notice up for a little longer. We definitely don't serve any content from rest.wikimedia.org any more, but there are still links from old announcements out there that some people might still follow.

Ok maybe I'm confused on this point. Testing requests like https://restbase.wikimedia.org/en.wikipedia.org/v1/page/mobile-text/Occitan_language, they seem to still work fine.

@BBlack, restbase.wikimedia.org was never published or in any way supposed to work. All public references were to rest.wikimedia.org, which now returns errors with a hint on the move: https://rest.wikimedia.org/en.wikipedia.org/v1/page/mobile-text/Occitan_language

So yes, it would be great to remove *restbase*.wikimedia.org from DNS, but leave the old official entry point *rest*.wikimedia.org around for a little longer.

Change 287682 had a related patch set uploaded (by BBlack):
Remove (citoid|cxserver|restbase).eqiad.wm.o hostnames

https://gerrit.wikimedia.org/r/287682

Change 287682 merged by BBlack:
Remove deprecated/dysfunctional wm.o service hostnames

https://gerrit.wikimedia.org/r/287682

restbase 600 IN DYNA geoip!text-addrs

Despite it being undocumented, beware that it was commonly used (albeit by accident) by several developers, including staff:

If something important breaks, we can put that name back in. But in that case, we should also update RB to deny service there with update info, like it does for rest.wikimedia.org requests.

Re the rest.wikimedia.org: we effectively shut off service back in April ( https://lists.wikimedia.org/pipermail/wikitech-l/2016-April/085309.html ) but the hostname is still live and connected to the RB backends through the caches, just to serve the notice that it moved. Has it been long enough now that we can go ahead and remove it? If not, do we have some kind of plan or timeline for when?

Hm, I can see 40k log entries for this domain in the logs for requests coming from clients from all over the world. Note that the are logged with a 10% probability, so the actual number of requests is much larger. However, since 2016-04-14 we've been returning the 410 Gone status, so I'd be +1 on sending yet another announcement and turning it off in a month or so.

Sounds good to me, you want to do it, or me, or @GWicke since he sent the first one?

Sounds good to me, you want to do it, or me, or @GWicke since he sent the first one?

I'd let @GWicke do it for consistency :P (but also because for some reason I can't post to api-announce).

Okay, I'll send out an announcement today. Lets say we'll switch it off by September 1st?

Okay, I'll send out an announcement today.

Awesome, thnx!

Lets say we'll switch it off by September 1st?

Yup, sounds good.

Change 308723 had a related patch set uploaded (by BBlack):
remove rest.wikimedia.org from DNS

https://gerrit.wikimedia.org/r/308723

Change 308724 had a related patch set uploaded (by BBlack):
text VCL: remove support for rest.wm.o

https://gerrit.wikimedia.org/r/308724

Change 308723 merged by BBlack:
remove rest.wikimedia.org from DNS

https://gerrit.wikimedia.org/r/308723

Change 308724 merged by BBlack:
text VCL: remove support for rest.wm.o

https://gerrit.wikimedia.org/r/308724

rest.wikimedia.org is gone. We've still got citoid.wikimedia.org and cxserver.wikimedia.org for eventual removal when they eventually move to a Restbased API.

tom29739 removed a subscriber: tom29739.
tom29739 added a subscriber: tom29739.

Change 350505 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[operations/puppet@production] Decom legacy citoid service hostname

https://gerrit.wikimedia.org/r/350505

Change 350748 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/dns@master] Remove the citoid.wm.org DNS record

https://gerrit.wikimedia.org/r/350748

Change 350748 merged by BBlack:
[operations/dns@master] Remove the citoid.wm.org DNS record

https://gerrit.wikimedia.org/r/350748

Change 350505 merged by BBlack:
[operations/puppet@production] Decom legacy citoid service hostname

https://gerrit.wikimedia.org/r/350505

So at this point basically only cxserver is remaining. Work on that is ongoing in T107914.

GWicke raised the priority of this task from Medium to Needs Triage.Aug 8 2017, 5:56 PM
GWicke triaged this task as Medium priority.
GWicke moved this task from doing to blocked on the Services board.
GWicke edited projects, added Services (blocked); removed Services (doing).

Change 471246 had a related patch set uploaded (by Mvolz; owner: Mvolz):
[operations/mediawiki-config@master] Removed decomissioned citoid url

https://gerrit.wikimedia.org/r/471246

Change 471246 merged by jenkins-bot:
[operations/mediawiki-config@master] Removed decomissioned citoid url

https://gerrit.wikimedia.org/r/471246

Mentioned in SAL (#wikimedia-operations) [2018-11-05T21:38:38Z] <thcipriani@deploy1001> Synchronized wmf-config/CommonSettings.php: [[gerrit:471246|Removed decomissioned citoid url]] T133001 (duration: 00m 53s)

@Pchelolo what about https://cxserver.wikimedia.org/ - Can it be removed? Or is it better to just ignore it until it all demises later, or?

@Pchelolo what about https://cxserver.wikimedia.org/ - Can it be removed? Or is it better to just ignore it until it all demises later, or?

https://cxserver.wikimedia.org/ is still in use (Production and elsewhere probably). Let me check relevant tasks and update.

The swap of Traffic for Traffic-Icebox in this ticket's set of tags was based on a bulk action for all such tickets that haven't been updated in 6 months or more. This does not imply any human judgement about the validity or importance of the task, and is simply the first step in a larger task cleanup effort. Further manual triage and/or requests for updates will happen this month for all such tickets. For more detail, have a look at the extended explanation on the main page of Traffic-Icebox . Thank you!