Page MenuHomePhabricator

Toolforge. introduce new domain toolforge.org
Open, MediumPublic

Description

In https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/DNS_domain_usage#Resolution we agreed on introducing the domain toolforge.org as the new general domain for Toolforge.

The path to introducing such domain contains several steps, first of all is probably how and when we will be introducing it. The most sensible option seems to be pairing the introduction of the new domain with the new kubernetes cluster. DNS will be managed via OpenStack Designate.

For the record, this is the current setup at whois level for wmflabs.org vs toolforge.org:

$ whois wmflabs.org | grep "Name Server"
Name Server: CLOUD-NS0.WIKIMEDIA.ORG
Name Server: CLOUD-NS1.WIKIMEDIA.ORG
$ whois toolforge.org | grep "Name Server"
Name Server: NS0.WIKIMEDIA.ORG
Name Server: NS1.WIKIMEDIA.ORG
Name Server: NS2.WIKIMEDIA.ORG

Announced to community: https://wikitech.wikimedia.org/wiki/News/Toolforge.org

Related Objects

StatusSubtypeAssignedTask
Resolvedaborrero
Openaborrero
ResolvedKrenair
ResolvedNone
ResolvedAndrew
Resolvedaborrero
Resolvedaborrero
Resolvedaborrero
DeclinedBstorm
Resolvedaborrero
Resolvedbd808
Resolvedbd808
ResolvedBstorm
ResolvedUrbanecm
Resolvedaborrero
OpenNone
Resolvedbd808
OpenNone
ResolvedBUG REPORTDannyS712
ResolvedBUG REPORTCyberpower678
ResolvedBUG REPORTZache

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-cloud) [2020-03-24T11:33:30Z] <arturo> merging tools-proxy patch https://gerrit.wikimedia.org/r/c/operations/puppet/+/579952/ (T234617) (second try with some additional bits in LUA)

Change 579952 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: support canonical redirects in urlproxy

https://gerrit.wikimedia.org/r/579952

Change 578413 merged by jenkins-bot:
[operations/software/tools-webservice@master] Add support for redirecting to toolforge.org

https://gerrit.wikimedia.org/r/578413

Mentioned in SAL (#wikimedia-cloud) [2020-04-08T23:49:21Z] <bd808> Running rebuild_all for Docker images to pick up toollabs-webservice v0.66 (T154504, T234617)

Mentioned in SAL (#wikimedia-cloud) [2020-04-09T04:29:55Z] <bd808> Running rebuild_all for Docker images to pick up toollabs-webservice v0.66 [try #2] (T154504, T234617)

MBH added a subscriber: MBH.Apr 13 2020, 11:44 AM

There is interwiki prefix [[toollabs:...]] now, that still leads to old url. Will it be updated or will be introduced a new prefix?

Mentioned in SAL (#wikimedia-cloud) [2020-04-13T12:41:18Z] <arturo> adding a new ingress 'default-route-toolforge.org' to redirect http://toolforge.org to the wikitech front page (T234617)

Mentioned in SAL (#wikimedia-cloud) [2020-04-13T12:43:00Z] <arturo> adding a new ingress 'default-route-www.toolforge.org' to redirect http://www.toolforge.org to the wikitech front page (T234617)

There is interwiki prefix [[toollabs:...]] now, that still leads to old url. Will it be updated or will be introduced a new prefix?

I think @bd808 has been working on this already. See https://wikitech.wikimedia.org/wiki/Tool:Iw

bd808 added a comment.Apr 13 2020, 5:29 PM

There is interwiki prefix [[toollabs:...]] now, that still leads to old url. Will it be updated or will be introduced a new prefix?

I think @bd808 has been working on this already. See https://wikitech.wikimedia.org/wiki/Tool:Iw

Some additional documentation at T247432: Preserve the ability to make interwiki links to Toolforge tools under the host based routing scheme. The short answer is yes we will keep interwiki links working. While we are mixing tools.wmflabs.org/$TOOL and $TOOL.toolforge.org IW links will keep working as a side effect of the proxy redirection. Once we stop letting new tools get a tools.wmflabs.org/$TOOL we will either introduce configuration changes in the IW table to point them all to the iw.toolforge.org tool I created, or we will do something closer to home on the MediaWiki side with hooks.

Abbe98 added a subscriber: Abbe98.Apr 13 2020, 7:02 PM
Hasley added a subscriber: Hasley.Apr 27 2020, 12:25 AM

Change 602054 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: legacy_redirector: refresh set of allowed tools

https://gerrit.wikimedia.org/r/602054

TheDJ added a subscriber: TheDJ.Jun 7 2020, 8:37 PM

i haven't been able to find info on tools-static.wmflabs.org.. can someone provide that info and update https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Static_file_server

bd808 added a comment.Jun 8 2020, 9:24 PM

i haven't been able to find info on tools-static.wmflabs.org.. can someone provide that info and update https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Static_file_server

No change is being made tools-static.wmflabs.org at this time.

Change 602054 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: legacy_redirector: refresh set of allowed tools

https://gerrit.wikimedia.org/r/602054

Mentioned in SAL (#wikimedia-cloud) [2020-06-17T10:40:10Z] <arturo> created VM tools-legacy-redirector, with the corresponding puppet prefix (T247236, T234617)

Magnus added a subscriber: Magnus.Mon, Jun 22, 3:07 PM

FWIW, I believe all my (active) tools now use the new canonical URL.

FWIW, I believe all my (active) tools now use the new canonical URL.

Thanks you very much!

Ghuron added a subscriber: Ghuron.Fri, Jun 26, 1:38 PM

One of tools I'm maintaining (https://www.mediawiki.org/wiki/MW2SPARQL) is effectively WDQS federated endpoint. WDQS has white list of supported endpoints and http://tools.wmflabs.org/mw2sparql/sparql is there. Who can replace it to http://mw2sparql.toolforge.org/sparql ?

One of tools I'm maintaining (https://www.mediawiki.org/wiki/MW2SPARQL) is effectively WDQS federated endpoint. WDQS has white list of supported endpoints and http://tools.wmflabs.org/mw2sparql/sparql is there. Who can replace it to http://mw2sparql.toolforge.org/sparql ?

This is currently set in https://github.com/wikimedia/wikidata-query-deploy/blob/master/whitelist.txt#L22

You can update that yourself from gerrit.

Change 608131 had a related patch set uploaded (by Ghuron; owner: Ghuron):
[wikidata/query/deploy@master] Update url to my endpoint because of toolforge.org migration

https://gerrit.wikimedia.org/r/c/wikidata/query/deploy/ /608131

Change 608131 merged by Gehel:
[wikidata/query/deploy@master] Update url to my endpoint because of toolforge.org migration

https://gerrit.wikimedia.org/r/608131

Ghuron removed a subscriber: Ghuron.Fri, Jul 3, 5:57 PM

Change 609855 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/software/tools-webservice@master] Remove --canonical argument to webservice

https://gerrit.wikimedia.org/r/609855

Change 609856 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/software/tools-webservice@master] kubernetes: remove legacy ingress generation

https://gerrit.wikimedia.org/r/609856

Change 610013 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: shinken: refresh URLs with new toolforge.org domain

https://gerrit.wikimedia.org/r/610013

Change 610013 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: shinken: refresh URLs with new toolforge.org domain

https://gerrit.wikimedia.org/r/610013

Change 610024 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: ssh banners: refresh URLs

https://gerrit.wikimedia.org/r/610024

Change 610024 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: ssh banners: refresh URLs

https://gerrit.wikimedia.org/r/610024

Change 610029 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: urlproxy: drop support for the legacy routing scheme

https://gerrit.wikimedia.org/r/610029

Change 609855 merged by jenkins-bot:
[operations/software/tools-webservice@master] Remove --canonical argument to webservice

https://gerrit.wikimedia.org/r/609855

Change 609856 merged by jenkins-bot:
[operations/software/tools-webservice@master] kubernetes: remove legacy ingress generation

https://gerrit.wikimedia.org/r/609856

Mentioned in SAL (#wikimedia-cloud) [2020-07-07T23:16:43Z] <bd808> Building webservice v0.73 (T234617, T257229)

Mentioned in SAL (#wikimedia-cloud) [2020-07-07T23:19:45Z] <bd808> Deploying webservice v0.73 via clush (T234617, T257229)

Mentioned in SAL (#wikimedia-cloud) [2020-07-07T23:22:53Z] <bd808> Rebuilding all Docker images to pick up webservice v0.73 (T234617, T257229)

Change 610029 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: urlproxy: drop support for the legacy routing scheme

https://gerrit.wikimedia.org/r/610029

MBH removed a subscriber: MBH.Wed, Jul 8, 11:25 AM
NicoV added a subscriber: NicoV.EditedWed, Jul 8, 6:09 PM

Problem with 308: Permanent Redirect.

With the migration, I see that my tool, WPCleaner , doesn't manage to update itself on user's installation due to the Permanent Redirect. T257495
I'm using getdown tool to have an automatic update when a user launches the tool: I submitted an issue there.

But I'm surprised to see that the tool replaced the content of some files with HTML code telling there's a Permanent Redirect.

<html>
<head><title>308 Permanent Redirect</title></head>
<body bgcolor="white">
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx/1.14.2</center>
</body>
</html>

Is this normal to have the HTML code, and not be redirected directly?
Is there any way to solve this kind of problem?

Change 610548 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] ncredir: Update mappings for pywikibot redirects

https://gerrit.wikimedia.org/r/610548

bd808 added a comment.Thu, Jul 9, 3:52 AM

Problem with 308: Permanent Redirect.

Let's figure this one out over on T257495: WPCleaner not updating after Toolforge migration.

Change 610548 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] ncredir: Update mappings for pywikibot redirects

https://gerrit.wikimedia.org/r/610548

NicoV removed a subscriber: NicoV.Thu, Jul 9, 2:54 PM