Page MenuHomePhabricator

Toolforge. introduce new domain toolforge.org
Open, MediumPublic

Description

In https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/DNS_domain_usage#Resolution we agreed on introducing the domain toolforge.org as the new general domain for Toolforge.

The path to introducing such domain contains several steps, first of all is probably how and when we will be introducing it. The most sensible option seems to be pairing the introduction of the new domain with the new kubernetes cluster. DNS will be managed via OpenStack Designate.

For the record, this is the current setup at whois level for wmflabs.org vs toolforge.org:

$ whois wmflabs.org | grep "Name Server"
Name Server: CLOUD-NS0.WIKIMEDIA.ORG
Name Server: CLOUD-NS1.WIKIMEDIA.ORG
$ whois toolforge.org | grep "Name Server"
Name Server: NS0.WIKIMEDIA.ORG
Name Server: NS1.WIKIMEDIA.ORG
Name Server: NS2.WIKIMEDIA.ORG

Details

Related Gerrit Patches:

Related Objects

Event Timeline

aborrero triaged this task as Medium priority.Oct 4 2019, 1:06 PM
aborrero created this task.
aborrero moved this task from Inbox to Important on the cloud-services-team (Kanban) board.
bd808 updated the task description. (Show Details)Oct 11 2019, 8:24 PM

Mentioned in SAL (#wikimedia-cloud) [2019-10-11T21:52:32Z] <jeh> create toolforge.org DNS zone in tools project T234617

The toolforge.org domain has been setup in designate

# OS_PROJECT_ID=tools openstack zone create --email 'root@toolforge.org' toolforge.org.
+----------------+--------------------------------------+
| Field          | Value                                |
+----------------+--------------------------------------+
| action         | CREATE                               |
| attributes     | {}                                   |
| created_at     | 2019-10-11T21:51:13.000000           |
| description    | None                                 |
| email          | root@toolforge.org                   |
| id             | 882807c4-f268-4abc-8c13-7826900ed15a |
| masters        |                                      |
| name           | toolforge.org.                       |
| pool_id        | 794ccc2c-d751-44fe-b57f-8894c9f5c842 |
| project_id     | tools                                |
| serial         | 1570830673                           |
| status         | PENDING                              |
| transferred_at | None                                 |
| ttl            | 3600                                 |
| type           | PRIMARY                              |
| updated_at     | None                                 |
| version        | 1                                    |
+----------------+--------------------------------------+
# OS_PROJECT_ID=tools openstack zone list
+--------------------------------------+--------------------+---------+------------+--------+--------+
| id                                   | name               | type    |     serial | status | action |
+--------------------------------------+--------------------+---------+------------+--------+--------+
| eae60a3b-a0df-47b2-9492-5fab480514fe | tools.wmflabs.org. | PRIMARY | 1559574839 | ACTIVE | NONE   |
| 882807c4-f268-4abc-8c13-7826900ed15a | toolforge.org.     | PRIMARY | 1570830673 | ACTIVE | NONE   |
+--------------------------------------+--------------------+---------+------------+--------+--------+
aborrero moved this task from Important to Doing on the cloud-services-team (Kanban) board.

Change 565556 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] dynamicproxy: urlproxy: introduce support for domain-based routing

https://gerrit.wikimedia.org/r/565556

Change 565575 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/software/tools-webservice@master] kubernetes: add support for domain-based routing in the new kubernetes cluster

https://gerrit.wikimedia.org/r/565575

aborrero added a comment.EditedJan 17 2020, 4:38 PM

I have two patches to address this:

My initial approach was to introduce DNS subdomain-based routing in parallel with the 'legacy' routing (URL-based). Tool owners can use both addressing methods during a transition period.
During this transition period, users are encouraged to make sure their webapp code is able to work in a new domain and URL: from tools.wmflabs.org/$toolname to $toolname.toolforge.org.
We can drop the the old routing mechanism when the transition period is over.

Worth noting:

  • this approach supports our 3 backends: the legacy k8s cluster, the new k8s cluster, and gridengine. For the new k8s cluster is as simple as having an additional ingress object.
  • the frontproxy nginx (urlproxy) would no longer be urlproxy. It would be something really similar to domain proxy, but heavily crafted for the Toolforge ingress
  • the frontproxy nginx does several tricks related to maintenance pages, error handling, etc, relying in the admin tool that would need to be reworked.
  • this was tested in toolsbeta with a couple different tool webservices. Apparently everything works fine at first sight.

Will need review by @Bstorm and specially by @bd808 .

Change 566045 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/software/tools-webservice@master] [RFC] kubernetes: add support for multiple objects of any kind

https://gerrit.wikimedia.org/r/566045

Change 566045 merged by jenkins-bot:
[operations/software/tools-webservice@master] kubernetes: add support for multiple objects of any kind

https://gerrit.wikimedia.org/r/566045

Change 565575 merged by Arturo Borrero Gonzalez:
[operations/software/tools-webservice@master] kubernetes: add support for domain-based routing in the new kubernetes cluster

https://gerrit.wikimedia.org/r/565575

Change 570433 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] toolviews: Support host based routing in Toolforge

https://gerrit.wikimedia.org/r/570433

Change 570433 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolviews: Support host based routing in Toolforge

https://gerrit.wikimedia.org/r/570433

Change 565556 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] dynamicproxy: urlproxy: introduce support for domain-based routing

https://gerrit.wikimedia.org/r/565556

Mentioned in SAL (#wikimedia-cloud) [2020-02-06T10:44:33Z] <arturo> merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/565556 which is a behavior change to the Toolforge front proxy (T234617)

TODO: after last changes, requests to https://tools.wmflabs.org/ no longer redirect to the admin tool. The end in fourohfour domains for it to handle. We should consider adding some special handing for this case in fourohfour. CC @bd808

Mentioned in SAL (#wikimedia-cloud) [2020-02-06T10:59:46Z] <arturo> restarted webservice to test how it works with latest front proxy changes (T234617)

Mentioned in SAL (#wikimedia-cloud) [2020-02-06T11:02:44Z] <arturo> requires creating the ingress object by hand. Will leave that to the tool author (bryan) (T234617)

Mentioned in SAL (#wikimedia-cloud) [2020-02-06T11:04:25Z] <arturo> restarted webservice to test how it works with latest front proxy changes (T234617) Then realized we lack newer version of webservice and this restart was for nothing

revi added a subscriber: revi.Thu, Feb 6, 11:29 AM

*snip*
My initial approach was to introduce DNS subdomain-based routing in parallel with the 'legacy' routing (URL-based). Tool owners can use both addressing methods during a transition period.
During this transition period, users are encouraged to make sure their webapp code is able to work in a new domain and URL: from tools.wmflabs.org/$toolname to $toolname.toolforge.org.
We can drop the the old routing mechanism when the transition period is over.
*snip*

Can we not drop old URLs? Even the toolserver.org is still called from various places after the service was shut down in 2014 (for example, T224265 is quite recent) and the principle of Don't delete redirects.

bd808 added a comment.Thu, Feb 6, 5:27 PM

Can we not drop old URLs? Even the toolserver.org is still called from various places after the service was shut down in 2014 (for example, T224265 is quite recent) and the principle of Don't delete redirects.

We will maintain redirects from tools.wmflabs.org/$tool to $tool.wmflabs.org indefinitely. That functionality will be moved into a separate service however and not maintained in the Toolforge front proxy at some point. As noted we are still maintaining a similar system for toolserver.org URLs.

Webservice changes will be deployed with version 0.61

toolsbeta.test@toolsbeta-sgebastion-04:~$ webservice migrate
/usr/lib/python2.7/dist-packages/urllib3/connection.py:337: SubjectAltNameWarning: Certificate for toolsbeta-k8s-master-01.toolsbeta.eqiad.wmflabs has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
Stopping webservice on legacy Kubernetes cluster
Switched to context "toolforge".
Starting webservice on 2020 Kubernetes cluster...............
toolsbeta.test@toolsbeta-sgebastion-04:~$ curl -L test.toolsbeta.wmflabs.org -H "Host:test.toolforge.org"
Hello World, from Toolsbeta!
toolsbeta.test@toolsbeta-sgebastion-04:~$ curl http://toolsbeta.wmflabs.org/test/
Hello World, from Toolsbeta!
toolsbeta.test@toolsbeta-sgebastion-04:~$ /usr/bin/kubectl get ingress
NAME             HOSTS                   ADDRESS   PORTS   AGE
test-legacy      toolsbeta.wmflabs.org             80      114s
test-subdomain   test.toolforge.org                80      114s
toolsbeta.test@toolsbeta-sgebastion-04:~$

Will deploy package to the rest today.

Bstorm added a comment.Sat, Feb 8, 1:15 AM

Spoke too soon. A regression from another patch requires a change first. This will have to be 0.62 on Monday.

Mentioned in SAL (#wikimedia-cloud) [2020-02-10T21:18:57Z] <bstorm_> upgraded toollabs-webservice package for stretch toolsbeta to 0.62 T244293 T244289 T234617 T156626

Mentioned in SAL (#wikimedia-cloud) [2020-02-10T21:25:10Z] <bstorm_> upgraded toollabs-webservice package for tools to 0.62 T244293 T244289 T234617 T156626