Page MenuHomePhabricator

adding new languages to DNS langs.tmpl doesn't work until zone template is edited as well
Closed, ResolvedPublic

Description

Every once in a while we want to add new projects in new languages (like for example T96468). This requires adding them to DNS where we use a templating system to generate the entries for all languages.

There is the template for the wikipedia.org zone file and it contains these lines:

67 ; All languages will automatically be included here.
68 {{ geolanglist('text-addrs') }}

This is generated by ./helpers/langlist.tmpl and the languages themselves are in ./helpers/langs.tmpl.

So a typical change to add a new language is only an edit to ./helpers.langs.tmpl. (like https://gerrit.wikimedia.org/r/#/c/205009/)

If this gets merged and deployed nothing will happen though.

It will only actually get generated if/when the zone template ./templates/wikipedia.org is touched in any way. It can be any kind of "null edit", just a dot in a comment for example but only then the new language will be added.

Can we find a way to make that either happen automatically or let jenkins warn about it or add some docs to always make a tiny edit so it's more obvious why the new language doesn't show up even after merge.

Event Timeline

Dzahn created this task.Apr 23 2015, 5:13 PM
Dzahn raised the priority of this task from to Needs Triage.
Dzahn updated the task description. (Show Details)
Dzahn added projects: DNS, acl*sre-team.
Dzahn added a subscriber: Dzahn.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 23 2015, 5:13 PM
Dzahn updated the task description. (Show Details)Apr 23 2015, 5:14 PM
Dzahn set Security to None.

Change 206146 had a related patch set uploaded (by Dzahn):
touch wikipedia.org template to generate 'gom'

https://gerrit.wikimedia.org/r/206146

Dzahn added a subscriber: BBlack.Apr 23 2015, 5:17 PM

Change 206146 merged by Dzahn:
touch wikipedia.org template to generate 'gom'

https://gerrit.wikimedia.org/r/206146

Dzahn added a comment.Apr 23 2015, 5:22 PM

example: adding this single dot here: https://gerrit.wikimedia.org/r/#/c/206146/1/templates/wikipedia.org

effectively added gom.wikipedia.org from an earlier merge now.

perhaps just running $ touch /path/file works? If it does, we could add that to authdns, right?

Dzahn triaged this task as Low priority.Apr 29 2015, 1:43 AM

Change 226079 had a related patch set uploaded (by Mjbmr):
Touch project domain templates to enable azb sumdomain

https://gerrit.wikimedia.org/r/226079

Restricted Application added a subscriber: Matanya. · View Herald TranscriptJul 21 2015, 2:27 PM

Change 226079 merged by Filippo Giunchedi:
Touch project domain templates to enable azb subdomain

https://gerrit.wikimedia.org/r/226079

I took a quick look at this, it should really happening automatically, for that to happen authdns-gen-zones.py I think should be aware of included templates and check the timestamp of those too. One way for that would get to load the template in question and check with jinja2.meta.find_referenced_templates all the dependent files too.

BBlack added a comment.Feb 3 2016, 5:59 PM

For now, one way to workaround this is:

Run authdns-update as normal, which takes care of some important stuff still on all 3x nameservers (like updating the git clone):

ns0# authdns-update

Manually run a forced authdns-gen-zones with appopriate parameters and then "gdnsd checkconfig && gdnsd reload-zones" on each of the nameservers separately:

ns0# authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones
ns1# authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones
ns2# authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones

Mentioned in SAL [2016-04-29T22:57:45Z] <mutante> DNS - forced authdns-gen-zones etc from https://phabricator.wikimedia.org/T97051#1994679 on ns0/ns1/ns2 to get new language added

Mentioned in SAL [2016-05-03T16:22:40Z] <bblack> authdns-update for https://gerrit.wikimedia.org/r/#/c/285085/ (yue lang) + workarounds from https://phabricator.wikimedia.org/T97051#1994679

Mentioned in SAL [2016-07-20T21:49:46Z] <mutante> DNS authdns-gen-zones on all servers to add new language tcy (bug T97051)

BBlack moved this task from Triage to DNS Infra on the Traffic board.Sep 30 2016, 2:10 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-04T16:30:00Z] <mutante> authdns commands from T97051#1994679 to add olo.wp for T146612

BBlack closed this task as Resolved.Dec 20 2018, 8:53 PM
BBlack claimed this task.

This is fixed now, no workarounds should be needed.