Page MenuHomePhabricator

adding new languages to DNS langs.tmpl doesn't work until zone template is edited as well
Closed, ResolvedPublic

Description

Every once in a while we want to add new projects in new languages (like for example T96468). This requires adding them to DNS where we use a templating system to generate the entries for all languages.

There is the template for the wikipedia.org zone file and it contains these lines:

67 ; All languages will automatically be included here.
68 {{ geolanglist('text-addrs') }}

This is generated by ./helpers/langlist.tmpl and the languages themselves are in ./helpers/langs.tmpl.

So a typical change to add a new language is only an edit to ./helpers.langs.tmpl. (like https://gerrit.wikimedia.org/r/#/c/205009/)

If this gets merged and deployed nothing will happen though.

It will only actually get generated if/when the zone template ./templates/wikipedia.org is touched in any way. It can be any kind of "null edit", just a dot in a comment for example but only then the new language will be added.

Can we find a way to make that either happen automatically or let jenkins warn about it or add some docs to always make a tiny edit so it's more obvious why the new language doesn't show up even after merge.

Event Timeline

Dzahn raised the priority of this task from to Needs Triage.
Dzahn updated the task description. (Show Details)
Dzahn added projects: DNS, acl*sre-team.
Dzahn subscribed.
Dzahn set Security to None.

Change 206146 had a related patch set uploaded (by Dzahn):
touch wikipedia.org template to generate 'gom'

https://gerrit.wikimedia.org/r/206146

Change 206146 merged by Dzahn:
touch wikipedia.org template to generate 'gom'

https://gerrit.wikimedia.org/r/206146

example: adding this single dot here: https://gerrit.wikimedia.org/r/#/c/206146/1/templates/wikipedia.org

effectively added gom.wikipedia.org from an earlier merge now.

perhaps just running $ touch /path/file works? If it does, we could add that to authdns, right?

Dzahn triaged this task as Low priority.Apr 29 2015, 1:43 AM

Change 226079 had a related patch set uploaded (by Mjbmr):
Touch project domain templates to enable azb sumdomain

https://gerrit.wikimedia.org/r/226079

Change 226079 merged by Filippo Giunchedi:
Touch project domain templates to enable azb subdomain

https://gerrit.wikimedia.org/r/226079

I took a quick look at this, it should really happening automatically, for that to happen authdns-gen-zones.py I think should be aware of included templates and check the timestamp of those too. One way for that would get to load the template in question and check with jinja2.meta.find_referenced_templates all the dependent files too.

For now, one way to workaround this is:

Run authdns-update as normal, which takes care of some important stuff still on all 3x nameservers (like updating the git clone):

ns0# authdns-update

Manually run a forced authdns-gen-zones with appopriate parameters and then "gdnsd checkconfig && gdnsd reload-zones" on each of the nameservers separately:

ns0# authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones
ns1# authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones
ns2# authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones

Mentioned in SAL [2016-04-29T22:57:45Z] <mutante> DNS - forced authdns-gen-zones etc from https://phabricator.wikimedia.org/T97051#1994679 on ns0/ns1/ns2 to get new language added

Mentioned in SAL [2016-07-20T21:49:46Z] <mutante> DNS authdns-gen-zones on all servers to add new language tcy (bug T97051)

Mentioned in SAL (#wikimedia-operations) [2016-10-04T16:30:00Z] <mutante> authdns commands from T97051#1994679 to add olo.wp for T146612

BBlack claimed this task.

This is fixed now, no workarounds should be needed.