Page MenuHomePhabricator

WCQS needs to be exposed through a wikimedia.org domain
Closed, ResolvedPublic2 Estimated Story Points

Description

As a WCQS user I want to be able to have WCQS accessible via commons-query.wikimedia.org so that I can that proper bot authentication is enabled.

We need to have both more representative address for WCQS and a way to set up oauth authentication for bots similarly to other projects. Easiest way to achieve both is to set up WCQS as a top level wikimedia project.

AC:

  • commons-query.wikimedia.org is configured and resolves to the appropriate IP

Event Timeline

MPhamWMF moved this task from Incoming to SDAW on the Wikidata-Query-Service board.
Zbyszko renamed this task from Set up WCQS top level wikimedia project to Set up WCQS top level wikimedia domain.Aug 23 2021, 3:38 PM
Gehel renamed this task from Set up WCQS top level wikimedia domain to WCQS needs to be exposed through a wikimedia.org domain.Aug 23 2021, 3:38 PM
MPhamWMF set the point value for this task to 2.Aug 23 2021, 3:39 PM
Gehel removed the point value for this task.
Gehel set the point value for this task to 2.Aug 23 2021, 3:43 PM

Change 717606 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/dns@master] Add cname for commons-query.wikimedia.org

https://gerrit.wikimedia.org/r/717606

Change 724520 had a related patch set uploaded (by Ryan Kemper; author: Ebernhardson):

[operations/dns@master] wcqs: add discovery record

https://gerrit.wikimedia.org/r/724520

Change 724520 merged by Ryan Kemper:

[operations/dns@master] wcqs: add discovery record

https://gerrit.wikimedia.org/r/724520

Mentioned in SAL (#wikimedia-operations) [2021-09-28T23:14:13Z] <ryankemper> T282117 ryankemper@authdns1001:~$ sudo -i authdns-update following merge of https://gerrit.wikimedia.org/r/724520

Mentioned in SAL (#wikimedia-operations) [2021-09-28T23:14:56Z] <ryankemper> !log T282117 error: plugin_geoip: Invalid resource name 'disc-wcqs' detected from zonefile lookup We must be missing a line, reverting change to fix

[Log of failed authdns-update following https://gerrit.wikimedia.org/r/724520]
ryankemper@authdns1001:~$ sudo -i authdns-update
Updating authdns1001.wikimedia.org (self)...
Pulling the current revision from https://gerrit.wikimedia.org/r/operations/dns.git
Reviewing c81f47118e6e6fea77cc8970885dd61b61c92bd0...

 templates/wmnet                        | 1 +
 utils/mock_etc/discovery-geo-resources | 1 +
 2 files changed, 2 insertions(+)

diff --git templates/wmnet templates/wmnet
index ea3d7279..9d554061 100644
--- templates/wmnet
+++ templates/wmnet
@@ -601,6 +601,7 @@ push-notifications 300/10 IN DYNA geoip!disc-push-notifications
 recommendation-api 300/10 IN DYNA geoip!disc-recommendation-api
 wdqs           300/10 IN DYNA geoip!disc-wdqs
 wdqs-internal  300/10 IN DYNA geoip!disc-wdqs-internal
+wcqs           300/10 IN DYNA geoip!disc-wcqs
 zotero         300/10 IN DYNA geoip!disc-zotero
 blubberoid     300/10 IN DYNA geoip!disc-blubberoid
 eventgate-analytics 300/10 IN DYNA geoip!disc-eventgate-analytics
diff --git utils/mock_etc/discovery-geo-resources utils/mock_etc/discovery-geo-resources
index d5f686e1..0d51b0bc 100644
--- utils/mock_etc/discovery-geo-resources
+++ utils/mock_etc/discovery-geo-resources
@@ -65,3 +65,4 @@ disc-shellbox-media      => { map => mock, dcmap => { mock => 192.0.2.1 } }
 disc-shellbox-syntaxhighlight => { map => mock, dcmap => { mock => 192.0.2.1 } }
 disc-shellbox-timeline   => { map => mock, dcmap => { mock => 192.0.2.1 } }
 disc-tegola-vector-tiles => { map => mock, dcmap => { mock => 192.0.2.1 } }
+disc-wcqs                => { map => mock, dcmap => { mock => 192.0.2.1 } }

Merge these changes? (yes/no)? yes
Updating e1e708d9..c81f4711
Fast-forward
 templates/wmnet                        | 1 +
 utils/mock_etc/discovery-geo-resources | 1 +
 2 files changed, 2 insertions(+)
Deploying via utils/deploy-check.py...
Assembling and testing data in /tmp/dns-check.uhizg7cf
 -- Generating zonefiles from zone templates
 -- Processed 211 zones into directory /tmp/dns-check.uhizg7cf/zones
OK: No tabs
Summary of violations:
    W001|MISSING_IP_FOR_NAME_AND_PTR: 37
    W002|MISSING_PTR_FOR_NAME_AND_IP: 26
    W103|MISSING_MGMT_FOR_NAME: 64
    W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 130 Warnings, 0 Ignored violations, 0 Ignored lines
 -- Copying automatically generated zone files under target tree
 -- Copying repo-driven real config files and admin_state
 -- Copying puppetized config and GeoIP from /etc/gdnsd
 -- Checking for illegal tabs in zonefiles
 -- Running zone_validator to check WMF rules
 -- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.uhizg7cf
Traceback (most recent call last):
  File "utils/deploy-check.py", line 276, in <module>
    main()
  File "utils/deploy-check.py", line 268, in main
    deploy_check(args.deploy, args.skip_reload, args.no_gdnsd, Path(tdir), gdir)
  File "utils/deploy-check.py", line 214, in deploy_check
    safe_cmd([GDNSD_BIN, '-c', str(tdir), 'checkconf'])
  File "utils/deploy-check.py", line 88, in safe_cmd
    p_err.decode('utf-8')))
Exception: Command /usr/sbin/gdnsd -c /tmp/dns-check.uhizg7cf checkconf failed with exit code 42, stderr:
info: gdnsd version 3.8.0 @ pid 22926
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.154.238:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 208.80.153.231:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 91.198.174.239:53
info: DNS listener threads (8 UDP + 8 TCP) configured for 198.35.27.27:53
info: DNS listener threads (8 TCP PROXY) configured for 127.0.0.1:535
info: DNS listener threads (1 UDP + 1 TCP) configured for 0.0.0.0:5353
info: DNS listener threads (1 UDP + 1 TCP) configured for [::]:5353
info: plugin_geoip: map 'generic-map': Loading GeoIP2 database '/tmp/dns-check.uhizg7cf/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-27 17:10:01 UTC
info: plugin_geoip: map 'generic-map' runtime db updated. nets: 1122136 dclists: 8
info: plugin_geoip: map 'checkdoh-map' runtime db updated. nets: 410 dclists: 2
info: plugin_geoip: map 'discovery-map': Loading GeoIP2 database '/tmp/dns-check.uhizg7cf/geoip/GeoIP2-City.mmdb': Version: 2.0, Type: GeoIP2-City, IPVersion: 6, Timestamp: 2021-09-27 17:10:01 UTC
info: plugin_geoip: map 'discovery-map' runtime db updated. nets: 464 dclists: 2
info: admin_state: checking state file '/tmp/dns-check.uhizg7cf/state/admin_state'...
error: plugin_geoip: Invalid resource name 'disc-wcqs' detected from zonefile lookup
error: Name 'wcqs.discovery.wmnet.': resolver plugin 'geoip' rejected resource name 'disc-wcqs'
fatal: Initial load of zone data failed

Change 724538 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[operations/dns@master] wcqs: add discovery record

https://gerrit.wikimedia.org/r/724538

Change 724538 merged by Ryan Kemper:

[operations/dns@master] wcqs: add discovery record

https://gerrit.wikimedia.org/r/724538

While this is ready from a code perspective, we are waiting to deploy it until everything it exposes (updater, oauth, etc) is deployed.

Change 755806 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[operations/dns@master] wcqs: add discovery record

https://gerrit.wikimedia.org/r/755806

Change 755806 merged by Bking:

[operations/dns@master] wcqs: add discovery record

https://gerrit.wikimedia.org/r/755806

Mentioned in SAL (#wikimedia-operations) [2022-01-24T22:32:24Z] <inflatador> T280001 T282117 Merged https://gerrit.wikimedia.org/r/c/operations/dns/+/755806 and ran sudo -i authdns update on authdns1001.wikimedia.org

Change 717606 merged by Ryan Kemper:

[operations/dns@master] Add cname for commons-query.wikimedia.org

https://gerrit.wikimedia.org/r/717606

Mentioned in SAL (#wikimedia-operations) [2022-02-01T01:48:17Z] <ryankemper> T282117 Merged https://gerrit.wikimedia.org/r/c/operations/dns/+/717606 and successfully ran sudo -i authdns-update on authdns1001. commons-query.wikimedia.org is online now. (sidenote: go-live date of service is 2022-02-01)