Page MenuHomePhabricator

WikimediaEventUtilities and produce_canary_events job should use api-ro.discovery.wmnet instead of meta.wikimedia.,org to get stream config
Open, Needs TriagePublic

Description

produce_canary_events currently looks up stream config from https://meta.wikimedia.org/w/api.php?format=json&action=streamconfigs&all_settings=true. To contact this URL, it must use webproxy (I think due to Analytics VLAN firewall).

We could open up a hole in the Analytics VLAN to this endpoint, but the more correct thing to do would be to use api-ro.discovery.wmnet with a Host header set to meta.wikimedia.org. We still might need a VLAN hole to api-ro.discovery.wmnet.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptTue, Feb 16, 10:06 PM

Oh, this is a bit more of a problem than just canary events. Camus is using webproxy to get to meta.wm.org API to discover topics to import.

We should open this hole in the vlan asap.

Ok, @akosiaris has webproxy turned back on for now. We need to do 2 things:

  • Make WikimediaEventUtilities take a headers parameter for its stream_config_uri, so we can properly use api-ro.discovery.wmnet with Host: meta.wikimedida.org
  • Open a hole in the Analytics VLAN firewall api-ro.discovery.wmnet

Change 665415 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[wikimedia-event-utilities@master] Use api-ro.discovery.wmnet as default Wikimedia EventStreamConfig URI

https://gerrit.wikimedia.org/r/665415

Ottomata added a comment.EditedFri, Feb 19, 10:25 PM

@razzi, @elukey, Analytics VLAN should be able to access api-ro.discovery.wmnet port 80, which resolves to 10.2.2.22 for me in both eqiad and codfw. I'm not 100% sure that is the right address, but it seems to be?

It might be handy if we could also access meta.wikimedia.org too, which resolves to 208.80.154.224 for me (dyna.wikimedia.org?) but I'm not sure if that is possible or correct.

Could one of you open do this? Thank you!

elukey added a comment.EditedMon, Feb 22, 7:19 AM
elukey@puppetmaster1001:~$ sudo -i confctl --quiet --object-type discovery select 'dnsdisc=api-ro' get
{"codfw": {"pooled": false, "references": [], "ttl": 300}, "tags": "dnsdisc=api-ro"}
{"eqiad": {"pooled": true, "references": [], "ttl": 300}, "tags": "dnsdisc=api-ro"}

I would add also the codfw IP just to be sure, if dns-disc settings are changed we'll have trouble, going to send a code review asap. In theory meta.wikimedia.org should be able to be accessible via app/api-servers (with Host: meta.wikimedia.org), otherwise if we want to use the external IP we should use the webproxy (with the caveat that if it has troubles we have them as well). I am going to send a code review and then we'll discuss in there :)

Change 665814 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/homer/public@master] Add a mediawiki-api term to the analytics-in4 filter

https://gerrit.wikimedia.org/r/665814

Change 665814 merged by Elukey:
[operations/homer/public@master] Add a mediawiki-api term to the analytics-in4 filter

https://gerrit.wikimedia.org/r/665814