Page MenuHomePhabricator
Create Task
Maniphest T233629

Public EventGate instance and endpoint for analytics event intake: eventgate-analytics-external
Closed, ResolvedPublic15 Estimated Story Points
Actions

Description

In Q2-Q4 FY2019-2020, Analytics Engineering will collaborate with Product Infrastructure to develop tooling to use Modern Event Platform components for instrumenting client side features. To do this, they will need a publicly POST-able EventGate endpoint.

The eventgate-analytics service is already handling internal 'analytics' events from Mediawiki server side: mediawiki/cirrussearch-request and mediawiki/api-request. These event streams are relatively high volume.

We will set up a new eventgate-analytics-external instance, exposed at intake-analytics.wikimedia.org, to serve events from external clients.

See also the Event Platform - Produce Usage design document.

  • Name - eventgate-analytics-external, routed externally from intake-analytics.wikimedia.org
  • Description - Validates and produces events to Kafka
  • Timeline - Q3 2019-2020
  • Point person - Andrew Otto
  • Technologies - EventGate

Additionally, T236386: Set up eventgate-logging-external in production is also a separate publicly POST-able instance of EventGate.

Helm & k8s:

LVS, Discovery, frontend routing:

Details

SubjectRepoBranchLines +/-
Remove old eventgate-analytics LVS port from Analyitcs VLAN firewalloperations/homer/publicmaster+1 -1
Route intake-analytics.wm.org to eventgate-analytics-externaloperations/puppetproduction+5 -0
Add discovery for eventgate-analytics-externaloperations/dnsmaster+2 -0
Add check_eventgate_analyltics_external_clusteroperations/puppetproduction+6 -0
discovery: Add eventgate-analytics-external to checkoperations/puppetproduction+3 -1
lvs: Switch eventgate-analytics-external to productionoperations/puppetproduction+1 -1
lvs: Switch eventgate-analytics-external to monitoring_setupoperations/puppetproduction+1 -1
lvs: Switch eventgate-analytics-external to lvs_setupoperations/puppetproduction+1 -1
Add LVS for eventgate-analytics-external on port 4692operations/puppetproduction+42 -1
Add eventgate-analytics-external.svc entriesoperations/dnsmaster+6 -0
Add discovery for eventgate-analytics-externaloperations/puppetproduction+2 -0
EventStreamConfig - allow eventgate to produce error eventsoperations/mediawiki-configmaster+8 -0
eventgate-analytics-external - remove extra type ? in stream_config_urioperations/deployment-chartsmaster+3 -3
eventgate-analytics-external - use http://api.svc to get stream configoperations/deployment-chartsmaster+3 -3
eventgate-analytics-external - Use api.svc to get stream configoperations/deployment-chartsmaster+12 -8
Add stream_config_uri_options configeventgate-wikimediamaster+18 -4
eventgate-analytics-external - Use proper TLS Kafka portoperations/deployment-chartsmaster+3 -3
eventgate-analytics-external: Add the namespace and calico rulesoperations/deployment-chartsmaster+66 -45
eventgate-analytics-external: Add k8s tokenoperations/puppetproduction+14 -0
New eventgate-analytics-external instance using remote EventStreamConfig APIoperations/deployment-chartsmaster+597 -159
Add intake-{logging,analytics}.wikimedia.orgoperations/dnsmaster+3 -0
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Ottomata added a project: Service-deployment-requests.Sep 23 2019, 2:21 PM
Ottomata updated the task description. (Show Details)
Ottomata edited subscribers, added: Muehlenhoff; removed: MoritzMuehlenhoff.
Restricted Application edited projects, added SRE, Services; removed Services (watching). · View Herald TranscriptSep 23 2019, 2:21 PM
Ottomata moved this task from Incoming to Event Platform on the Analytics board.Sep 23 2019, 3:21 PM
Ottomata removed Ottomata as the assignee of this task.Sep 23 2019, 3:26 PM
Ottomata triaged this task as Medium priority.
Ottomata updated the task description. (Show Details)Sep 23 2019, 5:41 PM
Ottomata moved this task from Backlog to Estimated/ Discussed on the Event-Platform board.Oct 2 2019, 6:28 PM
Ottomata added a subtask: T212853: Throttling /request dispatcher that fronts eventlogging /MEP endpoint .Oct 2 2019, 6:44 PM
Ottomata added subscribers: • ema, BBlack.Oct 15 2019, 5:37 PM

I'm inclined to just use the existent eventgate-analytics backend endpoint for now. We can consider setting up another eventgate instance for public analytics events later if necessary.

We need to figure out if this endpoint should be exposed at a single domain, e.g. events-analytics.wikimedia.org, or if it should somehow be reachable from each wiki domain, e.g. en.wikipedia.org/beacon/event-analytics. Whatever we choose, the endpoint will need to be POSTable by both mediawiki client side JS as well as remote apps. I don't have a preference, so I leave that decision up to the traffic engineers. Ping @ema and @BBlack.

akosiaris added a project: serviceops-radar.Oct 30 2019, 3:26 PM
akosiaris moved this task from Inbox to Backlog on the Service-deployment-requests board.Oct 30 2019, 3:29 PM
Ottomata added a comment.Oct 30 2019, 3:29 PM

I'm inclined to just use the existent eventgate-analytics backend endpoint for now.

Recent discussions about message size limits have made me change my mind on this. We'd like to limit both the external POST body size and the size of client event messages a bit more strictly than internal services. The easiest way to do this would be with another eventgate endpoint with different limit configs.

Perhaps a new eventgate instance called eventgate-analytics-external? BTW, this is closely related to the client side error logging project. It too will have a publicly reachable eventgate instance (eventgate-logging-external?) with similar message size limits.

Ottomata mentioned this in T236386: Set up eventgate-logging-external in production.Oct 30 2019, 8:41 PM
Ottomata added a subscriber: akosiaris.Nov 12 2019, 3:14 PM

Ping @akosiaris for namespaces.

gerritbot added a comment.Dec 11 2019, 4:03 PM

Change 556411 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/dns@master] Add intake-{logging,analytics}.wikimedia.org

https://gerrit.wikimedia.org/r/556411

gerritbot added a project: Patch-For-Review.Dec 11 2019, 4:03 PM
gerritbot added a comment.Dec 12 2019, 3:13 PM

Change 556411 merged by Ottomata:
[operations/dns@master] Add intake-{logging,analytics}.wikimedia.org

https://gerrit.wikimedia.org/r/556411

Ottomata updated the task description. (Show Details)Jan 9 2020, 4:04 PM
Ottomata set the point value for this task to 8.
Ottomata renamed this task from Public EventGate endpoint for analytics event intake to Public EventGate instance and endpoint for analytics event intake: eventgate-analytics-external.Jan 9 2020, 4:39 PM
Ottomata raised the priority of this task from Medium to High.
gerritbot added a comment.Jan 9 2020, 4:39 PM

Change 563211 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] [WIP] New eventgate-analytics-external instance using remote EventStreamConfig API

https://gerrit.wikimedia.org/r/563211

Ottomata mentioned this in T240985: Create and use new schema repositories.Jan 9 2020, 6:19 PM
mpopov mentioned this in T225237: Better Use of Data.Jan 10 2020, 3:44 PM
Ottomata moved this task from Estimated/ Discussed to In Progress Before Value Streams Kickoff (August 15th) on the Event-Platform board.Feb 18 2020, 9:12 PM
gerritbot added a comment.Feb 18 2020, 9:12 PM

Change 572979 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] EventStreamConfig - allow eventgate to produce error events

https://gerrit.wikimedia.org/r/572979

gerritbot added a comment.Feb 19 2020, 8:01 PM

Change 573362 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/dns@master] Add eventgate-analytics-external.svc entries

https://gerrit.wikimedia.org/r/573362

gerritbot added a comment.Feb 19 2020, 8:16 PM

Change 573365 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add LVS for eventgate-analytics-external on port 4692

https://gerrit.wikimedia.org/r/573365

gerritbot added a comment.Feb 19 2020, 8:16 PM

Change 573366 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add discovery for eventgate-analytics-external

https://gerrit.wikimedia.org/r/573366

gerritbot added a comment.Feb 19 2020, 8:17 PM

Change 573367 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/dns@master] Add discovery for eventgate-analytics-external

https://gerrit.wikimedia.org/r/573367

gerritbot added a comment.Feb 19 2020, 8:28 PM

Change 573369 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Route intake-analytics.wm.org to eventgate-analytics-external

https://gerrit.wikimedia.org/r/573369

Ottomata claimed this task.Feb 19 2020, 8:32 PM
Ottomata updated the task description. (Show Details)
Ottomata changed the point value for this task from 8 to 15.
Ottomata added a project: Analytics-Kanban.
Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.
gerritbot added a comment.Feb 20 2020, 2:11 PM

Change 563211 merged by Ottomata:
[operations/deployment-charts@master] New eventgate-analytics-external instance using remote EventStreamConfig API

https://gerrit.wikimedia.org/r/563211

Ottomata mentioned this in rDEPLOYCHARTS8e3b0c8d5a1a: New eventgate-analytics-external instance using remote EventStreamConfig API.Feb 20 2020, 2:11 PM
gerritbot added a comment.Feb 20 2020, 3:50 PM

Change 573602 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] eventgate-analytics-external: Add k8s token

https://gerrit.wikimedia.org/r/573602

gerritbot added a comment.Feb 20 2020, 4:40 PM

Change 573624 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/deployment-charts@master] eventgate-analytics-external: Add the namespace and calico rules

https://gerrit.wikimedia.org/r/573624

gerritbot added a comment.Feb 25 2020, 2:30 PM

Change 573602 merged by Alexandros Kosiaris:
[operations/puppet@production] eventgate-analytics-external: Add k8s token

https://gerrit.wikimedia.org/r/573602

gerritbot added a comment.Feb 25 2020, 2:35 PM

Change 573624 merged by Alexandros Kosiaris:
[operations/deployment-charts@master] eventgate-analytics-external: Add the namespace and calico rules

https://gerrit.wikimedia.org/r/573624

akosiaris mentioned this in rDEPLOYCHARTSdafca4cbb3f3: eventgate-analytics-external: Add the namespace and calico rules.Feb 25 2020, 2:36 PM
akosiaris added a comment.Feb 25 2020, 2:45 PM

@Ottomata, tokens created and being propagaged across the cluster as we speak, kubernetes namespaces created and calico rules applied, give it 30mins just to be sure the tokens are everywhere and you should be able to deploy.

Ottomata added a comment.Feb 25 2020, 2:45 PM

Thank you!

Ottomata updated the task description. (Show Details)Feb 25 2020, 2:46 PM
Ottomata updated the task description. (Show Details)Feb 25 2020, 4:37 PM
Ottomata updated the task description. (Show Details)Feb 25 2020, 4:39 PM
gerritbot added a comment.Feb 25 2020, 4:59 PM

Change 574800 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-analytics-external - Use proper TLS Kafka port

https://gerrit.wikimedia.org/r/574800

gerritbot added a comment.Feb 25 2020, 5:00 PM

Change 574800 merged by Ottomata:
[operations/deployment-charts@master] eventgate-analytics-external - Use proper TLS Kafka port

https://gerrit.wikimedia.org/r/574800

Ottomata mentioned this in rDEPLOYCHARTS6d520f53e0ab: eventgate-analytics-external - Use proper TLS Kafka port.Feb 25 2020, 5:01 PM
gerritbot added a comment.Feb 25 2020, 6:25 PM

Change 574816 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[eventgate-wikimedia@master] Add stream_config_uri_options config

https://gerrit.wikimedia.org/r/574816

gerritbot added a comment.Feb 25 2020, 6:28 PM

Change 574816 merged by Ottomata:
[eventgate-wikimedia@master] Add stream_config_uri_options config

https://gerrit.wikimedia.org/r/574816

gerritbot added a comment.Feb 25 2020, 7:14 PM

Change 574848 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-analytics-external - Use api.svc to get stream config

https://gerrit.wikimedia.org/r/574848

gerritbot added a comment.Feb 25 2020, 7:18 PM

Change 574848 merged by Ottomata:
[operations/deployment-charts@master] eventgate-analytics-external - Use api.svc to get stream config

https://gerrit.wikimedia.org/r/574848

Ottomata mentioned this in rDEPLOYCHARTS0840dbb4cbc1: eventgate-analytics-external - Use api.svc to get stream config.Feb 25 2020, 7:20 PM
gerritbot added a comment.Feb 25 2020, 7:24 PM

Change 574850 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-analytics-external - use http://api.svc to get stream config

https://gerrit.wikimedia.org/r/574850

gerritbot added a comment.Feb 25 2020, 7:25 PM

Change 574850 merged by Ottomata:
[operations/deployment-charts@master] eventgate-analytics-external - use http://api.svc to get stream config

https://gerrit.wikimedia.org/r/574850

Ottomata mentioned this in rDEPLOYCHARTSb6d42d50f83d: eventgate-analytics-external - use http://api.svc to get stream config.Feb 25 2020, 7:25 PM
gerritbot added a comment.Feb 25 2020, 7:42 PM

Change 574851 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-analytics-external - remove extra type ? in stream_config_uri

https://gerrit.wikimedia.org/r/574851

gerritbot added a comment.Feb 25 2020, 7:43 PM

Change 574851 merged by Ottomata:
[operations/deployment-charts@master] eventgate-analytics-external - remove extra type ? in stream_config_uri

https://gerrit.wikimedia.org/r/574851

Ottomata mentioned this in rDEPLOYCHARTSa0edeb0d20e2: eventgate-analytics-external - remove extra type ? in stream_config_uri.Feb 25 2020, 7:43 PM
Ottomata updated the task description. (Show Details)Feb 25 2020, 8:00 PM
gerritbot added a comment.Mar 2 2020, 7:30 PM

Change 572979 merged by Ottomata:
[operations/mediawiki-config@master] EventStreamConfig - allow eventgate to produce error events

https://gerrit.wikimedia.org/r/572979

Stashbot added a comment.Mar 2 2020, 7:35 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T19:35:13Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: EventStreamConfig - allow eventgate-analytics-external to produce error events - T233629 (duration: 00m 56s)

Stashbot added a comment.Mar 3 2020, 2:13 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-03T14:13:27Z] <ottomata> beginning procedure to add LVS and discovery for eventgate-analytics-external - T233629

gerritbot added a comment.Mar 3 2020, 2:14 PM

Change 573366 merged by Ottomata:
[operations/puppet@production] Add discovery for eventgate-analytics-external

https://gerrit.wikimedia.org/r/573366

gerritbot added a comment.Mar 3 2020, 2:18 PM

Change 573362 merged by Ottomata:
[operations/dns@master] Add eventgate-analytics-external.svc entries

https://gerrit.wikimedia.org/r/573362

Ottomata updated the task description. (Show Details)Mar 3 2020, 2:29 PM
gerritbot added a comment.Mar 3 2020, 4:28 PM

Change 573365 merged by Alexandros Kosiaris:
[operations/puppet@production] Add LVS for eventgate-analytics-external on port 4692

https://gerrit.wikimedia.org/r/573365

gerritbot added a comment.Mar 3 2020, 7:52 PM

Change 576431 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add check_eventgate_analyltics_external_cluster

https://gerrit.wikimedia.org/r/576431

• Mholloway unsubscribed.Mar 3 2020, 7:57 PM
gerritbot added a comment.Mar 4 2020, 9:37 AM

Change 576636 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] discovery: Add eventgate-analytics-external to check

https://gerrit.wikimedia.org/r/576636

gerritbot added a comment.Mar 4 2020, 9:39 AM

Change 576637 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] lvs: Switch eventgate-analytics-external to lvs_setup

https://gerrit.wikimedia.org/r/576637

gerritbot added a comment.Mar 4 2020, 9:47 AM

Change 576636 merged by Alexandros Kosiaris:
[operations/puppet@production] discovery: Add eventgate-analytics-external to check

https://gerrit.wikimedia.org/r/576636

gerritbot added a comment.Mar 4 2020, 9:48 AM

Change 576637 merged by Alexandros Kosiaris:
[operations/puppet@production] lvs: Switch eventgate-analytics-external to lvs_setup

https://gerrit.wikimedia.org/r/576637

gerritbot added a comment.Mar 4 2020, 9:56 AM

Change 576646 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] lvs: Switch eventgate-analytics-external to monitoring_setup

https://gerrit.wikimedia.org/r/576646

gerritbot added a comment.Mar 4 2020, 9:58 AM

Change 576647 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] lvs: Switch eventgate-analytics-external to production

https://gerrit.wikimedia.org/r/576647

gerritbot added a comment.Mar 4 2020, 1:06 PM

Change 576646 merged by Alexandros Kosiaris:
[operations/puppet@production] lvs: Switch eventgate-analytics-external to monitoring_setup

https://gerrit.wikimedia.org/r/576646

gerritbot added a comment.Mar 4 2020, 2:49 PM

Change 576647 merged by Alexandros Kosiaris:
[operations/puppet@production] lvs: Switch eventgate-analytics-external to production

https://gerrit.wikimedia.org/r/576647

gerritbot added a comment.Mar 4 2020, 4:15 PM

Change 576873 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/homer/public@master] Remove old eventgate-analytics LVS port from Analyitcs VLAN firewall

https://gerrit.wikimedia.org/r/576873

Ottomata updated the task description. (Show Details)Mar 4 2020, 4:26 PM
gerritbot added a comment.Mar 4 2020, 4:31 PM

Change 576431 merged by Ottomata:
[operations/puppet@production] Add check_eventgate_analyltics_external_cluster

https://gerrit.wikimedia.org/r/576431

Ottomata updated the task description. (Show Details)Mar 4 2020, 4:31 PM
gerritbot added a comment.Mar 4 2020, 4:53 PM

Change 573367 merged by Ottomata:
[operations/dns@master] Add discovery for eventgate-analytics-external

https://gerrit.wikimedia.org/r/573367

Ottomata updated the task description. (Show Details)Mar 4 2020, 4:56 PM
gerritbot added a comment.Mar 5 2020, 9:52 AM

Change 573369 merged by Alexandros Kosiaris:
[operations/puppet@production] Route intake-analytics.wm.org to eventgate-analytics-external

https://gerrit.wikimedia.org/r/573369

Ottomata updated the task description. (Show Details)Mar 5 2020, 3:29 PM
Ottomata moved this task from In Progress to Done on the Analytics-Kanban board.
gerritbot added a comment.Mar 5 2020, 5:27 PM

Change 576873 merged by Elukey:
[operations/homer/public@master] Remove old eventgate-analytics LVS port from Analyitcs VLAN firewall

https://gerrit.wikimedia.org/r/576873

• Nuria closed this task as Resolved.Mar 5 2020, 9:24 PM
Ottomata mentioned this in T226986: Client side error logging production launch.Sep 16 2020, 6:14 PM
Ottomata mentioned this in T263049: Avoid extra HTTPS connections for most Event Platform beacons.Sep 17 2020, 3:49 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL