Page MenuHomePhabricator

Ottomata (Andrew Otto)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Oct 9 2014, 4:50 PM (240 w, 4 d)
Availability
Available
IRC Nick
ottomata
LDAP User
Ottomata
MediaWiki User
Ottomata [ Global Accounts ]

Recent Activity

Fri, May 17

Ottomata renamed T222253: Upgrade Spark to 2.4.x from Upgrade Spark to 2.4.2 to Upgrade Spark to 2.4.x.
Fri, May 17, 7:40 PM · Analytics-Kanban, Analytics, Analytics-Cluster
Ottomata added a comment to T222253: Upgrade Spark to 2.4.x.

This only is a problem with pyarrow; we could upgrade spark to 2.4.3, but pyarrow wouldn't work.

Fri, May 17, 7:40 PM · Analytics-Kanban, Analytics, Analytics-Cluster
Ottomata added a comment to T222254: Pyspark on SWAP: Py4JJavaError: Import Error: no module named pyarrow.

Quite a few of the worker nodes DID NOT have pyarrow! They were running my older 2.3.1-bin-hadoop2.6-1~stretch1 version that didn't have pyarrow included. I just ran a cumin command to upgrade those that didn't to 2.3.1-bin-hadoop2.6-3~stretch1.

Fri, May 17, 7:36 PM · Analytics, Analytics-Cluster
Ottomata assigned T222267: Port usage of mediawiki_ApiAction to mediawiki_api_request to EvanProdromou.
Fri, May 17, 7:06 PM · Analytics-Kanban, EventBus, Analytics
Ottomata moved T222253: Upgrade Spark to 2.4.x from Next Up to In Progress on the Analytics-Kanban board.
Fri, May 17, 7:05 PM · Analytics-Kanban, Analytics, Analytics-Cluster
Ottomata edited projects for T223493: rack/setup/install kafka-main200[1-5], added: Analytics; removed Analytics-Kanban.
Fri, May 17, 7:05 PM · Analytics, Patch-For-Review, ops-codfw, Operations
Ottomata added a comment to T222254: Pyspark on SWAP: Py4JJavaError: Import Error: no module named pyarrow.

@JAllemandou I couldn't get your example to fail in either local or YARN.

Fri, May 17, 7:00 PM · Analytics, Analytics-Cluster
Ottomata added a comment to T222254: Pyspark on SWAP: Py4JJavaError: Import Error: no module named pyarrow.

So, I'm not sure why pyarrow wouldn't load in YARN, it is available on all the workers. hhm.

Fri, May 17, 6:47 PM · Analytics, Analytics-Cluster
Ottomata added a comment to T222253: Upgrade Spark to 2.4.x.

Spark 2.4.3 uses Arrow 0.10. pyarrow 0.10 has an issue where it builds against an older (1.10?) numpy version. We have 1.12 from Debian Stretch installed.

Fri, May 17, 6:44 PM · Analytics-Kanban, Analytics, Analytics-Cluster
Ottomata added a comment to T222254: Pyspark on SWAP: Py4JJavaError: Import Error: no module named pyarrow.

Ha, python-arrow is https://arrow.readthedocs.io/en/latest/, not Apache pyarrow

Fri, May 17, 6:12 PM · Analytics, Analytics-Cluster
Ottomata added a comment to T203597: CentralNoticeImpression occasionaly fails validation on device enum field.

@AndyRussG ping, not sure if anything happened on this.

Fri, May 17, 5:03 PM · Fundraising Sprint James Bond: Licensed to Share, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Analytics, Fundraising-Analysis
Ottomata closed T211983: EventLogging client side serialization saves integer decimals as decimal-less numbers as Resolved.

We are now using the JSONSchemas directly to create the Hive tables, so as long as folks properly set integers to integer type, and anything else to number, we should be good!

Fri, May 17, 5:02 PM · Analytics, Analytics-EventLogging
Ottomata closed T221379: Analytics admins no longer have needed Jenkins credentials as Resolved.
Fri, May 17, 5:00 PM · Release-Engineering-Team
Ottomata merged task T215043: Upgrade to Spark 2.4.0 into T222253: Upgrade Spark to 2.4.x.
Fri, May 17, 4:59 PM · Analytics
Ottomata merged T215043: Upgrade to Spark 2.4.0 into T222253: Upgrade Spark to 2.4.x.
Fri, May 17, 4:59 PM · Analytics-Kanban, Analytics, Analytics-Cluster
Ottomata renamed T211248: Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main from Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to EventGate to Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main.
Fri, May 17, 4:57 PM · Patch-For-Review, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata renamed T188136: Modern Event Platform: Stream Intake Service: Migrate Mediawiki monolog Kafka uses to eventgate-analytics from Modern Event Platform: Stream Intake Service: Migrate Mediawiki monolog Kafka uses to EventGate to Modern Event Platform: Stream Intake Service: Migrate Mediawiki monolog Kafka uses to eventgate-analytics.
Fri, May 17, 4:57 PM · Discovery, Analytics, Patch-For-Review, Analytics-Cluster
Ottomata added a comment to T219544: Make hadoop cluster able to push to swift .

A less elegant alternative would be to just write a wrapper that downloaded from HDFS to local filesystem and then uploaded to swift :/

Fri, May 17, 4:46 PM · Analytics-Kanban, Research, Operations, Discovery, Analytics
Ottomata updated the task description for T185233: Modern Event Platform (TEC2).
Fri, May 17, 3:55 PM · Core Platform Team Backlog (Watching / External), Core Platform Team (Modern Event Platform (TEC2)), Goal, Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata added a comment to T219544: Make hadoop cluster able to push to swift .

I think either of those solutions probably just made this task a lot harder...

Fri, May 17, 3:53 PM · Analytics-Kanban, Research, Operations, Discovery, Analytics
Ottomata added a comment to T223415: Strategy to be able to test eventlogging in beta in the absence of mysql .

Or, maybe we don't even need the files? Will people just log in and tail -f them? If so, we could just write a little wrapper script (or even quick simple GUI???) that wraps kafkacat (or eventlogging-consumer !). E.g.

Fri, May 17, 3:52 PM · Analytics, Analytics-EventLogging
Ottomata added a comment to T223493: rack/setup/install kafka-main200[1-5].

Awesome, ya that works!

Fri, May 17, 3:50 PM · Analytics, Patch-For-Review, ops-codfw, Operations
Ottomata created T223628: Replace Camus with Kafka Connect for event data imports.
Fri, May 17, 3:49 PM · Analytics, EventBus
Ottomata created T223626: Kafka Connect development work.
Fri, May 17, 3:39 PM · Analytics, EventBus
Ottomata updated the task description for T185233: Modern Event Platform (TEC2).
Fri, May 17, 3:26 PM · Core Platform Team Backlog (Watching / External), Core Platform Team (Modern Event Platform (TEC2)), Goal, Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata updated the task description for T185233: Modern Event Platform (TEC2).
Fri, May 17, 3:26 PM · Core Platform Team Backlog (Watching / External), Core Platform Team (Modern Event Platform (TEC2)), Goal, Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata updated the task description for T185233: Modern Event Platform (TEC2).
Fri, May 17, 3:25 PM · Core Platform Team Backlog (Watching / External), Core Platform Team (Modern Event Platform (TEC2)), Goal, Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata renamed T206824: Make it possible to use $ref in JSONSchemas from Decide whether to use schema references in the schema registry to Make it possible to use $ref in JSONSchemas.
Fri, May 17, 3:21 PM · Core Platform Team Backlog (Designing), Services (designing), Core Platform Team (Modern Event Platform (TEC2)), Analytics-EventLogging, EventBus, Analytics
Ottomata updated subscribers of T223493: rack/setup/install kafka-main200[1-5].

@herron, it might be worth renaming these hosts! E.g. kafka-main200[4-8] would be more appropriate! :)

Fri, May 17, 2:33 PM · Analytics, Patch-For-Review, ops-codfw, Operations
Ottomata added a comment to T223415: Strategy to be able to test eventlogging in beta in the absence of mysql .

Can we just use kafkatee and write kafka topics to individual per-schema files? I think folks really just need to verify that their events are flowing in as expected, not really query them in beta.

Fri, May 17, 2:31 PM · Analytics, Analytics-EventLogging
Ottomata added a comment to T223387: CamusPartitionChecker email alert should be more descriptive.

Whoops, yes, forgot to tag a Bug.

Fri, May 17, 2:30 PM · Analytics-Kanban, Analytics
Ottomata added a comment to T219544: Make hadoop cluster able to push to swift .

Is the tempauth middleware just for beta? I am pretty sure the same commands should work in production too, in case swift there is using something else for auth.

Fri, May 17, 2:27 PM · Analytics-Kanban, Research, Operations, Discovery, Analytics

Wed, May 15

Ottomata added a comment to T219544: Make hadoop cluster able to push to swift .

I seem to be able to use hadoop distcp while defining the required properties on the CLI, like:

Wed, May 15, 8:09 PM · Analytics-Kanban, Research, Operations, Discovery, Analytics
Ottomata moved T219544: Make hadoop cluster able to push to swift from Next Up to In Progress on the Analytics-Kanban board.
Wed, May 15, 7:49 PM · Analytics-Kanban, Research, Operations, Discovery, Analytics
Ottomata added a project to T219544: Make hadoop cluster able to push to swift : Analytics-Kanban.
Wed, May 15, 7:49 PM · Analytics-Kanban, Research, Operations, Discovery, Analytics
Ottomata committed rDEPLOYCHARTS048cca654bf5: Remove deprecated eventgate-analytics chart (authored by Ottomata).
Remove deprecated eventgate-analytics chart
Wed, May 15, 7:13 PM
Ottomata committed rDEPLOYCHARTS996da33420a4: Remove deprecated eventgate-analytics chart (authored by Ottomata).
Remove deprecated eventgate-analytics chart
Wed, May 15, 4:20 PM
Ottomata added a comment to T217142: [WIP] [Proposal] Use the Kafka-Logstash logging infrastructure to log client-side errors.

We should eventually review the custom fields we set, but I believe it would POST just fine with the addition of $schema and meta.stream.

Wed, May 15, 4:19 PM · Patch-For-Review, User-herron, Reading-Infrastructure-Team-Backlog, Wikimedia-Logstash
Ottomata moved T221848: Upgrade python-kafka from In Progress to In Code Review on the Analytics-Kanban board.
Wed, May 15, 4:01 PM · Patch-For-Review, Analytics-Kanban, Performance-Team (Radar), Operations, Analytics, Analytics-EventLogging
Ottomata added a project to T223389: Make it (just a bit) easier to spin up Hadoop cluster in Cloud VMs: Analytics-Cluster.
Wed, May 15, 3:52 PM · Analytics-Cluster, Analytics
Ottomata created T223389: Make it (just a bit) easier to spin up Hadoop cluster in Cloud VMs.
Wed, May 15, 3:50 PM · Analytics-Cluster, Analytics
Ottomata added a comment to T217142: [WIP] [Proposal] Use the Kafka-Logstash logging infrastructure to log client-side errors.

Ok, I've set up an eventgate-logging instance in beta at http://deployment-eventgate-1.deployment-prep.eqiad.wmflabs:8992/v1/events.

Wed, May 15, 3:32 PM · Patch-For-Review, User-herron, Reading-Infrastructure-Team-Backlog, Wikimedia-Logstash
Ottomata created T223387: CamusPartitionChecker email alert should be more descriptive.
Wed, May 15, 2:42 PM · Analytics-Kanban, Analytics

Tue, May 14

Ottomata added a comment to T222101: Update ServerSideAccountCreation schema whitelist.

Added a few others as reviewers who would probably know better than Joal or I! :)

Tue, May 14, 9:38 PM · Growth-Team (Current Sprint), Analytics, Product-Analytics
Ottomata added a comment to T219544: Make hadoop cluster able to push to swift .

Found some better docs here:
https://docs.openstack.org/sahara/latest/user/hadoop-swift.html

Tue, May 14, 7:20 PM · Analytics-Kanban, Research, Operations, Discovery, Analytics
Ottomata added a project to T222253: Upgrade Spark to 2.4.x: Analytics-Kanban.
Tue, May 14, 4:15 PM · Analytics-Kanban, Analytics, Analytics-Cluster
Ottomata updated subscribers of T222301: Upgrade pandas in spark SWAP notebooks.

Oh @Groceryheist! It is possible to run distributed pyspark jobs with custom virtualenvs. I've never done it, but @EBernhardson knows how. I think he has some documentation, but I can't find it.

Tue, May 14, 4:12 PM · Analytics
Ottomata moved T214080: Rewrite Avro schemas (ApiAction, CirrusSearchRequestSet) as JSONSchema and produce to EventGate from In Progress to Done on the Analytics-Kanban board.
Tue, May 14, 3:44 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), MW-1.33-notes (1.33.0-wmf.21; 2019-03-12), Analytics-Kanban, Patch-For-Review, Services (watching), Discovery, Analytics-EventLogging, EventBus, Analytics
Ottomata moved T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main from Ready to Deploy to Done on the Analytics-Kanban board.
Tue, May 14, 3:44 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata added a comment to T222966: > 2% of API wall time spent generating UUIDs.

Q: Is there any reason constructing the event couldn't be deferred until after the API request finishes? We already defer the send...

Tue, May 14, 3:05 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Core Platform Team Backlog (Next), EventBus, Core Platform Team (Modern Event Platform (TEC2)), Services-next, Analytics, Performance
Ottomata added a comment to T222966: > 2% of API wall time spent generating UUIDs.

If Mediawiki retries the HTTP request to produce the event, or EventGate's (or change-prop's or EventLogging's or whatever's) Kafka client retries the Kafka produce request, it is possible to get duplicate events. Kafka is (by default) at least once guarantee.

Tue, May 14, 2:59 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Core Platform Team Backlog (Next), EventBus, Core Platform Team (Modern Event Platform (TEC2)), Services-next, Analytics, Performance
Ottomata moved T222822: Make EventBus extension support configurable per-event/stream EventServiceName from Backlog to Next Up on the EventBus board.
Tue, May 14, 1:34 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Core Platform Team (Modern Event Platform (TEC2)), Core Platform Team Kanban (Doing), Services (doing), EventBus, Analytics
Ottomata moved T222966: > 2% of API wall time spent generating UUIDs from Backlog to Next Up on the EventBus board.
Tue, May 14, 1:34 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Core Platform Team Backlog (Next), EventBus, Core Platform Team (Modern Event Platform (TEC2)), Services-next, Analytics, Performance
Ottomata moved T222268: Port usage of mediawiki_CirrusSearchRequestSet to mediawiki_cirrussearch_request from Backlog to In Progress on the EventBus board.
Tue, May 14, 1:34 PM · Product-Analytics, Discovery-Search, Discovery-Analysis, Discovery, Analytics-Kanban, EventBus, Analytics
Ottomata moved T222962: Use new eventgate chart release analytics for eventgate-analytics service. from Backlog to Done on the EventBus board.
Tue, May 14, 1:33 PM · Patch-For-Review, serviceops, Analytics-Kanban, Services (watching), EventBus, Analytics
Ottomata moved T215442: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals from In Progress to Done on the EventBus board.
Tue, May 14, 1:33 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, Analytics-Kanban, EventBus, Analytics
Ottomata moved T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main from In Progress to Done on the EventBus board.
Tue, May 14, 1:33 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata assigned T222268: Port usage of mediawiki_CirrusSearchRequestSet to mediawiki_cirrussearch_request to EBernhardson.

Heya Erik! I know you said you would try to get to this after the Hackathon this coming weekend. I'm assigning to you so we don't forget.

Tue, May 14, 1:30 PM · Product-Analytics, Discovery-Search, Discovery-Analysis, Discovery, Analytics-Kanban, EventBus, Analytics
Ottomata added a comment to T222267: Port usage of mediawiki_ApiAction to mediawiki_api_request.

@EvanProdromou whatcha think about a timeline on this? Turning off the ApiAction events is a goal for this quarter, and this task has to be resolved before we do that.

Tue, May 14, 1:29 PM · Analytics-Kanban, EventBus, Analytics
Ottomata added a comment to T128592: Add redundancy to IRC recent changes service.

It'd be really fun to build a replaceement IRC service based on the mediawiki.recentchange Kafka topic. Perhaps one day I will have time... :)

Tue, May 14, 1:27 PM · Operations, Availability (MediaWiki-MultiDC), codfw-rollout
Ottomata added a comment to T222966: > 2% of API wall time spent generating UUIDs.

Could the WebRequest::getRequestId() suffice instead

We also log this as meta.request_id. Since you mention it, we are hoping that https://gerrit.wikimedia.org/r/c/mediawiki/core/+/505668 and T221976: Have Varnish set the `X-Request-Id` header for incoming external requests get solved soonish. It will be pretty cool to be able to group several events streams by the request_id, to see all of the events that a single http request might generate.

Tue, May 14, 1:25 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Core Platform Team Backlog (Next), EventBus, Core Platform Team (Modern Event Platform (TEC2)), Services-next, Analytics, Performance
Ottomata closed T97226: Include the request ID in API request logs as Resolved.

This is done in the new mediawiki/api/request event as part of T214080: Rewrite Avro schemas (ApiAction, CirrusSearchRequestSet) as JSONSchema and produce to EventGate

Tue, May 14, 1:23 PM · MediaWiki-API
Ottomata added a comment to T220235: Migrate Beta cluster services to use Kubernetes .

eventgate-main is up in both beta in via service::docker and in prod k8s. Both eventgate-analytics and eventgate-main are running in beta on the deployment-eventgate-1 VM.

Tue, May 14, 1:20 PM · Patch-For-Review, Editing-team, Core Platform Team Backlog (Next), Services (next), Kubernetes, Release Pipeline, serviceops, Beta-Cluster-Infrastructure

Mon, May 13

Ottomata set the point value for T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main to 13.
Mon, May 13, 7:30 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata moved T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main from In Progress to Ready to Deploy on the Analytics-Kanban board.
Mon, May 13, 7:30 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata moved T222899: Set up LVS for eventgate-main on port 32192 from Next Up to Ready to Deploy on the Analytics-Kanban board.
Mon, May 13, 7:30 PM · Analytics-Kanban, Patch-For-Review, serviceops, Core Platform Team Backlog (Watching / External), Services (watching), EventBus, Analytics
Ottomata added a comment to T211248: Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main.

@Pchelolo T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main is pretty much done! We are ready to start porting events, I guess after T222822: Make EventBus extension support configurable per-event/stream EventServiceName is done too.

Mon, May 13, 7:30 PM · Patch-For-Review, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata removed a parent task for T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main : T206785: Modern Event Platform: Stream Intake Service (EventGate): Implementation.
Mon, May 13, 7:27 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata removed a subtask for T206785: Modern Event Platform: Stream Intake Service (EventGate): Implementation: T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main .
Mon, May 13, 7:27 PM · Analytics-Kanban, Patch-For-Review, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata added a subtask for T211248: Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main: T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main .
Mon, May 13, 7:27 PM · Patch-For-Review, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata added a parent task for T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main : T211248: Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main.
Mon, May 13, 7:27 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata added a subtask for T211248: Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main: T222822: Make EventBus extension support configurable per-event/stream EventServiceName.
Mon, May 13, 7:26 PM · Patch-For-Review, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics-Kanban
Ottomata edited parent tasks for T222822: Make EventBus extension support configurable per-event/stream EventServiceName, added: T211248: Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main; removed: T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main .
Mon, May 13, 7:26 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Core Platform Team (Modern Event Platform (TEC2)), Core Platform Team Kanban (Doing), Services (doing), EventBus, Analytics
Ottomata removed a subtask for T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main : T222822: Make EventBus extension support configurable per-event/stream EventServiceName.
Mon, May 13, 7:26 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata updated the task description for T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main .
Mon, May 13, 7:25 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata assigned T222899: Set up LVS for eventgate-main on port 32192 to akosiaris.
Mon, May 13, 7:24 PM · Analytics-Kanban, Patch-For-Review, serviceops, Core Platform Team Backlog (Watching / External), Services (watching), EventBus, Analytics
Ottomata added a comment to T222966: > 2% of API wall time spent generating UUIDs.

Am fine with any uuid that matches the expected format. IIRC, we chose v4 because Gabriel liked the way it encoded the event timestamp. With the meta.dt in every event anyway, I don't see any reason to require that.

Mon, May 13, 6:59 PM · MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Core Platform Team Backlog (Next), EventBus, Core Platform Team (Modern Event Platform (TEC2)), Services-next, Analytics, Performance
Ottomata added a comment to T222899: Set up LVS for eventgate-main on port 32192.

@akosiaris when you have time tomorrow: https://gerrit.wikimedia.org/r/c/operations/dns/+/509912

Mon, May 13, 6:33 PM · Analytics-Kanban, Patch-For-Review, serviceops, Core Platform Team Backlog (Watching / External), Services (watching), EventBus, Analytics
Ottomata moved T222962: Use new eventgate chart release analytics for eventgate-analytics service. from Next Up to Done on the Analytics-Kanban board.
Mon, May 13, 6:23 PM · Patch-For-Review, serviceops, Analytics-Kanban, Services (watching), EventBus, Analytics
Ottomata set the point value for T222962: Use new eventgate chart release analytics for eventgate-analytics service. to 5.
Mon, May 13, 6:23 PM · Patch-For-Review, serviceops, Analytics-Kanban, Services (watching), EventBus, Analytics
Ottomata added a comment to T222933: Upgrade R in SWAP notebooks to 3.4+.

This isn't easy :(

Mon, May 13, 3:28 PM · Analytics-SWAP, Analytics
Ottomata updated the task description for T218346: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main .
Mon, May 13, 2:46 PM · serviceops, Patch-For-Review, Analytics-Kanban, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata renamed T222962: Use new eventgate chart release analytics for eventgate-analytics service. from Change LVS port for eventlogging-analytics from 31192 to 33192 to Use new eventgate chart release analytics for eventgate-analytics service..
Mon, May 13, 2:23 PM · Patch-For-Review, serviceops, Analytics-Kanban, Services (watching), EventBus, Analytics
Ottomata added a comment to T222962: Use new eventgate chart release analytics for eventgate-analytics service..

Ok, in that case, it will be much less annoying to temporarily disable the events in mediawiki-config, purge the deployed eventgate-analytics chart in staging and production helm/k8s, and then upgrade the new eventgate release analytics chart to use the same 31192 LVS port.

Mon, May 13, 2:22 PM · Patch-For-Review, serviceops, Analytics-Kanban, Services (watching), EventBus, Analytics
Ottomata added a comment to T220235: Migrate Beta cluster services to use Kubernetes .

Could we use image version: latest in beta hiera? And somehow pull down the new latest and restart the image whenever a new version is created and uploaded to the registry?

Mon, May 13, 1:27 PM · Patch-For-Review, Editing-team, Core Platform Team Backlog (Next), Services (next), Kubernetes, Release Pipeline, serviceops, Beta-Cluster-Infrastructure

Fri, May 10

Ottomata added a comment to T222962: Use new eventgate chart release analytics for eventgate-analytics service..

Hm, question. Currently mediawiki-config ProductionServices.php has:

Fri, May 10, 3:58 PM · Patch-For-Review, serviceops, Analytics-Kanban, Services (watching), EventBus, Analytics
Ottomata created T222962: Use new eventgate chart release analytics for eventgate-analytics service..
Fri, May 10, 3:56 PM · Patch-For-Review, serviceops, Analytics-Kanban, Services (watching), EventBus, Analytics
Ottomata added a comment to T218609: Figure out future for newly created deployment-prep jessie instances.

I verified that this works. I had to run puppet, apt-get update, run puppet, and then run puppet again before it worked, but it did! :)

Fri, May 10, 3:47 PM · Beta-Cluster-Infrastructure
Ottomata added a comment to T196066: Add prometheus metrics for varnishkafka instances running on caching hosts.

I'm not sure I see the value in breaking up the broker name into broker_hostname, broker_id, and broker_port labels.

Fri, May 10, 1:32 PM · Patch-For-Review, Analytics-Kanban, Traffic, Operations, Analytics
Ottomata added a comment to T221848: Upgrade python-kafka.

That'd be fine!

Fri, May 10, 1:27 PM · Patch-For-Review, Analytics-Kanban, Performance-Team (Radar), Operations, Analytics, Analytics-EventLogging

Thu, May 9

Ottomata added a comment to T218609: Figure out future for newly created deployment-prep jessie instances.

GREAT I just made a stretch instance deployment-eventgate-1, and will move eventgate-analytics there. I will also run eventgate-main there in a different container, but I need to use the same image for both!

Thu, May 9, 8:46 PM · Beta-Cluster-Infrastructure
Ottomata added a comment to T220235: Migrate Beta cluster services to use Kubernetes .

An example of environmental differences: service-runner uses statsd. In prod we use prometheus-statsd-exporter in a k8s container with service specific metric mappings to get those metrics into prometheus.

Thu, May 9, 8:43 PM · Patch-For-Review, Editing-team, Core Platform Team Backlog (Next), Services (next), Kubernetes, Release Pipeline, serviceops, Beta-Cluster-Infrastructure
Ottomata moved T206785: Modern Event Platform: Stream Intake Service (EventGate): Implementation from In Progress to Done on the EventBus board.
Thu, May 9, 7:33 PM · Analytics-Kanban, Patch-For-Review, Core Platform Team Backlog (Watching / External), Services (watching), Analytics-EventLogging, EventBus, Analytics
Ottomata added a comment to T215442: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals.

I'd say this task is done. We still need to migrate all of the EventBus ones to use JSONSchemas from schema.svc, but we'll do that one at a time as we move them to eventgate-main as part of T211248: Modern Event Platform: Stream Intake Service: Migrate Mediawiki Eventbus events to eventgate-main

Thu, May 9, 6:41 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, Analytics-Kanban, EventBus, Analytics
Ottomata changed the point value for T215442: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals from 8 to 13.
Thu, May 9, 6:40 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, Analytics-Kanban, EventBus, Analytics
Ottomata moved T215442: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals from Ready to Deploy to Done on the Analytics-Kanban board.
Thu, May 9, 6:40 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, Analytics-Kanban, EventBus, Analytics
Ottomata added a comment to T215442: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals.
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ClickTiming` (total # refined records: 78241)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppiOSFeed` (total # refined records: 9295)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`WikibaseTermboxInteraction` (total # refined records: 78)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppProtectedEditAttempt` (total # refined records: 67)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppArticleSuggestions` (total # refined records: 531)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`HomepageModule` (total # refined records: 9)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppFeed` (total # refined records: 2318)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppNotificationPreferences` (total # refined records: 1)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppiOSLoginAction` (total # refined records: 30)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppiOSSettingAction` (total # refined records: 5)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`QuickSurveysResponses` (total # refined records: 1195)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ContentTranslationCTA` (total # refined records: 1152)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ServerSideAccountCreation` (total # refined records: 566)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`CitationUsagePageLoad` (total # refined records: 596)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWebSearch` (total # refined records: 1352)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EditAttemptStep` (total # refined records: 36698)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppOnThisDay` (total # refined records: 43)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ElementTiming` (total # refined records: 32441)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`CentralAuth` (total # refined records: 3618)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MediaViewer` (total # refined records: 70664)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`UploadWizardErrorFlowEvent` (total # refined records: 201)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`WikidataCompletionSearchClicks` (total # refined records: 2638)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`CitationUsage` (total # refined records: 20)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppLanguageSearching` (total # refined records: 353)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ContentTranslation` (total # refined records: 49)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`CentralNoticeTiming` (total # refined records: 32307)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`LayoutJank` (total # refined records: 15295)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppLinkPreview` (total # refined records: 301864)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppPageScroll` (total # refined records: 10883)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppFeedConfigure` (total # refined records: 163)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`PrefUpdate` (total # refined records: 4395)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MultimediaViewerAttribution` (total # refined records: 1061)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`Popups` (total # refined records: 1)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppNavMenu` (total # refined records: 16)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EchoInteraction` (total # refined records: 3002)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`NavigationTiming` (total # refined records: 69744)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppAppearanceSettings` (total # refined records: 1016)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`UploadWizardUploadFlowEvent` (total # refined records: 312)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`HomepageVisit` (total # refined records: 1)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`SearchSatisfactionErrors` (total # refined records: 247)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`GuidedTourExited` (total # refined records: 53)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`UploadWizardExceptionFlowEvent` (total # refined records: 24)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ReadingDepth` (total # refined records: 3270556)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MultimediaViewerDuration` (total # refined records: 680)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EventError` (total # refined records: 1577)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`WikipediaPortal` (total # refined records: 1345)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppIntents` (total # refined records: 723)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppShareAFact` (total # refined records: 12817)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`UniversalLanguageSelector` (total # refined records: 3713)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`TranslationRecommendationAPIRequests` (total # refined records: 14)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EventTiming` (total # refined records: 32311)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppWiktionaryPopup` (total # refined records: 8)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppNotificationInteraction` (total # refined records: 16)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`SaveTiming` (total # refined records: 11356)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`GuidedTourGuiderHidden` (total # refined records: 39)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`CpuBenchmark` (total # refined records: 45924)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppSessions` (total # refined records: 674424)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`TestSearchSatisfaction2` (total # refined records: 25681)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppiOSReadingLists` (total # refined records: 317)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`AdvancedSearchRequest` (total # refined records: 23028)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`HelpPanel` (total # refined records: 110)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`UploadWizardStep` (total # refined records: 1199)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppInstallReferrer` (total # refined records: 888)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ContentTranslationError` (total # refined records: 4)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`CentralNoticeBannerHistory` (total # refined records: 1823)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`CentralNoticeImpression` (total # refined records: 51072)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`VirtualPageView` (total # refined records: 3581226)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppToCInteraction` (total # refined records: 128318)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`GettingStartedRedirectImpression` (total # refined records: 264)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppLogin` (total # refined records: 832)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`QuickSurveyInitiation` (total # refined records: 44386)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`UploadWizardFlowEvent` (total # refined records: 7)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EchoMail` (total # refined records: 454)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppEdit` (total # refined records: 1223)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppCreateAccount` (total # refined records: 214)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWebMainMenuClickTracking` (total # refined records: 5598)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppTabs` (total # refined records: 982)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppiOSSearch` (total # refined records: 17366)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppReadingLists` (total # refined records: 4608)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ExternalGuidance` (total # refined records: 14027)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`RUMSpeedIndex` (total # refined records: 47637)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppRandomizer` (total # refined records: 167)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MultimediaViewerDimensions` (total # refined records: 3927)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppMediaGallery` (total # refined records: 422)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`GuidedTourButtonClick` (total # refined records: 318)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`VisualEditorFeatureUse` (total # refined records: 2529)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppFindInPage` (total # refined records: 532)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppiOSSessions` (total # refined records: 14466)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppStuffHappens` (total # refined records: 1)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ContentTranslationSuggestion` (total # refined records: 80)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`WikimediaBlogVisit` (total # refined records: 45)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`PaintTiming` (total # refined records: 100002)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`GuidedTourExternalLinkActivation` (total # refined records: 37)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`UploadWizardTutorialActions` (total # refined records: 530)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ResourceTiming` (total # refined records: 47262)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MediaWikiPingback` (total # refined records: 11417)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppiOSUserHistory` (total # refined records: 1387)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppSearch` (total # refined records: 3103)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EditConflict` (total # refined records: 122)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EditorJourney` (total # refined records: 344)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppSavedPages` (total # refined records: 204)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`GuidedTourGuiderImpression` (total # refined records: 495)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ServerTiming` (total # refined records: 50503)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MultimediaViewerNetworkPerformance` (total # refined records: 3155)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`ChangesListFilterGrouping` (total # refined records: 1029)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`LandingPageImpression` (total # refined records: 18)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppLanguageSettings` (total # refined records: 636)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`EditorActivation` (total # refined records: 175)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`TwoColConflictConflict` (total # refined records: 6)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`TemplateWizard` (total # refined records: 12027)
19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset partitions into table `event`.`MobileWikiAppDailyStats` (total # refined records: 37095)
Thu, May 9, 6:40 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, Analytics-Kanban, EventBus, Analytics
Ottomata added a comment to T196066: Add prometheus metrics for varnishkafka instances running on caching hosts.

BTW for reference here's what EvenGate is currently exporting:

Thu, May 9, 4:08 PM · Patch-For-Review, Analytics-Kanban, Traffic, Operations, Analytics
Ottomata added a comment to T196066: Add prometheus metrics for varnishkafka instances running on caching hosts.

I think there are a few more branches:

Thu, May 9, 3:34 PM · Patch-For-Review, Analytics-Kanban, Traffic, Operations, Analytics
Ottomata created T222899: Set up LVS for eventgate-main on port 32192.
Thu, May 9, 3:21 PM · Analytics-Kanban, Patch-For-Review, serviceops, Core Platform Team Backlog (Watching / External), Services (watching), EventBus, Analytics