Page MenuHomePhabricator
Feed Advanced Search

Fri, Dec 13

Ottomata added a comment to T238326: Make hdfs-rsync process sub-folders recursively.
[@stat1007:/home/otto] $ which hdfs-rsync
/usr/local/bin/hdfs-rsync
[@stat1007:/home/otto] $ hdfs-rsync --help
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
hdfs-rsync Provides an rsync equivalent to copy data between HDFS and local FS.
Usage: hdfs-rsync [options] src...[dst]
Fri, Dec 13, 10:07 PM · Analytics-Kanban, Analytics
Ottomata moved T236386: Set up eventgate-logging-external in production from In Progress to Done on the Analytics-Kanban board.
Fri, Dec 13, 8:53 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata moved T236386: Set up eventgate-logging-external in production from In Progress to Done on the Event-Platform board.
Fri, Dec 13, 8:52 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata set the point value for T236386: Set up eventgate-logging-external in production to 21.
Fri, Dec 13, 8:52 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata updated the task description for T236386: Set up eventgate-logging-external in production.
Fri, Dec 13, 8:50 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata committed rDEPLOYCHARTS928832f01d6b: Bump eventgate-logging-external eventgate image to 2019-12-13-200604-production (authored by Ottomata).
Bump eventgate-logging-external eventgate image to 2019-12-13-200604-production
Fri, Dec 13, 8:29 PM
Ottomata created T240719: Reset 2FA for Andrew Otto (Ottomata) on wikitech.
Fri, Dec 13, 8:17 PM · Cloud-Services, Trust-and-Safety
Ottomata removed a project from T240702: mediawiki.job.cirrusSearchElasticaWrite topics need more partitions!: Search-Platform-Programs.
Fri, Dec 13, 7:01 PM · Discovery-Search, Core Platform Team
Ottomata created T240702: mediawiki.job.cirrusSearchElasticaWrite topics need more partitions!.
Fri, Dec 13, 7:00 PM · Discovery-Search, Core Platform Team
Ottomata added a comment to T240549: Configure ORES to publish new drafttopic scores to Kafka.

@Gehel not sure I totally understand your requirements, but maybe what you need can be done by filtering and transforming the mediawiki.revision-score stream (and or Hive table?) into whatever you need?

Fri, Dec 13, 3:09 PM · Scoring-platform-team

Thu, Dec 12

Ottomata added a comment to T240461: Clients may need to generate a UUID for sending events..

The sole use case

It also helps in troubleshooting problems when they happen. EventGate uses the id field for logging error messages as well, so it helps to be able to correlate logging messages to the events that cause the problem.

Thu, Dec 12, 7:30 PM · Better Use Of Data
Ottomata added a comment to T240461: Clients may need to generate a UUID for sending events..

It is currently a requirement in the common schema's meta field. We could remove the requirement if we had to. But! We decided that we have to have EventGate augment events anyway, so we might as well set a meta.it uuid there if one is not provided by the client.

Thu, Dec 12, 3:41 PM · Better Use Of Data
Ottomata added a comment to T240460: Clients need to generate an ISO 8601 formatted timestamp.

(removed comment about sortability, that was wrong)

Thu, Dec 12, 3:40 PM · Better Use Of Data
Ottomata added a comment to T240460: Clients need to generate an ISO 8601 formatted timestamp.

It isn't just hive. The dt is used for the Kafka message timestamp (it should get propertly translated into a UTC unix ts before set though independent of what the ISO TZ is), as well as for hive hourly bucket partitioning. I believe that Camus will properly interpret the ISO TZ when bucketing, but this means that we would partitions that might say e.g. hour=0, but records in that partition might have meta.dt = 2019-12-12T02:00:00+0500. Not a huge deal, but could be confusing.

Thu, Dec 12, 3:34 PM · Better Use Of Data
Ottomata added a comment to T240560: rsyslogd: omkafka: action will suspended due to kafka error -187: Local: All broker connections are down.

This error message might be a false positive. I think librdkafka spits it out (is rsyslog using librdkafka?) when there is no data to send for a period of time.

Thu, Dec 12, 3:11 PM · Operations, serviceops, observability
Ottomata added a comment to T240460: Clients need to generate an ISO 8601 formatted timestamp.

If we can, everything in UTC is preferred. I think if we want to record the client's timezone, we should either use a different dt field, or just record the timezone itself as its own field.

Thu, Dec 12, 3:01 PM · Better Use Of Data
Ottomata added a comment to T240549: Configure ORES to publish new drafttopic scores to Kafka.

If you just put this score into the existing topic, it will show up in the event.mediawiki_revision_score hive table.

Thu, Dec 12, 2:24 PM · Scoring-platform-team
Ottomata added a comment to T240549: Configure ORES to publish new drafttopic scores to Kafka.

I think (hope!) this event will be fine!

Thu, Dec 12, 2:24 PM · Scoring-platform-team
Ottomata updated subscribers of T240549: Configure ORES to publish new drafttopic scores to Kafka.

I think if drafttopic is added to the list of 'precache' scores for changeprop, it will automatically get added. Ping @Pchelolo

Thu, Dec 12, 2:15 PM · Scoring-platform-team

Wed, Dec 11

Ottomata added a comment to T240477: eventgate-wikimedia should fill in defaults for some important fields.

Oh, we already made a an http.client_ip field for api/request and cirrusearch/request. I guess that's where it goes! Hm. I guess we still have to get it from X-Client-IP though, right?

Wed, Dec 11, 8:30 PM · Analytics-EventLogging, Event-Platform, Analytics
Ottomata updated subscribers of T240477: eventgate-wikimedia should fill in defaults for some important fields.

Hm, actually, we get X-Client-IP as a request header from varnish VCL right now. I guess that will continue to be the case in the ATS world? If so, we can also stick it in the http.request_headers map. @ema can we continue counting on the fact that all requests that go through a Varnish/ATS frontend will have the X-Client-IP request header set?

Wed, Dec 11, 8:11 PM · Analytics-EventLogging, Event-Platform, Analytics
Ottomata added a comment to T240477: eventgate-wikimedia should fill in defaults for some important fields.

Ok, to do this we need to standardize and bikeshed the client_ip and user_agent fields, since this is not yet in any official common schema.

Wed, Dec 11, 8:00 PM · Analytics-EventLogging, Event-Platform, Analytics
Ottomata created T240477: eventgate-wikimedia should fill in defaults for some important fields.
Wed, Dec 11, 4:33 PM · Analytics-EventLogging, Event-Platform, Analytics
Ottomata added a comment to T240461: Clients may need to generate a UUID for sending events..

If anything, I think we should remove the format qualifier from the JSONSchema for this field. I think this id should be usable for whatever id the client thinks makes the most sense for their event.

Wed, Dec 11, 3:22 PM · Better Use Of Data
Ottomata added a comment to T240181: Documentation improvements for Eventstreams.

The problem is that the SSE connection breaks every now and then

Wed, Dec 11, 2:35 PM · Analytics-Kanban, Event-Platform, User-Elukey, Analytics
Ottomata added a comment to T240439: Move https termination from nginx to envoy (if possible).

I will also need to do this for the schema service too! https://phabricator.wikimedia.org/T233630#5644875

Wed, Dec 11, 2:28 PM · Analytics

Tue, Dec 10

Ottomata moved T240355: Update MW Vagrant to work with EventLogging and EventGate changes from Next Up to In Progress on the Analytics-Kanban board.
Tue, Dec 10, 5:04 PM · Event-Platform, Analytics, Analytics-Kanban
Ottomata created T240355: Update MW Vagrant to work with EventLogging and EventGate changes.
Tue, Dec 10, 4:14 PM · Event-Platform, Analytics, Analytics-Kanban
Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

Changing the URL is easy

is not really that easy :/ Possible though.

Tue, Dec 10, 3:13 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T240255: an-coord1001 hive metastore not listening on ipv6.

I doubt we want to prefer IPv6, (do we?) but maybe we can make Hive listen on both IPs?

Tue, Dec 10, 2:40 PM · Analytics-Kanban, Analytics, Analytics-Cluster
Ottomata added a comment to T240181: Documentation improvements for Eventstreams.

Just to understand, does this need a deployment of ES or something else?

Tue, Dec 10, 2:19 PM · Analytics-Kanban, Event-Platform, User-Elukey, Analytics

Mon, Dec 9

Ottomata updated subscribers of T236386: Set up eventgate-logging-external in production.

Hey also, before I go through with this; is there any issue with CORS here? If we go with a separate (non wikimedia) domain, will client side JS have a problem submitting to this domain? I guess not if we use navigator.sendBeacon? Ping @Nuria, @Pchelolo

Mon, Dec 9, 8:15 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

@jlinehan thoughts? I'm considering moving forward with intake-{analytics,logging}.

Mon, Dec 9, 6:30 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T240182: Event Stream's stream to replace irc.wikimedia.org's #central channel feed.

Ya, before we create this steam (and expose it publicly), we should know why we need it and if we should keep it. Maybe we should, but if we don't need it...let's not keep it?

Mon, Dec 9, 6:21 PM · Event-Platform, User-Elukey, Analytics
Ottomata moved T238657: eventgate-wikimedia should support using remote stream configuration from Backlog to In Progress on the Event-Platform board.
Mon, Dec 9, 5:35 PM · MW-1.35-notes (1.35.0-wmf.3; 2019-10-22), Performance-Team (Radar), Better Use Of Data, CPT Initiatives (Modern Event Platform (TEC2)), Goal, Services (watching), Analytics-EventLogging, Event-Platform, Analytics
Ottomata added a comment to T234234: Redesign architecture of irc-recentchanges on top of Kafka.

Hm, a thought. Since each fake-IRCd server will consume directly from Kafka and then fan-out the messages to connected clients, they could each use a distinct Kafka consumer group. Then, when the server is restarted, it will be able to consume from where it stopped.

Mon, Dec 9, 5:31 PM · User-Elukey, Analytics
Ottomata added a comment to T240181: Documentation improvements for Eventstreams.

BTW, I don't think the IRC recentchanges stuff needs to consider historical consumption. The current IRC stuff doesn't support that now. I think we can always start consuming from latest offset (-1).

Mon, Dec 9, 4:10 PM · Analytics-Kanban, Event-Platform, User-Elukey, Analytics
Ottomata added a comment to T240182: Event Stream's stream to replace irc.wikimedia.org's #central channel feed.

Interesting! Likely, CentralAuth should fire a new hook whenever a new account is 'a local account record for the given wiki to the central database' is added. We'll also want to add a new JSONSchema for a mediawiki/centralauth/account-add (or whatever) event. Then the EventBus extension can then produce the event.

Mon, Dec 9, 3:52 PM · Event-Platform, User-Elukey, Analytics
Ottomata added a comment to T240181: Documentation improvements for Eventstreams.

Description of parameters in https://stream.wikimedia.org/?doc#/Streams/get_v2_stream_recentchange (and all the others) seems broken

I believe some of this will be fixed in some recent EventStreams work I did for the k8s migration.

Mon, Dec 9, 3:48 PM · Analytics-Kanban, Event-Platform, User-Elukey, Analytics
Ottomata added a comment to T239996: Explore sending batches of events from EPC libraries.

So the critical perf improvements (when it comes to the battery) are gained from walking up the radio less frequently.

Mon, Dec 9, 3:10 PM · Analytics, Event-Platform, Better Use Of Data, Product-Infrastructure-Team-Backlog
Ottomata added a comment to T239996: Explore sending batches of events from EPC libraries.

an optimal batch size

The current (adjustable) limits are independent of the batch array length, but are instead about byte length of individual
messages, and also the total POST body byte length.

Mon, Dec 9, 3:09 PM · Analytics, Event-Platform, Better Use Of Data, Product-Infrastructure-Team-Backlog

Thu, Dec 5

Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

Is /v1/events plural with the intention that eventually EventGate will support batches of events in the same request?

Thu, Dec 5, 8:30 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T236386: Set up eventgate-logging-external in production.
  • logging-sink & analytics-sink (or sink-*) ?
Thu, Dec 5, 7:25 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

Where <instance> is either 'logging' or 'analytics'.

Thu, Dec 5, 7:22 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T226986: Client side error logging production launch.

Ah, I won't be able to make tomorrow's meeting, I'm not working and I have a grocery coop shift!

Thu, Dec 5, 7:18 PM · Patch-For-Review, observability, Wikimedia-Logstash, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic, Analytics
Ottomata updated the task description for T226986: Client side error logging production launch.
Thu, Dec 5, 6:34 PM · Patch-For-Review, observability, Wikimedia-Logstash, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic, Analytics
Ottomata added a comment to T239121: VE edit data stopped due to statsv falling over (?) on webperf1001.

Done. BTW same errors were present in recent logs.

Thu, Dec 5, 6:28 PM · Performance-Team (Radar), observability, Analytics, Editing-team

Wed, Dec 4

Ottomata closed T225129: Move eventgate logs to new logging infrastructure, a subtask of T225125: Migrate Elasticsearch from deprecated Gelf logstash input to rsyslog Kafka logging pipeline, as Resolved.
Wed, Dec 4, 2:49 PM · Patch-For-Review, Discovery-Search (Current work), observability, Elasticsearch, Operations, Wikimedia-Logstash
Ottomata closed T225129: Move eventgate logs to new logging infrastructure as Resolved.
Wed, Dec 4, 2:49 PM · observability, Analytics, Event-Platform, Operations, Wikimedia-Logstash
Ottomata moved T225129: Move eventgate logs to new logging infrastructure from In Progress to Done on the Event-Platform board.
Wed, Dec 4, 2:49 PM · observability, Analytics, Event-Platform, Operations, Wikimedia-Logstash

Tue, Dec 3

Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

How about:

Tue, Dec 3, 4:42 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

Sigh, it turns out SRE wants us to not rewrite paths. So if we use path based routing, the app needs to handle whatever comes in from the public request, i.e. EventGate would need to understand /produce/logging/v1/events. I'd prefer not to make EventGate change its API based on the installation of it, so we are back to using domains to route public -> app backend. Back to the domain name bikeshed!

Tue, Dec 3, 4:34 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata committed rDEPLOYCHARTS4e8410cc8a84: eventgate-analytics - stream config for new sparql-query streams (authored by Ottomata).
eventgate-analytics - stream config for new sparql-query streams
Tue, Dec 3, 3:31 PM
Ottomata updated the task description for T236386: Set up eventgate-logging-external in production.
Tue, Dec 3, 2:24 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T236180: Deploy search platform airflow service.

Let's do it on Analytics Meta. I think it will be fine.

Tue, Dec 3, 1:55 PM · Patch-For-Review, Discovery-Search
Ottomata added a comment to T225538: Request for a large request data set for caching research and tuning.

Ok! Find me on IRC let's do it!

Tue, Dec 3, 1:54 PM · Analytics-Kanban, Analytics

Mon, Dec 2

Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

@akosiaris I merged and applied https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/551610 in staging. My main app isn't coming up, and I suspect it is because it can't reach Kafka at the 9093 TLS port on the logstash hosts. I noticed we don't have the logstash IPv6 entries in https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/551610/8/helmfile.d/admin/staging/calico/default-kubernetes-policy.yaml, but do we need/want them?

Mon, Dec 2, 8:43 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata committed rDEPLOYCHARTSc5bf7cfc8b73: Use Kafka TLS port for eventgate-logging-external (authored by Ottomata).
Use Kafka TLS port for eventgate-logging-external
Mon, Dec 2, 8:35 PM
Ottomata committed rDEPLOYCHARTSf182b483528d: Enable Kafka Producer TLS for eventgate-logging-external (authored by Ottomata).
Enable Kafka Producer TLS for eventgate-logging-external
Mon, Dec 2, 8:24 PM
Ottomata committed rDEPLOYCHARTS9b7654a1e379: Kafka producer TLS support for eventgate charts (authored by Ottomata).
Kafka producer TLS support for eventgate charts
Mon, Dec 2, 8:16 PM
Ottomata added a comment to T239121: VE edit data stopped due to statsv falling over (?) on webperf1001.

Also possibly relevant:

Mon, Dec 2, 8:06 PM · Performance-Team (Radar), observability, Analytics, Editing-team
Ottomata added a comment to T239121: VE edit data stopped due to statsv falling over (?) on webperf1001.

In statsv service logs on webperf1001:

Mon, Dec 2, 8:06 PM · Performance-Team (Radar), observability, Analytics, Editing-team
Ottomata added a comment to T217059: Spike [2019-2020 work] Oozie Replacement. Airflow Study / Argo Study.

https://medium.com/flyr-labs-blog/why-were-switching-off-airflow-sort-of-780c4f58a660

Mon, Dec 2, 2:56 PM · User-ArielGlenn, Analytics

Fri, Nov 29

Ottomata updated the task description for T236386: Set up eventgate-logging-external in production.
Fri, Nov 29, 3:39 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic

Wed, Nov 27

Ottomata updated the task description for T236386: Set up eventgate-logging-external in production.
Wed, Nov 27, 4:29 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata added a comment to T239220: page-links-change EventStream doesn't appear to be outputting events.

FYI, we will be moving EventStreams to Kubernetes next quarter, which should help it scale a little better.

Wed, Nov 27, 2:44 PM · User-Elukey, Wikimedia-Stream, Internet-Archive, MediaWiki-extensions-WikimediaEvents, The-Wikipedia-Library

Mon, Nov 25

Ottomata added a comment to T234234: Redesign architecture of irc-recentchanges on top of Kafka.
  • Do we need to support full IRC spec? I see in Faidon's PoC he's attempting to at least respond to them all. AFACIT we don't actually support any commands other than joining channels. I guess we want to keep track of user nicks too? Do we need to? We could just keep the list of connected clients in channels and broadcast the incoming messages. Could we just ignore things we don't support? E.g. if not JOIN or whatever else we need: respond with some 'command unsupported' message.
Mon, Nov 25, 5:42 PM · User-Elukey, Analytics
Ottomata added a project to T238937: Eliminate usage of mocha-eslint from eventstreams: Analytics-Kanban.
Mon, Nov 25, 4:29 PM · Analytics-Kanban, Analytics, Event-Platform, Wikimedia-Stream
Ottomata added a comment to T234234: Redesign architecture of irc-recentchanges on top of Kafka.

new software component will be created, to bridge the current bot interface (IRC) with a more recent and simple one (EventStreams).

Mon, Nov 25, 4:28 PM · User-Elukey, Analytics

Fri, Nov 22

Ottomata added a comment to T238833: Create NRPE check to alert when cergen certificates are due to expire.

I will also update that doc for --force who wrote that!? ò_ô

Fri, Nov 22, 5:31 PM · Patch-For-Review, User-jbond, Puppet, Operations
Ottomata added a comment to T238833: Create NRPE check to alert when cergen certificates are due to expire.

I'll try to find some time soon to make cergen chmod after creating files.

Fri, Nov 22, 5:28 PM · Patch-For-Review, User-jbond, Puppet, Operations
Ottomata added a comment to T238833: Create NRPE check to alert when cergen certificates are due to expire.

Hm, would running cergen --generate with --force be enough?

Fri, Nov 22, 5:27 PM · Patch-For-Review, User-jbond, Puppet, Operations
Ottomata added a comment to T238833: Create NRPE check to alert when cergen certificates are due to expire.

Could we make the cergen script itself modify the permissions after it creates the files? It won't ensure things are right ongoing, but at least they'd be right from the start.

Fri, Nov 22, 3:37 PM · Patch-For-Review, User-jbond, Puppet, Operations
Ottomata added a comment to T238905: Allow analytics-search-users members to sudo as the airflow user.

Since this instance is maintained by the search team, I think re-using analytics-search-users makes sense to me. We can re-evaluate that when if/we run an official airflow.

Fri, Nov 22, 3:36 PM · Operations, SRE-Access-Requests, Discovery-Search

Thu, Nov 21

Ottomata added a comment to T236386: Set up eventgate-logging-external in production.

Ok thanks for the help today @akosiaris and @Joe, HTTPS via envoyproxy is finally working! I will be off tomorrow but working on Monday. Could yall merge and deploy the LVS and discovery changes by the time start working Monday morning?

Thu, Nov 21, 6:49 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata updated the task description for T236386: Set up eventgate-logging-external in production.
Thu, Nov 21, 6:45 PM · Patch-For-Review, Services, Service-deployment-requests, Operations, serviceops, Event-Platform, Analytics-Kanban, User-Elukey, User-fgiunchedi, Better Use Of Data, Product-Infrastructure-Team-Backlog, Epic
Ottomata committed rDEPLOYCHARTSd8bf97aae937: eventgate-0.0.15 - Fix services app selector for envoyproxy tls port (authored by Ottomata).
eventgate-0.0.15 - Fix services app selector for envoyproxy tls port
Thu, Nov 21, 6:42 PM
Ottomata committed rDEPLOYCHARTSd97ffdf78bf5: eventgate-0.0.14 (authored by Ottomata).
eventgate-0.0.14
Thu, Nov 21, 6:02 PM
Ottomata added a comment to T238833: Create NRPE check to alert when cergen certificates are due to expire.

Why does this script need sudo? Doesn't it just need to read the expiry out of the public certs?

Thu, Nov 21, 5:20 PM · Patch-For-Review, User-jbond, Puppet, Operations
Ottomata renamed T238833: Create NRPE check to alert when cergen certificates are due to expire from Create NRPE check to alert when certgen certificates are due to expire to Create NRPE check to alert when cergen certificates are due to expire.
Thu, Nov 21, 5:16 PM · Patch-For-Review, User-jbond, Puppet, Operations
Ottomata committed rDEPLOYCHARTS3ee57aa61908: Adding missed eventgate-0.0.13.tgz (authored by Ottomata).
Adding missed eventgate-0.0.13.tgz
Thu, Nov 21, 2:47 PM
Ottomata committed rDEPLOYCHARTS300ff9b381dd: Enable TLS envoyproxy for eventgate-logging-external instances (authored by Ottomata).
Enable TLS envoyproxy for eventgate-logging-external instances
Thu, Nov 21, 2:46 PM
Ottomata committed rDEPLOYCHARTS6fce0380560b: eventgate 0.0.13 - envoyproxy tls support (authored by Ottomata).
eventgate 0.0.13 - envoyproxy tls support
Thu, Nov 21, 2:44 PM
Ottomata committed rDEPLOYCHARTS5636df9e244f: TLS envoyproxy support for eventgate chart (authored by Ottomata).
TLS envoyproxy support for eventgate chart
Thu, Nov 21, 2:27 PM
Ottomata added a comment to T206789: Modern Event Platform: Schema Registry: Implementation.

I'm good with primary / secondary

Thu, Nov 21, 2:15 PM · Analytics-Kanban, CPT Initiatives (Modern Event Platform (TEC2)), Services (watching), Analytics-EventLogging, Event-Platform, Analytics

Wed, Nov 20

Ottomata added a comment to T206789: Modern Event Platform: Schema Registry: Implementation.

@Nuria I thought you didn't like 'experiments'? Doesn't that imply that the schemas there are intended to be for short term usage?

Wed, Nov 20, 9:55 PM · Analytics-Kanban, CPT Initiatives (Modern Event Platform (TEC2)), Services (watching), Analytics-EventLogging, Event-Platform, Analytics
Ottomata added a comment to T206789: Modern Event Platform: Schema Registry: Implementation.

@Milimetric how do you feel about 'exploratory' vs 'experimental'? While I like the idea that the non-restricted repo should be used for 'temporary' things, Nuria said she thinks that is a bad requirement. Schemas will more realistically live there forever. Exploratory seems ok to me, as it doesn't imply temporariness.

Wed, Nov 20, 8:52 PM · Analytics-Kanban, CPT Initiatives (Modern Event Platform (TEC2)), Services (watching), Analytics-EventLogging, Event-Platform, Analytics
Ottomata closed T177927: Refactor kafka_config.rb and and kafka_cluster_name.rb in puppet to avoid explicit hiera calls as Declined.

Old task, I think we aren't likely to do this. Declining, feel free to reopen if wrong.

Wed, Nov 20, 8:16 PM · Analytics, Traffic, Operations, User-Elukey
Ottomata closed T210759: Refine Monitor should be a systemd timer such if process cannot start we get notified as Resolved.

Hm? They are timers no? Perhaps this just changed in the last year.

Wed, Nov 20, 7:54 PM · Analytics
Ottomata closed T213976: Workflow to be able to move data files computed in jobs from analytics cluster to production , a subtask of T213566: Transferring data from Hadoop to production MySQL database, as Resolved.
Wed, Nov 20, 7:51 PM · serviceops-radar, Core Platform Team Legacy (Watching / External), Services (watching), User-Marostegui, Operations, Article-Recommendation, Analytics
Ottomata closed T213976: Workflow to be able to move data files computed in jobs from analytics cluster to production as Resolved.

This was finished back in July. Resolving.

Wed, Nov 20, 7:51 PM · Article-Recommendation, Patch-For-Review, Operations, Discovery, Analytics
Ottomata closed T213566: Transferring data from Hadoop to production MySQL database, a subtask of T214074: Productionize article recommender systems, as Resolved.
Wed, Nov 20, 7:50 PM · Article-Recommendation, Epic
Ottomata closed T213566: Transferring data from Hadoop to production MySQL database as Resolved.

This is now supported via Kafka, Swift and an Oozie workflow.
T213976: Workflow to be able to move data files computed in jobs from analytics cluster to production

Wed, Nov 20, 7:50 PM · serviceops-radar, Core Platform Team Legacy (Watching / External), Services (watching), User-Marostegui, Operations, Article-Recommendation, Analytics
Ottomata closed T224291: db1107 and db1108 buffer pool size configuration as Declined.

Since we'll be doing T159170: Sunset MySQL data store for eventlogging soon, declining this.

Wed, Nov 20, 7:47 PM · Analytics-EventLogging, Analytics
Ottomata closed T219922: Move eventstreams logging to new logging pipeline, a subtask of T211125: Move service-runner to new logging infrastructure, as Declined.
Wed, Nov 20, 7:45 PM · observability, Core Platform Team Legacy (Watching / External), Patch-For-Review, service-runner, Wikimedia-Logstash, Operations
Ottomata closed T219922: Move eventstreams logging to new logging pipeline as Declined.

We'll be moving EventStreams to k8s next quarter, which will take advantage of new logging pipeline automatically. Declining this.

Wed, Nov 20, 7:45 PM · observability, Analytics, Core Platform Team Legacy (Watching / External), Services (watching), service-runner, Wikimedia-Logstash, Operations
Ottomata merged T227122: Move Eventstreams to kubernetes deployment pipeline into T238658: Migrate EventStreams to k8s deployment pipeline.
Wed, Nov 20, 7:43 PM · Analytics, Patch-For-Review, Release-Engineering-Team (Pipeline), Services (watching), Release Pipeline
Ottomata merged task T227122: Move Eventstreams to kubernetes deployment pipeline into T238658: Migrate EventStreams to k8s deployment pipeline.
Wed, Nov 20, 7:43 PM · Analytics
Ottomata moved T233634: Modern Event Platform: Stream Configuration: Implementation from Next Up to In Progress on the Analytics-Kanban board.
Wed, Nov 20, 7:39 PM · Core Platform Team Workboards (Clinic Duty Team), Analytics-Kanban, Product-Infrastructure-Team-Backlog (Kanban), MW-1.35-notes (1.35.0-wmf.3; 2019-10-22), Patch-For-Review, Performance-Team (Radar), Better Use Of Data, CPT Initiatives (Modern Event Platform (TEC2)), Goal, Services (watching), Analytics-EventLogging, Event-Platform, Analytics
Ottomata added a project to T233634: Modern Event Platform: Stream Configuration: Implementation: Analytics-Kanban.
Wed, Nov 20, 7:39 PM · Core Platform Team Workboards (Clinic Duty Team), Analytics-Kanban, Product-Infrastructure-Team-Backlog (Kanban), MW-1.35-notes (1.35.0-wmf.3; 2019-10-22), Patch-For-Review, Performance-Team (Radar), Better Use Of Data, CPT Initiatives (Modern Event Platform (TEC2)), Goal, Services (watching), Analytics-EventLogging, Event-Platform, Analytics