Page MenuHomePhabricator

mobrovac (Marko Obrovac)
Spy

Projects (46)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Dec 16 2014, 7:40 PM (226 w, 6 d)
Availability
Available
IRC Nick
mobrovac
LDAP User
Mobrovac
MediaWiki User
Mobrovac [ Global Accounts ]

Recent Activity

Sun, Apr 21

mobrovac edited P8423 Simplified Parsoid tables - beta.
Sun, Apr 21, 8:58 PM · Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac edited P8422 Simplified Parsoid tables - production.
Sun, Apr 21, 8:57 PM · Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac added projects to T221530: Drop old mobile-sections keyspaces: RESTBase-Cassandra, Cassandra.

Here are the statements:

Sun, Apr 21, 8:49 PM · Cassandra, RESTBase-Cassandra, Core Platform Team Backlog (Next), Services (next), Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac created P8424 Drop old mobile-sections tables.
Sun, Apr 21, 8:49 PM · Core Platform Team (RESTBase Split (CDP2)), RESTBase-Cassandra
mobrovac added a parent task for T221530: Drop old mobile-sections keyspaces: T215960: Simplify MCS storage model.
Sun, Apr 21, 8:36 PM · Cassandra, RESTBase-Cassandra, Core Platform Team Backlog (Next), Services (next), Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac added a subtask for T215960: Simplify MCS storage model: T221530: Drop old mobile-sections keyspaces.
Sun, Apr 21, 8:36 PM · Core Platform Team Kanban (Doing), Services (doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, Reading-Infrastructure-Team-Backlog, RESTBase
mobrovac triaged T221530: Drop old mobile-sections keyspaces as Normal priority.
Sun, Apr 21, 8:35 PM · Cassandra, RESTBase-Cassandra, Core Platform Team Backlog (Next), Services (next), Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac added a comment to T221528: Create new Parsoid tables.

Here are the creation statements for dev and production:

Sun, Apr 21, 8:21 PM · RESTBase-Cassandra, Cassandra, Core Platform Team (RESTBase Split (CDP2)), RESTBase, Core Platform Team Kanban (Doing), Services (doing)
mobrovac created P8423 Simplified Parsoid tables - beta.
Sun, Apr 21, 8:19 PM · Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac created P8422 Simplified Parsoid tables - production.
Sun, Apr 21, 8:17 PM · Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac updated the task description for T221528: Create new Parsoid tables.
Sun, Apr 21, 7:28 PM · RESTBase-Cassandra, Cassandra, Core Platform Team (RESTBase Split (CDP2)), RESTBase, Core Platform Team Kanban (Doing), Services (doing)
mobrovac added a parent task for T221528: Create new Parsoid tables: T215956: Consider stashing data-parsoid for VE .
Sun, Apr 21, 7:20 PM · RESTBase-Cassandra, Cassandra, Core Platform Team (RESTBase Split (CDP2)), RESTBase, Core Platform Team Kanban (Doing), Services (doing)
mobrovac added a subtask for T215956: Consider stashing data-parsoid for VE : T221528: Create new Parsoid tables.
Sun, Apr 21, 7:20 PM · Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, User-mobrovac, Parsoid, VisualEditor, RESTBase
mobrovac triaged T221528: Create new Parsoid tables as High priority.
Sun, Apr 21, 7:20 PM · RESTBase-Cassandra, Cassandra, Core Platform Team (RESTBase Split (CDP2)), RESTBase, Core Platform Team Kanban (Doing), Services (doing)
mobrovac removed a project from T215956: Consider stashing data-parsoid for VE : Operations.
Sun, Apr 21, 7:05 PM · Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, User-mobrovac, Parsoid, VisualEditor, RESTBase

Sat, Apr 20

mobrovac edited projects for T215956: Consider stashing data-parsoid for VE , added: Core Platform Team Kanban (Doing), Services (doing); removed Core Platform Team Backlog (Designing), Services (designing).

Since in either case we don't need to have special Varnish rules, I think we need both the query parameter and UA matching in order to ease the transition. The plan would be:

Sat, Apr 20, 8:46 PM · Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, User-mobrovac, Parsoid, VisualEditor, RESTBase
mobrovac added a comment to T215960: Simplify MCS storage model.

The simplification code has been deployed without issues, and we have populated the new table with dumps for en, de, fr, it, es, pt, pl, ja and ru wikipedias. The last step here is to delete the old, unused keyspaces.

Sat, Apr 20, 8:04 PM · Core Platform Team Kanban (Doing), Services (doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, Reading-Infrastructure-Team-Backlog, RESTBase
mobrovac added a comment to T221368: cdnPurge and other jobs fail completely to execute.

Incident report available here.

Sat, Apr 20, 5:17 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), WMF-JobQueue, Performance-Team, MediaWiki-JobQueue, Wikimedia-production-error

Fri, Apr 19

mobrovac edited projects for T221432: New swagger-ui try it out feature returns 406 for some endpoints, added: Core Platform Team Kanban (Waiting for Review); removed Core Platform Team Kanban (Doing).

Heh, no, it turns out we have forgotten to update the response profile version in the spec. PR #1122 is the fix.

Fri, Apr 19, 11:44 PM · Core Platform Team Kanban (Waiting for Review), Core Platform Team (RESTBase Split (CDP2)), RESTBase-API, RESTBase, Services (doing)
mobrovac claimed T221432: New swagger-ui try it out feature returns 406 for some endpoints.

This is likely a bug in Swagger-UI; there is probably a problem with the Accept header having quotes which are probably not escaped. I'll take a look.

Fri, Apr 19, 11:11 PM · Core Platform Team Kanban (Waiting for Review), Core Platform Team (RESTBase Split (CDP2)), RESTBase-API, RESTBase, Services (doing)
mobrovac added a subtask for T218218: Make RESTBase spec standard compliant and switch to OpenAPI 3.0: T221432: New swagger-ui try it out feature returns 406 for some endpoints.
Fri, Apr 19, 11:08 PM · Patch-For-Review, Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2)), RESTBase, RESTBase-API, serviceops, Operations
mobrovac added a parent task for T221432: New swagger-ui try it out feature returns 406 for some endpoints: T218218: Make RESTBase spec standard compliant and switch to OpenAPI 3.0.
Fri, Apr 19, 11:08 PM · Core Platform Team Kanban (Waiting for Review), Core Platform Team (RESTBase Split (CDP2)), RESTBase-API, RESTBase, Services (doing)
mobrovac added a comment to T218692: read only on mediawiki generates "LoadBalancer.php: Cannot access the database: Unknown error".
  1. When the MediaWiki database is in read-only mode, jobs must not be run. Otherwise, the jobs' writes would fail with a fatal error, then the job runs out of retries (if any are allowed) and the scheduled work and parameters lost indefinitely (e.g. sending a newsletter, deleting a page, clearing a watchlist, etc.)
Fri, Apr 19, 10:46 PM · Core Platform Team Backlog (Watching / External), WMF-JobQueue, Wikimedia-production-error, MediaWiki-Database
mobrovac edited projects for T215339: No jobs running on beta cluster, added: Services (done), serviceops; removed Patch-For-Review, Services (next).
Fri, Apr 19, 10:03 PM · serviceops, Services (done), Wikidata, SDC General, Beta-Cluster-Infrastructure
mobrovac added a comment to T215339: No jobs running on beta cluster.

@Joe I just tried to send a mass message on the beta cluster, and the message was "queued" but isn't actually being delivered. Is that part of this task?

Fri, Apr 19, 10:03 PM · serviceops, Services (done), Wikidata, SDC General, Beta-Cluster-Infrastructure
mobrovac added a comment to T221295: Wikidata changes doesn't appear in recent changes.

seems to happen again - no updates from Wikidata in the last 30 minutes in enwiki

Fri, Apr 19, 8:14 PM · Wikidata-Campsite, Regression, Wikidata, Growth-Team, MediaWiki-Recent-changes, MediaWiki-extensions-WikibaseClient

Thu, Apr 18

mobrovac closed T221390: Make sure any monolog + eventbus fields with binary data are base64 encoded as Resolved.

Fix deployed, all looking good.

Thu, Apr 18, 11:17 PM · MW-1.34-notes (1.34.0-wmf.1; 2019-04-16), Analytics-Kanban, Services (watching), Discovery, EventBus, Analytics
mobrovac closed T221390: Make sure any monolog + eventbus fields with binary data are base64 encoded, a subtask of T214080: Rewrite Avro schemas (ApiAction, CirrusSearchRequestSet) as JSONSchema and produce to EventGate, as Resolved.
Thu, Apr 18, 11:17 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), MW-1.33-notes (1.33.0-wmf.21; 2019-03-12), Analytics-Kanban, Patch-For-Review, Services (watching), Discovery, Analytics-EventLogging, EventBus, Analytics
mobrovac reassigned T221368: cdnPurge and other jobs fail completely to execute from aaron to Krinkle.
Thu, Apr 18, 8:56 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), WMF-JobQueue, Performance-Team, MediaWiki-JobQueue, Wikimedia-production-error
mobrovac edited projects for T221368: cdnPurge and other jobs fail completely to execute, added: Services (done), Core Platform Team Kanban (Done with CPT); removed Core Platform Team Kanban (Doing), Services (doing), Patch-For-Review.
Thu, Apr 18, 8:56 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), WMF-JobQueue, Performance-Team, MediaWiki-JobQueue, Wikimedia-production-error
mobrovac raised the priority of T221368: cdnPurge and other jobs fail completely to execute from High to Unbreak Now!.

Still UBN, waiting on the mw-core side for this.

Thu, Apr 18, 6:22 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), WMF-JobQueue, Performance-Team, MediaWiki-JobQueue, Wikimedia-production-error
mobrovac lowered the priority of T221368: cdnPurge and other jobs fail completely to execute from Unbreak Now! to High.

The immediate problem with jobs failing because of invalid titles has been addressed, so lowering the priority. Next step (which have to wait for the train to fully complete) include removing the title and namespace parameters from the job schema and not using them when enqueuing/dequeuing the jobs.

Thu, Apr 18, 6:13 PM · MW-1.34-notes (1.34.0-wmf.3; 2019-04-30), Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), WMF-JobQueue, Performance-Team, MediaWiki-JobQueue, Wikimedia-production-error
mobrovac added a comment to T216308: RFC Process: 2019 amendments.

Maybe we should just have an "attic" column.

Thu, Apr 18, 5:06 PM · TechCom-RFC, TechCom
mobrovac added a project to T221158: Parsoid REST API in PHP: Services (watching).
Thu, Apr 18, 4:20 PM · Core Platform Team (Parsoid REST API in PHP (CDP2)), Parsoid-PHP, Services (watching), Goal, MediaWiki-Parser
mobrovac added projects to T221158: Parsoid REST API in PHP: Parsoid, MediaWiki-Parser, Goal, Parsing-Team, Core Platform Team.
Thu, Apr 18, 4:19 PM · Core Platform Team (Parsoid REST API in PHP (CDP2)), Parsoid-PHP, Services (watching), Goal, MediaWiki-Parser
mobrovac edited projects for T221315: Determine future of bare-metal hosting for services like WDQS, ElasticSearch, RESTBase Cassandra, etc., added: RESTBase, Services (designing), Core Platform Team Backlog (Designing); removed RESTBase-Cassandra.

What we originally discussed was basically to leave databases outside of the realm of k8s. So MariaDB, Postgres, Cassandra and friends would still stay on bare metal. We have also touched upon the question of whether the services that directly access these storage engines would stay there or not, and we opted for keeping them as-are for now. In the first phase of the move to k8s we would move only stateless services, but I can see us moving those directly using storage too in a while. Whether the storage engines themselves would ever be moved if up for discussion, but IMHO that's not what containers are for.

Thu, Apr 18, 12:08 AM · Core Platform Team Backlog (Designing), Services (designing), RESTBase, Discovery-Search, Wikidata, serviceops, Release Pipeline, Elasticsearch, Wikidata-Query-Service

Wed, Apr 17

mobrovac added a comment to T34478: AbuseFilter not setting utf-8 flag.

Since the plan is to fix the DB storage format and add new back-compat code, we can also start adding the utf-8 flag as part of T213006.

Wed, Apr 17, 10:06 PM · Core Platform Team Kanban (Doing), User-Daimona, AbuseFilter, MediaWiki-Database
mobrovac edited projects for T34478: AbuseFilter not setting utf-8 flag, added: Core Platform Team Kanban (Doing); removed Core Platform Team Backlog.
Wed, Apr 17, 7:34 PM · Core Platform Team Kanban (Doing), User-Daimona, AbuseFilter, MediaWiki-Database
mobrovac updated subscribers of T34478: AbuseFilter not setting utf-8 flag.

So now we have a great opportunity to fix this, which is T213006. The task outlines the tasks of a new script used to clean afl_var_dump and related, and since this is strongly related, we should handle it as well.
My only question is how to fix it. Should we add the 'utf-8' flag to every entry, or what else? Just let me know and I can amend the patch.

Wed, Apr 17, 7:34 PM · Core Platform Team Kanban (Doing), User-Daimona, AbuseFilter, MediaWiki-Database
mobrovac triaged T182259: Create scripts to estimate Kafka queue size per wiki as Normal priority.

@Pchelolo let's add it to ops/puppet?

Wed, Apr 17, 6:45 PM · Core Platform Team Kanban (Doing), Services (doing), MediaWiki-JobQueue, EventBus, Analytics
mobrovac triaged T175874: RESTBase's and Parsoid's configs in MW-Vagrant don't contain additional domains as Low priority.
Wed, Apr 17, 6:44 PM · RESTBase, Core Platform Team Backlog (Later), Services (later), Patch-For-Review, Parsoid, MediaWiki-Vagrant
mobrovac assigned T175874: RESTBase's and Parsoid's configs in MW-Vagrant don't contain additional domains to Pchelolo.

@Pchelolo assigning to you since you have a patch up, but perhaps this could be a good one for @holger.knust

Wed, Apr 17, 6:44 PM · RESTBase, Core Platform Team Backlog (Later), Services (later), Patch-For-Review, Parsoid, MediaWiki-Vagrant
mobrovac edited projects for T177650: Cassandra compaction throughput rate limiting, added: Services (attic), Core Platform Team Backlog (Attic); removed Core Platform Team Kanban (Doing), Services (doing).
Wed, Apr 17, 6:40 PM · Core Platform Team Backlog (Attic), Services (attic), User-Eevans, Cassandra
mobrovac lowered the priority of T127472: Investigate reducing impact of single-node Cassandra latencies from High to Low.
Wed, Apr 17, 6:39 PM · Core Platform Team Backlog (Later), Services (later), User-Eevans, Cassandra, RESTBase-Cassandra

Tue, Apr 16

mobrovac removed projects from T172497: Fix mediawiki heartbeat model, change pt-heartbeat model to not use super-user, avoid SPOF and switch automatically to the real master without puppet dependency: User-mobrovac, Core Platform Team (Multi-DC (TEC1)).

@mobrovac I think this task is confusing, we should use separate tasks for T172497#4905268 instead, and separate the non-cross dc work away.

Tue, Apr 16, 11:52 PM · Services (watching), Core Platform Team Backlog (Watching / External), Performance-Team (Radar), MediaWiki-Database, Wikimedia-Incident, DBA
mobrovac removed a parent task for T172497: Fix mediawiki heartbeat model, change pt-heartbeat model to not use super-user, avoid SPOF and switch automatically to the real master without puppet dependency: T88445: MediaWiki active/active datacenter investigation and work (tracking).
Tue, Apr 16, 11:51 PM · Services (watching), Core Platform Team Backlog (Watching / External), Performance-Team (Radar), MediaWiki-Database, Wikimedia-Incident, DBA
mobrovac removed a subtask for T88445: MediaWiki active/active datacenter investigation and work (tracking): T172497: Fix mediawiki heartbeat model, change pt-heartbeat model to not use super-user, avoid SPOF and switch automatically to the real master without puppet dependency.
Tue, Apr 16, 11:51 PM · Core Platform Team (Multi-DC (TEC1)), User-mobrovac, Services (designing), Core Platform Team Backlog (Epic), Performance-Team (Radar), Availability (MediaWiki-MultiDC), Epic
mobrovac added projects to T221159: FY18/19 TEC1.6 Q4: Improve or replace GTID + pt-heartbeat logic for cross-DC: Services (watching), User-mobrovac.
Tue, Apr 16, 11:51 PM · User-mobrovac, Services (watching), Goal, Core Platform Team Backlog (Watching / External), MediaWiki-Database, Core Platform Team (Multi-DC (TEC1)), Performance-Team, DBA
mobrovac added a subtask for T88445: MediaWiki active/active datacenter investigation and work (tracking): T221159: FY18/19 TEC1.6 Q4: Improve or replace GTID + pt-heartbeat logic for cross-DC.
Tue, Apr 16, 11:50 PM · Core Platform Team (Multi-DC (TEC1)), User-mobrovac, Services (designing), Core Platform Team Backlog (Epic), Performance-Team (Radar), Availability (MediaWiki-MultiDC), Epic
mobrovac added a parent task for T221159: FY18/19 TEC1.6 Q4: Improve or replace GTID + pt-heartbeat logic for cross-DC: T88445: MediaWiki active/active datacenter investigation and work (tracking).
Tue, Apr 16, 11:50 PM · User-mobrovac, Services (watching), Goal, Core Platform Team Backlog (Watching / External), MediaWiki-Database, Core Platform Team (Multi-DC (TEC1)), Performance-Team, DBA
mobrovac added a comment to T221159: FY18/19 TEC1.6 Q4: Improve or replace GTID + pt-heartbeat logic for cross-DC.

@aaron @jcrespo @Marostegui the task desc is a draft, please feel free to update it appropriately.

Tue, Apr 16, 11:50 PM · User-mobrovac, Services (watching), Goal, Core Platform Team Backlog (Watching / External), MediaWiki-Database, Core Platform Team (Multi-DC (TEC1)), Performance-Team, DBA
mobrovac triaged T221159: FY18/19 TEC1.6 Q4: Improve or replace GTID + pt-heartbeat logic for cross-DC as High priority.
Tue, Apr 16, 11:49 PM · User-mobrovac, Services (watching), Goal, Core Platform Team Backlog (Watching / External), MediaWiki-Database, Core Platform Team (Multi-DC (TEC1)), Performance-Team, DBA
mobrovac edited projects for T220246: Session storage service Cassandra schema, added: Core Platform Team (Session Management Service (CDP2)); removed Core Platform Team (Multi-DC (TEC1)).
Tue, Apr 16, 11:16 PM · Core Platform Team (Session Management Service (CDP2)), User-Clarakosi, Core Platform Team Backlog (Next), User-Eevans
mobrovac edited projects for T221031: Create new mobile storage tables, added: Services (done), Core Platform Team Kanban (Done with CPT); removed Core Platform Team Kanban (Doing), Services (doing).
Tue, Apr 16, 8:15 PM · Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, Reading-Infrastructure-Team-Backlog, RESTBase

Mon, Apr 15

mobrovac edited projects for T215960: Simplify MCS storage model, added: Services (doing), Core Platform Team Kanban (Doing); removed Core Platform Team Backlog (Designing), Services (designing).
Mon, Apr 15, 11:32 PM · Core Platform Team Kanban (Doing), Services (doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, Reading-Infrastructure-Team-Backlog, RESTBase
mobrovac added a comment to T221031: Create new mobile storage tables.

+1, sounds like a plan!

Mon, Apr 15, 8:46 PM · Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, Reading-Infrastructure-Team-Backlog, RESTBase
mobrovac triaged T221031: Create new mobile storage tables as High priority.
Mon, Apr 15, 8:32 PM · Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, Reading-Infrastructure-Team-Backlog, RESTBase
mobrovac added a comment to T217359: Possibly expand Kafka main-{eqiad,codfw} clusters in Q4 2019..

+1 for SSDs. We will be directly exposing time-based messages to consumers, so I imagine random disk access will greatly increase in the mid-term.

Mon, Apr 15, 8:28 PM · User-herron, Core Platform Team (Modern Event Platform (TEC2)), Core Platform Team Backlog (Watching / External), Services (watching), EventBus, Analytics, Operations
mobrovac added a parent task for T172497: Fix mediawiki heartbeat model, change pt-heartbeat model to not use super-user, avoid SPOF and switch automatically to the real master without puppet dependency: T88445: MediaWiki active/active datacenter investigation and work (tracking).
Mon, Apr 15, 8:20 PM · Services (watching), Core Platform Team Backlog (Watching / External), Performance-Team (Radar), MediaWiki-Database, Wikimedia-Incident, DBA
mobrovac added a subtask for T88445: MediaWiki active/active datacenter investigation and work (tracking): T172497: Fix mediawiki heartbeat model, change pt-heartbeat model to not use super-user, avoid SPOF and switch automatically to the real master without puppet dependency.
Mon, Apr 15, 8:20 PM · Core Platform Team (Multi-DC (TEC1)), User-mobrovac, Services (designing), Core Platform Team Backlog (Epic), Performance-Team (Radar), Availability (MediaWiki-MultiDC), Epic
mobrovac added projects to T172497: Fix mediawiki heartbeat model, change pt-heartbeat model to not use super-user, avoid SPOF and switch automatically to the real master without puppet dependency: Core Platform Team (Multi-DC (TEC1)), Services (watching), User-mobrovac.
Mon, Apr 15, 8:19 PM · Services (watching), Core Platform Team Backlog (Watching / External), Performance-Team (Radar), MediaWiki-Database, Wikimedia-Incident, DBA
mobrovac closed T220010: Remove listing endpoints from REST API as Resolved.

Merged, resolving.

Mon, Apr 15, 2:43 AM · Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (RESTBase Split (CDP2)), RESTBase

Fri, Apr 12

mobrovac triaged T186371: Redesign revision-related event schemas for MCR as Normal priority.
Fri, Apr 12, 11:08 PM · Core Platform Team Backlog (Next), Services (next), Multi-Content-Revisions, EventBus, Analytics
mobrovac edited projects for T186371: Redesign revision-related event schemas for MCR, added: Services (next), Core Platform Team Backlog (Next); removed Core Platform Team Kanban (Doing), Services (doing).
Fri, Apr 12, 11:08 PM · Core Platform Team Backlog (Next), Services (next), Multi-Content-Revisions, EventBus, Analytics
mobrovac closed T190689: FY17/18 Q4 Program 7 Services Goal: Language variants support, a subtask of T111231: Page previews for Wikidata, as Resolved.
Fri, Apr 12, 11:07 PM · Readers-Web-Backlog (Design), Mobile-Content-Service, Reading-Infrastructure-Team-Backlog, Patch-For-Review, Wikimania-Hackathon-2018, Wikimedia-Site-requests, User-aude, Beta-Feature, Story, Page-Previews, MediaWiki-extensions-WikibaseRepository, Wikidata
mobrovac closed T190689: FY17/18 Q4 Program 7 Services Goal: Language variants support as Resolved.

The RESTBase part has been done long ago.

Fri, Apr 12, 11:07 PM · Core Platform Team Kanban (Done with CPT), Services (done), Goal, RESTBase-Cassandra, RESTBase-API, RESTBase, Cassandra
mobrovac closed T190689: FY17/18 Q4 Program 7 Services Goal: Language variants support, a subtask of T148854: Use RESTBase for zhwiki, as Resolved.
Fri, Apr 12, 11:07 PM · Services (watching), Core Platform Team Backlog (Watching / External), RESTBase-API, Chinese-Sites, Wikipedia-Android-App-Backlog
mobrovac lowered the priority of T173447: Generalized rate limiting, deduplication, and job scheduling module from High to Low.
Fri, Apr 12, 11:06 PM · Core Platform Team Backlog (Later), Services (later), EventBus, MediaWiki-JobQueue, Analytics, ChangeProp
mobrovac added a comment to T220855: Split the RESTBase execution paths.

PR #1103 is the current work-in-progress patch for the back-end service's config.

Fri, Apr 12, 9:23 PM · Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing), RESTBase-API, RESTBase
mobrovac updated the task description for T220449: Split RESTBase in two services: storage service and API router/proxy.
Fri, Apr 12, 9:20 PM · User-mobrovac, serviceops, Epic, RESTBase, Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing)
mobrovac triaged T220855: Split the RESTBase execution paths as Normal priority.
Fri, Apr 12, 9:19 PM · Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing), RESTBase-API, RESTBase
mobrovac added a comment to T220661: EventGate service runner worker occasionally killed, usually during higher load.

Having said that the shutdown approach is incorrect for 1 worker situation I still think that the proper fix for this would be increasing the number of workers to at least 2.

Fri, Apr 12, 5:38 PM · Analytics-Kanban, Patch-For-Review, Services (watching), EventBus, Analytics

Thu, Apr 11

mobrovac updated the task description for T220449: Split RESTBase in two services: storage service and API router/proxy.
Thu, Apr 11, 8:29 PM · User-mobrovac, serviceops, Epic, RESTBase, Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing)
mobrovac added a subtask for T220449: Split RESTBase in two services: storage service and API router/proxy: T220761: Determine the names of the RESTBase front- and back-end services.
Thu, Apr 11, 8:25 PM · User-mobrovac, serviceops, Epic, RESTBase, Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing)
mobrovac added a parent task for T220761: Determine the names of the RESTBase front- and back-end services: T220449: Split RESTBase in two services: storage service and API router/proxy.
Thu, Apr 11, 8:25 PM · RESTBase, Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2))
mobrovac triaged T220761: Determine the names of the RESTBase front- and back-end services as Normal priority.
Thu, Apr 11, 8:24 PM · RESTBase, Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2))
mobrovac removed a project from T205911: Track and install additional npm packages for all service container images: Patch-For-Review.
Thu, Apr 11, 7:18 PM · Core Platform Team Kanban (Done with CPT), Services (done), Release-Engineering-Team (Watching / External), Operations, Release Pipeline
mobrovac committed rMSCR6a6e0dd358c9: Bump service-runner to v2.6.17 (authored by mobrovac).
Bump service-runner to v2.6.17
Thu, Apr 11, 6:44 PM
mobrovac triaged T220725: Factor lib/kafka.js out of eventgate and change-propagation into its own library as Normal priority.
Thu, Apr 11, 6:38 PM · Core Platform Team (Modern Event Platform (TEC2)), Core Platform Team Backlog (Next), Services (next), EventBus, Analytics
mobrovac edited projects for T220725: Factor lib/kafka.js out of eventgate and change-propagation into its own library, added: Services (next), Core Platform Team Backlog (Next), Core Platform Team (Modern Event Platform (TEC2)); removed Services (watching).
Thu, Apr 11, 6:38 PM · Core Platform Team (Modern Event Platform (TEC2)), Core Platform Team Backlog (Next), Services (next), EventBus, Analytics
mobrovac added projects to T220709: Upgrade statsd_exporter to 0.9: Services (watching), Core Platform Team Backlog (Watching / External).
Thu, Apr 11, 5:34 PM · Core Platform Team Backlog (Watching / External), Services (watching), Analytics, EventBus, monitoring, User-fgiunchedi, Operations

Wed, Apr 10

mobrovac added a comment to T220657: Establish Architecture Principles as a policy.

+1. We should also find a way to evangelise it. Perhaps by doing a tech talk or something similar would do and/or present it at the relevant conferences and venues (Hackathon, Wikimania at least).

Wed, Apr 10, 8:31 PM · TechCom-RFC, TechCom
mobrovac added a comment to T210651: Switch all PDF render traffic to new Proton service.

We have been splitting the traffic (thus load testing Proton on real traffic) for a long time now, so the switch by now is only switching the content that's actually served to the clients. I do not think it worths making a multi-stage deployment here.

Wed, Apr 10, 8:20 PM · Reading-Infrastructure-Team-Backlog, Core Platform Team Backlog (Later), Services (next), Readers-Web-Backlog (Tracking), Proton
mobrovac closed T197862: Increase the CPU count for proton[12]00[12] as Resolved.

Given that overall proton has been (relatively) stable in the current config and that a plan to move it to k8s exists, I'll go ahead and resolve this.

Wed, Apr 10, 7:41 PM · Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Kanban (Done with CPT), Services (done), Reading-Infrastructure-Team-Backlog, Operations, Proton
mobrovac closed T197862: Increase the CPU count for proton[12]00[12], a subtask of T181084: [EPIC] Deploy the mediawiki-services-chromium-render service (Proton), as Resolved.
Wed, Apr 10, 7:41 PM · Reading-Infrastructure-Team-Backlog, Readers-Web-Backlog, Readers-Web-Kanbanana-Board-Old, Proton, Epic
mobrovac edited projects for T220574: Continuous errors on several REST API resources (probably related to MCS release), added: Services (done), Core Platform Team Kanban (Done with CPT), Core Platform Team (Security, stability, performance and scalability (TEC1)); removed Patch-For-Review, Services.
Wed, Apr 10, 4:42 PM · Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Kanban (Done with CPT), Services (done), Reading-Infrastructure-Team-Backlog (Kanban), RESTBase, Mobile-Content-Service, Operations
mobrovac added a project to T215465: Development policy: Require use of storage abstraction layers: User-mobrovac.
Wed, Apr 10, 4:38 PM · User-mobrovac, Services (watching), Core Platform Team Backlog (Watching / External), Performance-Team, TechCom

Tue, Apr 9

mobrovac edited projects for T208087: Replace remaining Samsung SSDs, added: Core Platform Team Kanban (Doing), Services (doing); removed Patch-For-Review, Core Platform Team Backlog (Watching / External).
Tue, Apr 9, 10:13 PM · Patch-For-Review, Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (Security, stability, performance and scalability (TEC1)), User-Eevans, Cassandra
mobrovac added a comment to T211881: graphoid: Code stewardship request.

Hi all - we had a meeting on this and a question emerged. Is it possible to run Graph without Graphoid if the clients are JavaScript-only?

Tue, Apr 9, 7:26 PM · Core Platform Team Backlog (Watching / External), Services (watching), Release-Engineering-Team (Kanban), Operations, Code-Stewardship-Reviews, Graphoid
mobrovac updated the task description for T218218: Make RESTBase spec standard compliant and switch to OpenAPI 3.0.
Tue, Apr 9, 7:22 PM · Patch-For-Review, Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2)), RESTBase, RESTBase-API, serviceops, Operations
mobrovac added a comment to T218218: Make RESTBase spec standard compliant and switch to OpenAPI 3.0.

One other thing left to do here: replace optional parameters in the /sys hierarchy specs.

Tue, Apr 9, 7:22 PM · Patch-For-Review, Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2)), RESTBase, RESTBase-API, serviceops, Operations
mobrovac closed T220477: Eventbus errors: Failed processing event: Failed validating at path rev_id as Resolved.

Patch merged, will go out with today's train, resolving.

Tue, Apr 9, 5:14 PM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Kanban (Done with CPT), Services (done), EventBus, Analytics, Operations

Mon, Apr 8

mobrovac added subtasks for T220461: Simplify storage semantics: T215960: Simplify MCS storage model, T219548: restbase-mod-table-* simplification and improvements, T215956: Consider stashing data-parsoid for VE .
Mon, Apr 8, 11:23 PM · RESTBase, Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing)
mobrovac added a parent task for T215956: Consider stashing data-parsoid for VE : T220461: Simplify storage semantics.
Mon, Apr 8, 11:23 PM · Services (doing), Core Platform Team Kanban (Doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, User-mobrovac, Parsoid, VisualEditor, RESTBase
mobrovac added a parent task for T215960: Simplify MCS storage model: T220461: Simplify storage semantics.
Mon, Apr 8, 11:23 PM · Core Platform Team Kanban (Doing), Services (doing), Core Platform Team (RESTBase Split (CDP2)), User-Eevans, Reading-Infrastructure-Team-Backlog, RESTBase
mobrovac added a parent task for T219548: restbase-mod-table-* simplification and improvements: T220461: Simplify storage semantics.
Mon, Apr 8, 11:23 PM · Core Platform Team Backlog (Designing), Services (designing), Core Platform Team (RESTBase Split (CDP2)), RESTBase
mobrovac triaged T220461: Simplify storage semantics as High priority.
Mon, Apr 8, 11:21 PM · RESTBase, Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing)
mobrovac edited projects for T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons, added: Services (done), Core Platform Team Kanban (Done with CPT); removed Services (doing), Core Platform Team Kanban (Doing), Patch-For-Review.
Mon, Apr 8, 9:09 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus
mobrovac added a project to T220449: Split RESTBase in two services: storage service and API router/proxy: User-mobrovac.
Mon, Apr 8, 8:54 PM · User-mobrovac, serviceops, Epic, RESTBase, Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing)
mobrovac triaged T220449: Split RESTBase in two services: storage service and API router/proxy as Normal priority.
Mon, Apr 8, 8:54 PM · User-mobrovac, serviceops, Epic, RESTBase, Core Platform Team (RESTBase Split (CDP2)), Core Platform Team Kanban (Doing), Services (doing)