Page MenuHomePhabricator
Feed Advanced Search

Aug 1 2019

Pchelolo closed T229238: Change-prop producer not connected log spam as Resolved.

After deploying it to production the chatty useless log messages no longer show up.

Aug 1 2019, 1:36 PM · ChangeProp, Platform Engineering (Needs Cleaning - Services Operations), Platform Team Workboards (Clinic Duty Team)
Pchelolo closed T158265: runner.stop() only calls worker.stop() if num_workers === 0 as Resolved.

Merged and deployed. resolving.

Aug 1 2019, 1:35 PM · Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)), Platform Team Workboards (Clinic Duty Team), service-runner
Pchelolo added a comment to T182827: Support X-Wikimedia-Debug header for services.

@Pchelolo Would it be hard to include maps as well?

Aug 1 2019, 1:20 PM · Parsoid (Tracking), Platform Engineering (Icebox), Mobile-Content-Service, Product-Infrastructure-Team-Backlog-Deprecated
Pchelolo added a comment to T147991: Create MySQL RESTBase backend.

sure, a PR will be nice!

Aug 1 2019, 1:51 AM · restbase-mysql, Platform Team Workboards (Clinic Duty Team), RESTBase
Pchelolo added a comment to T229521: Cannot read property 'type' of undefined in MCS.

Unfortunately, it doesn't look like our logging is capturing the request URLs in this case, so I don't know how to reproduce this.

Aug 1 2019, 1:50 AM · Product-Infrastructure-Team-Backlog-Deprecated (Kanban), wikimedia-page-library, Page Content Service
Pchelolo created T229521: Cannot read property 'type' of undefined in MCS.
Aug 1 2019, 12:09 AM · Product-Infrastructure-Team-Backlog-Deprecated (Kanban), wikimedia-page-library, Page Content Service

Jul 31 2019

Pchelolo closed T122245: REST API entry point web request statistics at the Varnish level, a subtask of T102079: Metrics about the use of the Wikimedia web APIs, as Declined.
Jul 31 2019, 10:44 PM · User-bd808, Analytics-Radar, Product-Infrastructure-Team-Backlog-Deprecated, Reading-Admin, Epic, DevRel-September-2015, ECT-August-2015, Research-consulting, MediaWiki-Action-API, ECT-July-2015, Developer-Advocacy
Pchelolo closed T122245: REST API entry point web request statistics at the Varnish level, a subtask of T159264: Prototype counting of requests with real time (streaming data), as Declined.
Jul 31 2019, 10:44 PM · Analytics
Pchelolo closed T122245: REST API entry point web request statistics at the Varnish level as Declined.

In rare cases when we need it we can run a query on hadoop.

Jul 31 2019, 10:44 PM · Analytics-Radar, Platform Team Legacy (Later), RESTBase
Pchelolo merged T211501: Page Previews should take FlaggedRevs' stabilization setting into account into T169116: Support flagged revisions in RESTBase.
Jul 31 2019, 10:44 PM · Platform Engineering (Icebox), RESTBase-API, RESTBase, VisualEditor, Product-Infrastructure-Team-Backlog-Deprecated, Mobile-Content-Service
Pchelolo merged task T211501: Page Previews should take FlaggedRevs' stabilization setting into account into T169116: Support flagged revisions in RESTBase.
Jul 31 2019, 10:44 PM · Platform Team Legacy (Later), Services (later), Web-Team-Backlog (Tracking), Page-Previews, MediaWiki-extensions-FlaggedRevs
Pchelolo closed T127387: Split slash decoding from general percent normalization in Varnish VCL, a subtask of T127370: Varnish not purging RESTBase URIs, as Resolved.
Jul 31 2019, 10:41 PM · Product-Infrastructure-Team-Backlog-Deprecated, Traffic, Mobile-Content-Service, RESTBase, SRE, Services
Pchelolo closed T127387: Split slash decoding from general percent normalization in Varnish VCL as Resolved.

doesn't seem to be any more movement or things to be done here.

Jul 31 2019, 10:41 PM · Platform Team Legacy (Later), Patch-For-Review, Services (later), Traffic, RESTBase, SRE
Pchelolo edited projects for T121571: Improve logstash logging for graphoid, mobileapps, mathoid, added: Platform Team Workboards (Clinic Duty Team); removed Platform Team Legacy (Later), Services (next).

I need to verify this is still the case and file subtasks for the appropriate teams.

Jul 31 2019, 10:40 PM · Platform Team Workboards (Clinic Duty Team), User-mobrovac, Services-next
Pchelolo closed T94574: Switch Flow from ExternalStore to RESTBase as Declined.

No movement in 3 years clearly indicates this is not needed. Please reopen if I'm mistaken.

Jul 31 2019, 10:38 PM · Platform Team Legacy (Later), Growth-Team, Services (later), Patch-For-Review, RESTBase, StructuredDiscussions, Collaboration-Team-Triage, Parsoid
Pchelolo closed T90633: Support conditional requests as Declined.
Jul 31 2019, 10:37 PM · Platform Team Legacy (Later), Services (later), RESTBase-release-1.0, RESTBase-API, RESTBase
Pchelolo closed T141902: RESTBase can't access MW API for donate.wikimedia.org as Declined.
Jul 31 2019, 10:36 PM · Platform Team Legacy (Later), Services (later), RESTBase
Pchelolo closed T149215: Create the overall processing delay metric for ChangeProp as Declined.

No real need for this has been.

Jul 31 2019, 10:35 PM · Platform Team Legacy (Later), Services (later)
Pchelolo closed T153776: Support switching change-prop backlog processing to other DC as Declined.

I don't think we will/need to implement this.

Jul 31 2019, 10:35 PM · Platform Team Legacy (Later), Services (next), ChangeProp
Pchelolo added a comment to T153801: File and global user pages should not be redirected.

Do we still need to work on this or it is resolved on the app side?

Jul 31 2019, 10:26 PM · Platform Team Legacy (Later), Patch-For-Review, Product-Infrastructure-Team-Backlog-Deprecated, Services (next), Parsoid, RESTBase, Mobile-Content-Service
Pchelolo added a comment to T149619: Consider removing basefields option from citoid API.

@Mvolz is it done or still considered or can be closed?

Jul 31 2019, 10:14 PM · Platform Team Legacy (Later), Services (next), User-mobrovac, VisualEditor, Services-next, RESTBase-API, RESTBase, Citoid
Pchelolo closed T94187: Design & implement useful global title and revision listings, a subtask of T89568: RESTBase 1.0 release, as Declined.
Jul 31 2019, 10:12 PM · Services, RESTBase-release-1.0, RESTBase
Pchelolo closed T94187: Design & implement useful global title and revision listings as Declined.

Given REST API in MediaWiki, this is not needed anymore.

Jul 31 2019, 10:12 PM · Platform Team Legacy (Later), Services (later), RESTBase-release-1.0, RESTBase
Pchelolo closed T123237: Provide production jessie image with node 4.2; use this for service-runner build command, a subtask of T107762: Switch RESTBase to use Node.js 4.2, as Invalid.
Jul 31 2019, 10:11 PM · Blocked-on-Operations, SRE, Services, RESTBase
Pchelolo closed T123237: Provide production jessie image with node 4.2; use this for service-runner build command as Invalid.

We're updating to k8s and not running node 4 anymore.

Jul 31 2019, 10:11 PM · Platform Team Legacy (Later), Services (later), RESTBase, SRE
Pchelolo closed T124919: Story: Improve testing framework for node projects as Invalid.
Jul 31 2019, 10:10 PM · Math, Platform Team Legacy (Later), Services (later), Story, Mathoid, service-template-node
Pchelolo closed T157649: Partition the transclusions topic in ChangeProp as Declined.

We never actually got to the point where this was needed. we can reevaluate in the future.

Jul 31 2019, 10:09 PM · Patch-For-Review, serviceops, Platform Team Workboards (Clinic Duty Team), ChangeProp
Pchelolo closed T144104: Enhance section retrieval API as Resolved.

old section retrieval API has been deprecated and removed.

Jul 31 2019, 10:09 PM · Platform Team Legacy (Later), Services (next), RESTBase-API, RESTBase
Pchelolo closed T157620: Support providing source sha1 to the deploy build script as Declined.

We're moving to k8s, so the build script will be removed soon.

Jul 31 2019, 10:07 PM · Platform Team Legacy (Later), service-runner, Services (later)
Pchelolo merged T162332: Refactor shared code out of node template into T151399: Make service-template-node more modular.
Jul 31 2019, 10:07 PM · Platform Engineering (Icebox), service-template-node
Pchelolo merged task T162332: Refactor shared code out of node template into T151399: Make service-template-node more modular.
Jul 31 2019, 10:07 PM · Platform Team Legacy (Later), Services (next), Technical-Debt, service-template-node
Pchelolo closed T156209: Design notes for scalable and cost-effective revision archival storage, a subtask of T120171: RFC: Differentiate storage strategies for archival storage vs. hot current data, as Declined.
Jul 31 2019, 10:06 PM · Platform Team Legacy (Later), Services (later), Proposal, RESTBase
Pchelolo closed T156209: Design notes for scalable and cost-effective revision archival storage as Declined.
Jul 31 2019, 10:06 PM · Platform Team Legacy (Later), User-mobrovac, Services (later), Proposal, RESTBase
Pchelolo merged T153595: Unapproved article version shown in mobile apps into T169116: Support flagged revisions in RESTBase.
Jul 31 2019, 10:06 PM · Platform Engineering (Icebox), RESTBase-API, RESTBase, VisualEditor, Product-Infrastructure-Team-Backlog-Deprecated, Mobile-Content-Service
Pchelolo merged task T153595: Unapproved article version shown in mobile apps into T169116: Support flagged revisions in RESTBase.
Jul 31 2019, 10:05 PM · Platform Team Legacy (Later), Services (next), RESTBase-API, RESTBase, Product-Infrastructure-Team-Backlog-Deprecated, Mobile-Content-Service, iOS-app-Bugs, Android-app-Bugs, Wikipedia-Android-App-Backlog, MediaWiki-extensions-FlaggedRevs
Pchelolo closed T151518: Investigate adoption of flow in node services as Declined.

I believe this will never be done. Declining.

Jul 31 2019, 10:05 PM · Platform Team Legacy (Later), Services (later)
Pchelolo closed T117364: Get wikitext from restbase as Declined.
Jul 31 2019, 10:03 PM · Platform Team Legacy (Later), Services (later), RESTBase-API, RESTBase
Pchelolo closed T130641: Allocate a storage bucket for Parsoid's data-mw attribute , a subtask of T78676: Store & load data-mw separately, as Invalid.
Jul 31 2019, 10:02 PM · Growth-Team-Filtering, Parsoid, Platform Team Legacy (Later), Growth-Team, CX-cxserver, Services (later), Services-next, StructuredDiscussions, VisualEditor-Initialisation, Epic, VisualEditor-Performance, RESTBase, VisualEditor
Pchelolo closed T130641: Allocate a storage bucket for Parsoid's data-mw attribute as Invalid.
Jul 31 2019, 10:02 PM · Platform Team Legacy (Later), Services (next), Services-next, RESTBase
Pchelolo closed T125453: Consider improving API support for access by page id as Declined.
Jul 31 2019, 10:01 PM · Platform Team Legacy (Later), Services (later), RESTBase
Pchelolo closed T159479: Sampled rate-limiting for logger as Declined.
Jul 31 2019, 9:57 PM · Platform Team Legacy (Later), Services (later), service-runner
Pchelolo merged T164660: Restbase graphite alerts often showing up as UNKNOWN in icinga into T185089: Alert UNKNOWN for restbase cassandra graphite alerts.
Jul 31 2019, 9:56 PM · Platform Team Workboards (Clinic Duty Team), User-Eevans, RESTBase-Cassandra, RESTBase, Cassandra, Goal
Pchelolo merged task T164660: Restbase graphite alerts often showing up as UNKNOWN in icinga into T185089: Alert UNKNOWN for restbase cassandra graphite alerts.
Jul 31 2019, 9:56 PM · Platform Team Legacy (Later), Services (next), RESTBase, Cassandra
Pchelolo moved T179374: Use one keyspace per storage group from Inbox to Services Tech Debt on the Platform Engineering board.
Jul 31 2019, 9:41 PM · Platform Engineering, Cassandra, RESTBase
Pchelolo edited projects for T179374: Use one keyspace per storage group, added: Platform Engineering; removed Platform Team Legacy (Designing), Services (designing).
Jul 31 2019, 9:40 PM · Platform Engineering, Cassandra, RESTBase
Pchelolo closed T188667: Support dynamic rates for ChangeProp, a subtask of T157088: [EPIC] Develop a JobQueue backend based on EventBus, as Declined.
Jul 31 2019, 9:39 PM · Data-Engineering, Analytics-Radar, WMF-JobQueue, Platform Team Workboards (Done with CPT), Services (done), MediaWiki-Core-JobQueue, Epic, User-mobrovac, ChangeProp, Event-Platform
Pchelolo closed T188667: Support dynamic rates for ChangeProp as Declined.

This is definitely overcomplication of the system.

Jul 31 2019, 9:39 PM · Analytics-Radar, Platform Team Legacy (Designing), Services (designing), MediaWiki-Core-JobQueue, User-mobrovac, ChangeProp, Event-Platform
Pchelolo closed T87393: Figure out long-term page rename handling and page history strategy, especially for old revisions, a subtask of T89568: RESTBase 1.0 release, as Invalid.
Jul 31 2019, 9:36 PM · Services, RESTBase-release-1.0, RESTBase
Pchelolo closed T87393: Figure out long-term page rename handling and page history strategy, especially for old revisions, a subtask of T110152: RESTBase doesn't purge renamed page -> VisualEditor unable to create moved page, as Invalid.
Jul 31 2019, 9:36 PM · User-Ryasmeen, VisualEditor, Parsoid, RESTBase
Pchelolo closed T87393: Figure out long-term page rename handling and page history strategy, especially for old revisions as Invalid.

Given that we only store latest revision now, it's not a problem any longer.

Jul 31 2019, 9:36 PM · Platform Team Legacy (Designing), Services (designing), RESTBase-release-1.0, RESTBase
Pchelolo merged T163462: FlaggedRevs for Mobile Content Service into T169116: Support flagged revisions in RESTBase.
Jul 31 2019, 9:35 PM · Platform Engineering (Icebox), RESTBase-API, RESTBase, VisualEditor, Product-Infrastructure-Team-Backlog-Deprecated, Mobile-Content-Service
Pchelolo merged task T163462: FlaggedRevs for Mobile Content Service into T169116: Support flagged revisions in RESTBase.
Jul 31 2019, 9:35 PM · Platform Team Legacy (Designing), Product-Infrastructure-Team-Backlog-Deprecated, RESTBase-API, RESTBase, Services (designing), Mobile-Content-Service, MediaWiki-extensions-FlaggedRevs
Pchelolo closed T199427: Separate dev Change-Prop from production Kafka cluster as Declined.

We don't really have/use change-prop in dev cluster anymore and I don't think we will.

Jul 31 2019, 9:34 PM · Analytics-Radar, Sustainability (Incident Followup), Platform Team Legacy (Designing), ChangeProp, SRE, Services (designing)
Pchelolo moved T182827: Support X-Wikimedia-Debug header for services from Inbox to Services Tech Debt on the Platform Engineering board.
Jul 31 2019, 9:33 PM · Parsoid (Tracking), Platform Engineering (Icebox), Mobile-Content-Service, Product-Infrastructure-Team-Backlog-Deprecated
Pchelolo moved T199431: Consider the possibility of separating ChangeProp and JobQueue on Kafka level from Inbox to Services Tech Debt on the Platform Engineering board.
Jul 31 2019, 9:32 PM · Data-Engineering-Icebox, Analytics-Radar, Sustainability (Incident Followup), Platform Engineering (Icebox), ChangeProp, MediaWiki-Core-JobQueue
Pchelolo updated subscribers of T164236: Include size of article in Summary.

@bearND Do you think it is still needed?

Jul 31 2019, 9:32 PM · Platform Team Legacy (Designing), Services (designing), Reading List Service, RESTBase-API, Product-Infrastructure-Team-Backlog-Deprecated
Pchelolo edited projects for T199431: Consider the possibility of separating ChangeProp and JobQueue on Kafka level, added: Platform Engineering; removed Platform Team Legacy (Designing), Services (designing).
Jul 31 2019, 9:31 PM · Data-Engineering-Icebox, Analytics-Radar, Sustainability (Incident Followup), Platform Engineering (Icebox), ChangeProp, MediaWiki-Core-JobQueue
Pchelolo merged T168922: Refactor Change-Prop to use filters into T201526: Simplify change-propagation configuration.
Jul 31 2019, 9:31 PM · ChangeProp, Platform Engineering (Icebox), Dependency-Tracking, User-mobrovac
Pchelolo merged task T168922: Refactor Change-Prop to use filters into T201526: Simplify change-propagation configuration.
Jul 31 2019, 9:31 PM · Platform Team Legacy (Designing), ChangeProp, Services (designing)
Pchelolo closed T172815: Improve stability and maintainability of our browser-based PDF render service, a subtask of T159922: pdfrender fails to serve requests since Mar 8 00:30:32 UTC on scb1003, as Invalid.
Jul 31 2019, 9:30 PM · Services (done), Web-Team-Backlog (Tracking), SRE, Electron-PDFs
Pchelolo closed T172815: Improve stability and maintainability of our browser-based PDF render service as Invalid.

After switching to Proton this is not an issue any longer.

Jul 31 2019, 9:30 PM · Platform Team Legacy (Designing), Electron-PDFs, OfflineContentGenerator, SRE, Services (designing)
Pchelolo closed T175957: Options for implementing JobQueue statistics methods, a subtask of T157088: [EPIC] Develop a JobQueue backend based on EventBus, as Declined.
Jul 31 2019, 9:29 PM · Data-Engineering, Analytics-Radar, WMF-JobQueue, Platform Team Workboards (Done with CPT), Services (done), MediaWiki-Core-JobQueue, Epic, User-mobrovac, ChangeProp, Event-Platform
Pchelolo closed T175957: Options for implementing JobQueue statistics methods as Declined.

There has not been any complains about kafka queue not supporting these statistics, so I don't think it's worth investing time into it.

Jul 31 2019, 9:29 PM · Analytics-Radar, Platform Team Legacy (Designing), Services (designing), MediaWiki-Core-JobQueue, User-mobrovac, ChangeProp, Event-Platform
Pchelolo edited projects for T182827: Support X-Wikimedia-Debug header for services, added: Platform Engineering; removed Platform Team Legacy (Designing), Services (designing).
Jul 31 2019, 9:28 PM · Parsoid (Tracking), Platform Engineering (Icebox), Mobile-Content-Service, Product-Infrastructure-Team-Backlog-Deprecated
Pchelolo closed T189728: Design strategy for updating MCS/PCS content, a subtask of T189750: Expose new PCS JSON endpoints via RESTBase, as Declined.
Jul 31 2019, 9:28 PM · Services (done), RESTBase-API, Product-Infrastructure-Team-Backlog-Deprecated (Kanban), Page Content Service
Pchelolo closed T189728: Design strategy for updating MCS/PCS content as Declined.

Given that matching TID for content pieces never became a requirement for the apps, we don't need to overcomplicate things here.

Jul 31 2019, 9:28 PM · Platform Team Legacy (Designing), Mobile-Content-Service, RESTBase, Product-Infrastructure-Team-Backlog-Deprecated, Services (designing)
Pchelolo closed T163116: Define persistence backends for Push Notification Service as Declined.

doesn't seem to be needed anymore.

Jul 31 2019, 9:27 PM · Platform Team Legacy (Designing), Product-Infrastructure-Team-Backlog-Deprecated, User-mobrovac, Services (designing), Push-Notification-Service (Project Setup), Reading Epics (Push Notifications)
Pchelolo closed T159585: HyperSwitch module caching somewhat limited by config object equality as Declined.
Jul 31 2019, 9:26 PM · Platform Team Legacy (Designing), Services (designing), RESTBase
Pchelolo closed T156734: Create Location Search APIs in RESTBase as Declined.

There was neither movement on implementing it nor further requests from audiences. I don't think it's needed anymore.

Jul 31 2019, 9:25 PM · Platform Team Legacy (Designing), Product-Infrastructure-Team-Backlog-Deprecated, Services (designing), Reading Epics (Location Search), Mobile-Content-Service
Pchelolo closed T130663: WIP RFC: Reference API requirements and options as Invalid.

Reading infrastructure has created references endpoint for the use of mobile.

Jul 31 2019, 9:24 PM · Platform Team Legacy (Designing), TechCom-RFC, Web-Team-Backlog (Tracking), Services (designing), Cite, Reading-Admin, Internet-Archive, Parsoid
Pchelolo closed T116840: Cached REST end point for imageinfo requests, a subtask of T115876: High-traffic API endpoints to cover in RESTBase, as Invalid.
Jul 31 2019, 9:23 PM · Platform Team Legacy (Later), Web-Team-Backlog (Tracking), Services (next), Parsoid, RESTBase-API, RESTBase
Pchelolo closed T116840: Cached REST end point for imageinfo requests as Invalid.

Given introduction of the REST API in MediaWiki, I believe this is no longer nesessary in RESTBase

Jul 31 2019, 9:23 PM · Platform Team Legacy (Designing), Services (designing), Web-Team-Backlog (Tracking), RESTBase, RESTBase-API, Parsoid
Pchelolo closed T87556: Thoughts on element IDs, sections, incremental parsing and fast section editing, a subtask of T91533: Services team Q3 (Jan - March 2015) quarterly goal tracking, as Invalid.
Jul 31 2019, 9:22 PM · Services
Pchelolo closed T87556: Thoughts on element IDs, sections, incremental parsing and fast section editing as Invalid.

I believe this is invalid after so many years and changes in thinking.

Jul 31 2019, 9:22 PM · Platform Team Legacy (Designing), Services (designing), Blocked-on-Parsoid, VisualEditor, VisualEditor-Performance, RESTBase-API, Parsoid
Pchelolo moved T201526: Simplify change-propagation configuration from Inbox to Services Tech Debt on the Platform Engineering board.
Jul 31 2019, 9:20 PM · ChangeProp, Platform Engineering (Icebox), Dependency-Tracking, User-mobrovac
Pchelolo edited projects for T201526: Simplify change-propagation configuration, added: Platform Engineering; removed Platform Team Legacy (Designing), Services (designing).
Jul 31 2019, 9:19 PM · ChangeProp, Platform Engineering (Icebox), Dependency-Tracking, User-mobrovac
Pchelolo removed a project from T229019: ChangePropagation should mirror reparse events to both Parsoid/PHP and Parsoid/JS: Platform Engineering.
Jul 31 2019, 9:17 PM · Platform Team Workboards (Architecture Review Workboard), Platform Team Initiatives (Parsoid PHP (CDP2)), ChangeProp, Parsoid-PHP
Pchelolo moved T229487: Deprecate and remove support for x-request/setup-handler from Inbox to Services Tech Debt on the Platform Engineering board.
Jul 31 2019, 9:16 PM · Platform Engineering, RESTBase
Pchelolo updated the task description for T229487: Deprecate and remove support for x-request/setup-handler.
Jul 31 2019, 9:16 PM · Platform Engineering, RESTBase
Pchelolo created T229487: Deprecate and remove support for x-request/setup-handler.
Jul 31 2019, 9:16 PM · Platform Engineering, RESTBase
Pchelolo closed T219900: Evaluate url-template package for use in hyperswitch as Declined.

I have evaluated the library, and it's a good piece of software, but currently, the URI templating functionality supported/required by HyperSwitch and RESTBase is too different from what the library provides. I don't believe using the library would worth the investment.

Jul 31 2019, 8:44 PM · Platform Team Initiatives (RESTBase Split (CDP2)), Platform Team Workboards (Clinic Duty Team), RESTBase
Pchelolo added a comment to T229286: Resolve service instability due to excessive event loop blockage since starting PCS response pregeneration.

Weird. Why are we still getting internal requests for /feed/onthisday/all, then?

Jul 31 2019, 7:01 PM · Patch-For-Review, Product-Infrastructure-Team-Backlog-Deprecated, Epic, Page Content Service, Mobile-Content-Service, serviceops
Pchelolo added a comment to T229286: Resolve service instability due to excessive event loop blockage since starting PCS response pregeneration.

So one issue right off the bat is that we're generating /feed/onthisday/all in the service in addition to the individual pieces, which seems like a lot of redundant work. How much work would it be to compose /feed/onthisday/all in RESTBase instead, the way we've been doing with /feed/featured for some time now?

Jul 31 2019, 6:51 PM · Patch-For-Review, Product-Infrastructure-Team-Backlog-Deprecated, Epic, Page Content Service, Mobile-Content-Service, serviceops
Pchelolo moved T158265: runner.stop() only calls worker.stop() if num_workers === 0 from Inbox to Blocked Externally on the Platform Team Workboards (Clinic Duty Team) board.
Jul 31 2019, 6:49 PM · Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)), Platform Team Workboards (Clinic Duty Team), service-runner
Pchelolo added a comment to T158265: runner.stop() only calls worker.stop() if num_workers === 0.

https://github.com/wikimedia/service-runner/pull/220

Jul 31 2019, 6:49 PM · Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)), Platform Team Workboards (Clinic Duty Team), service-runner
Pchelolo added a comment to T229286: Resolve service instability due to excessive event loop blockage since starting PCS response pregeneration.

Aside from that, I would have expected that the pregeneration jobs would be finishing up by now and the worker deaths subsiding, but instead it looks like they're getting more frequent over the past couple of days. (Any idea when those will be finished, @Pchelolo?)

Jul 31 2019, 6:37 PM · Patch-For-Review, Product-Infrastructure-Team-Backlog-Deprecated, Epic, Page Content Service, Mobile-Content-Service, serviceops
Pchelolo lowered the priority of T182372: Make Kafka JobQueue use Special:RunSingleJob from Medium to Low.
Jul 31 2019, 2:35 PM · Analytics-Radar, Platform Engineering, MediaWiki-Core-JobQueue, User-mobrovac, ChangeProp, Event-Platform

Jul 30 2019

Pchelolo committed rMSCP08a3e6be07bb: Report 400 errors by default..
Report 400 errors by default.
Jul 30 2019, 10:17 PM
Pchelolo closed T229277: Change-Prop should report 400 errors from endpoints as Resolved.
Jul 30 2019, 10:15 PM · Platform Engineering (Needs Cleaning - Services Operations), Platform Team Workboards (Clinic Duty Team), ChangeProp, Wikimedia-Incident
Pchelolo committed rMSCD76b6639e1221: Update change-propagation to 7a39183.
Update change-propagation to 7a39183
Jul 30 2019, 10:11 PM
Pchelolo moved T226109: Write incident report for jobs not being executed on 1.34.0-wmf.10 from Doing(WIP:5) to Blocked Externally on the Platform Team Workboards (Clinic Duty Team) board.
Jul 30 2019, 10:06 PM · Analytics-Radar, Platform Team Workboards (Clinic Duty Team), Event-Platform, SRE, WMF-JobQueue, MassMessage
Pchelolo closed T229060: Enable language_variants_filter for PCS endpoints, a subtask of T229012: zhwiki language variant support for references endpoint, as Resolved.
Jul 30 2019, 10:03 PM · Chinese-Sites, Product-Infrastructure-Team-Backlog-Deprecated (Kanban), Page Content Service
Pchelolo closed T229060: Enable language_variants_filter for PCS endpoints as Resolved.
Jul 30 2019, 10:03 PM · Platform Team Workboards (Clinic Duty Team), Product-Infrastructure-Team-Backlog-Deprecated
Pchelolo added a comment to T229060: Enable language_variants_filter for PCS endpoints.

The fix has been deployed. Now all PCS endpoints support language variants.

Jul 30 2019, 10:03 PM · Platform Team Workboards (Clinic Duty Team), Product-Infrastructure-Team-Backlog-Deprecated
Pchelolo committed rGRBDc7e0e3313a08: Update restbase to 54919b3.
Update restbase to 54919b3
Jul 30 2019, 9:54 PM
Pchelolo created T229375: Handle nested <references/> in <references /> correctly.
Jul 30 2019, 9:46 PM · Parsoid, Page Content Service, Product-Infrastructure-Team-Backlog-Deprecated
Pchelolo committed rMWVA96192c9bead3: Use new style of RESTBase config..
Use new style of RESTBase config.
Jul 30 2019, 9:39 PM
Pchelolo closed T227842: RESTBase with VisualEditor gives TypeError: Cannot set property 'etag' of undefined as Resolved.
Jul 30 2019, 9:31 PM · Platform Team Workboards (Clinic Duty Team), RESTBase
Pchelolo closed T228939: Restructure vagrant RESTBase config to match production as Resolved.
Jul 30 2019, 9:24 PM · Patch-For-Review, MediaWiki-Vagrant, RESTBase, Platform Team Workboards (Clinic Duty Team), Platform Engineering (Needs Cleaning - Code Health (TEC13))
Pchelolo added a comment to T229247: [Bug] mobile-html output not varying based on accept-language header.

The fix will be deployed today, see the T229060 for the details.

Jul 30 2019, 4:12 PM · Platform Team Workboards (Clinic Duty Team), Page Content Service, Product-Infrastructure-Team-Backlog-Deprecated