Pchelolo
User

Projects (6)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Jun 24 2015, 10:23 AM (139 w, 1 d)
Availability
Available
LDAP User
Unknown
MediaWiki User
Pchelolo

Recent Activity

Yesterday

Pchelolo added a comment to T171052: Add disambiguation page handling in Page Summary API.

From what I remember is that there wasn't a use case for it.

Thu, Feb 22, 10:18 PM · Reading-Infrastructure-Team-Backlog (Kanban), Readers-Web-Backlog (Tracking), Mobile-Content-Service, Page-Previews
Pchelolo added a project to T187362: Failure in EventBus schema for mediawiki/revision/visibility-change: Services (watching).

@awight I've checked the message that you've provided and that one passes the validation.

Thu, Feb 22, 6:21 PM · Services (watching), Analytics-Kanban, EventBus
Pchelolo created T188027: EventBus schema validation should report the name of the failed property.
Thu, Feb 22, 6:13 PM · Services (watching), Analytics, EventBus
Pchelolo reopened T169006: Correctly redirect in Parsoid /transform/wikitext/to/lint endpoint as "Open".

Right now a POST to /v3/transform/wikitext/to/lint/{title} returns a 307 redirect to /en.wikipedia.org/v3/page/lint/{title}/{revision}, but the former URI doesn't exist - POST to it returns a 404, GET returns an error that {"error":"Invalid page format: lint"}.

Thu, Feb 22, 1:29 PM · Patch-For-Review, Parsoid-Web-API, Parsoid, Services (watching), MediaWiki-extensions-Linter
Pchelolo reopened T169006: Correctly redirect in Parsoid /transform/wikitext/to/lint endpoint, a subtask of T164006: Suggestion: API for fetching lint errors for a specific revision, as Open.
Thu, Feb 22, 1:29 PM · Services (doing), RESTBase-API, Parsing-Team, MediaWiki-extensions-Linter
Pchelolo edited projects for T164006: Suggestion: API for fetching lint errors for a specific revision, added: Services (doing); removed Services (blocked).

Ok since the Parsoid issue was fixed I can do a quick RESTBase patch for this. Do we still need it?

Thu, Feb 22, 12:55 PM · Services (doing), RESTBase-API, Parsing-Team, MediaWiki-extensions-Linter

Wed, Feb 21

Pchelolo added a comment to T177219: [Spike 3hr] Determine how to store PDFs in order to get file size.

Since we're exposing the PDF service via RESTBase we can add temporary storage for the rendered articles in Cassandra. RESTBase supports HEAD requests and emits the Content-Length header, so the client can issue a head request, the render will be stored in Cassandra for a little while and the second GET request will fetch the PDF without waiting for the service to render the article again.

Wed, Feb 21, 10:20 PM · Readers-Web-Kanbanana-Board, Spike, Readers-Web-Backlog, Proton, New-Readers
Pchelolo added a comment to T181636: Content service incorrectly reports article as "deleted".

@Pchelolo Do you think this could be the case of T184556, as well?

Wed, Feb 21, 10:00 PM · Services (done), Reading-Infrastructure-Team-Backlog
Pchelolo closed T181636: Content service incorrectly reports article as "deleted" as Resolved.

The above PR has been deployed and I've tested that now no-cache requests for HTML update the deleted state of the page. I've checked on the cases listed here, all the rest of the cases will gradually get cleaned up as the articles are edited and rerendered. Resolving.

Wed, Feb 21, 9:25 PM · Services (done), Reading-Infrastructure-Team-Backlog
Pchelolo added a comment to T181636: Content service incorrectly reports article as "deleted".

This has to fix it: https://github.com/wikimedia/restbase/pull/958

Wed, Feb 21, 7:45 PM · Services (done), Reading-Infrastructure-Team-Backlog
Pchelolo closed T187123: vagrant git-update breaks restbase as Resolved.

Ok, the patch was merged, should work fine now. Resolving.

Wed, Feb 21, 6:40 PM · Services (done), RESTBase, MediaWiki-Vagrant
Pchelolo closed T158437: Change ORES rules to send all events to new "/precache" endpoint as Resolved.

Ok, after the second deploy everything seems to be fine. Resolving. Again.

Wed, Feb 21, 6:26 PM · Services (done), Scoring-platform-team (Current), ORES, ChangeProp
Pchelolo closed T158437: Change ORES rules to send all events to new "/precache" endpoint as Resolved.

The change has been deployed and seems to work fine. Resolving.

Wed, Feb 21, 4:43 PM · Services (done), Scoring-platform-team (Current), ORES, ChangeProp
Pchelolo closed T158437: Change ORES rules to send all events to new "/precache" endpoint, a subtask of T162892: Deploy ORES mid-April, as Resolved.
Wed, Feb 21, 4:43 PM · ORES, Scoring-platform-team (Current)

Tue, Feb 20

Pchelolo moved T187821: Choose a server for the chromium-render service from Backlog to watching on the Services board.
Tue, Feb 20, 9:49 PM · Readers-Web-Backlog, Services (watching), Proton, Operations
Pchelolo created T187815: RESTBase deployment fails in beta cluster.
Tue, Feb 20, 5:40 PM · Release-Engineering-Team (Kanban), Scap, Services (watching)
Pchelolo added a comment to T186338: The production server configuration is outdated.

@pmiazga the configs in the source repo work for example/dev/test/documentation purposes, so it's up to you - technically if you don't need it, you can delete it.

Tue, Feb 20, 3:54 PM · Patch-For-Review, Readers-Web-Kanbanana-Board, Proton, Readers-Web-Backlog
Pchelolo edited projects for T181636: Content service incorrectly reports article as "deleted", added: Services (doing); removed Services.

This is indeed a RESTBase issue. A bit of background context:

Tue, Feb 20, 3:27 PM · Services (done), Reading-Infrastructure-Team-Backlog
Pchelolo added a comment to T187343: Changeprop should be able to read multiple config files.

The only catch is that @bd808 gave me a strong warning to not create this kind of dependency between modules, since compilation order during dependency resolution is hard to control, if I understood correctly. I'd like to get his seal of approval before continuing with the suggested patch...

Tue, Feb 20, 2:29 PM · Services (designing), service-runner, ChangeProp

Fri, Feb 16

Pchelolo added a comment to T168073: Switch ORES to dedicated cluster.

Ok, so found the issue, the LVS interface on scb is still registered, so on scb ores.svc.eqiad.wmnet still resolves locally: inet 10.2.2.10/32 scope global lo:LVS valid_lft forever preferred_lft forever

Fri, Feb 16, 8:38 PM · Scoring-platform-team (Current), Services (watching), ORES
Pchelolo added a project to T168073: Switch ORES to dedicated cluster: Services (watching).
Fri, Feb 16, 8:11 PM · Scoring-platform-team (Current), Services (watching), ORES
Pchelolo added a comment to T168073: Switch ORES to dedicated cluster.

I've been investigating some other SCB issue related to memory usage and noticed that for some reason ORES on scb1002 is still receiving precache requests on scb1002 from ChangeProp ( See grafana )

Fri, Feb 16, 8:10 PM · Scoring-platform-team (Current), Services (watching), ORES
Pchelolo edited projects for T179684: ChangeProp workers die if they can't connect to redis, added: Services (doing); removed Services (done).

Right now it's problematic to dig the logs because it's hard to filter all the logs by worker id to understand the sequence of events happening with a particular worker. I will continue investigation when we deploy a better logging in service-runner

Fri, Feb 16, 7:29 PM · Services (doing), Analytics, EventBus
Pchelolo added a comment to T187123: vagrant git-update breaks restbase.

Somehow gerrit bot didn't add a comment about the patch I've made for this. Here it is https://gerrit.wikimedia.org/r/#/c/410222/

Fri, Feb 16, 7:23 PM · Services (done), RESTBase, MediaWiki-Vagrant
Pchelolo renamed T187241: Add page-related topics to EventStreams from Add page delete/undelete and prop changes topics to EventStreams to Add page-related topics to EventStreams.
Fri, Feb 16, 7:20 PM · Analytics-Kanban, Patch-For-Review, Services (watching), Cloud-VPS, EventBus
Pchelolo edited projects for T187343: Changeprop should be able to read multiple config files, added: Services (designing); removed Services.

@awight I don't think we need multiple config.yaml files, because they're generic for all the services and it's not entirely clear how to merge them? Recursively merge all the objects in several configs? That could be very error-prone and can easily get not understandable.

Fri, Feb 16, 6:10 PM · Services (designing), service-runner, ChangeProp
Pchelolo moved T187418: Enable multiple topics in EventStreams URL from Backlog to watching on the Services board.
Fri, Feb 16, 5:51 PM · Services (watching), EventBus, Analytics
Pchelolo reopened T179684: ChangeProp workers die if they can't connect to redis as "Open".
Fri, Feb 16, 4:14 PM · Services (doing), Analytics, EventBus
Pchelolo triaged T179684: ChangeProp workers die if they can't connect to redis as High priority.

This happened again, so reopening. A couple of notes:

Fri, Feb 16, 4:14 PM · Services (doing), Analytics, EventBus
Pchelolo created T187554: Investigate why disk usage on Kafka nodes is 2 times lower in codfw.
Fri, Feb 16, 2:47 PM · Analytics-Kanban, Services (later), EventBus
Pchelolo added a comment to T187296: Increase kafka event retention to 14 or 21 days.
Fri, Feb 16, 2:40 PM · Analytics-Kanban, Wikimedia-Stream, Discovery, Wikidata, Wikidata-Query-Service
Pchelolo added a comment to T187296: Increase kafka event retention to 14 or 21 days.

Currently on kafka-main machines the disk utiliation is really low, so I think we can easily do it without kafka-jumbo.

Fri, Feb 16, 2:39 PM · Analytics-Kanban, Wikimedia-Stream, Discovery, Wikidata, Wikidata-Query-Service
Pchelolo added a comment to T180037: [Spike] Can the new render service run on Debian Stretch?.

heh, I thought about the building process, but of course, locally you can just use a stretch container with the packages fetched from nodesource.com (this is only important if your dependencies use any sort of shared library, which is sometimes the case).

Fri, Feb 16, 1:48 PM · Readers-Web-Kanbanana-Board, Proton, Spike, Readers-Web-Backlog
Pchelolo added a comment to T158437: Change ORES rules to send all events to new "/precache" endpoint.

The config patch above smoke tests correctly under mw-vagrant.

Fri, Feb 16, 12:59 PM · Services (done), Scoring-platform-team (Current), ORES, ChangeProp

Tue, Feb 13

Pchelolo moved T187241: Add page-related topics to EventStreams from Backlog to watching on the Services board.
Tue, Feb 13, 8:12 PM · Analytics-Kanban, Patch-For-Review, Services (watching), Cloud-VPS, EventBus
Pchelolo added a comment to T187241: Add page-related topics to EventStreams.

Oh, ok, I've misread EventStreams for EventBus :)

Tue, Feb 13, 8:01 PM · Analytics-Kanban, Patch-For-Review, Services (watching), Cloud-VPS, EventBus
Pchelolo added a comment to T187241: Add page-related topics to EventStreams.

@Smalyshev But we already have topics and schemas for these events, see https://github.com/wikimedia/mediawiki-event-schemas/blob/master/config/eventbus-topics.yaml#L17

Tue, Feb 13, 7:59 PM · Analytics-Kanban, Patch-For-Review, Services (watching), Cloud-VPS, EventBus
Pchelolo created T187235: Parsoid crashing with cancel after 0 retries! message.
Tue, Feb 13, 7:14 PM · Parsoid, Services (watching)
Pchelolo closed T176126: Update node-rdkafka version to v2.x as Resolved.

Agreed, that's what I've been thinking as well. So, resolving the ticket!

Tue, Feb 13, 6:02 PM · Services (doing), Analytics, EventBus, Reading-Infrastructure-Team-Backlog, ChangeProp
Pchelolo added a comment to T187123: vagrant git-update breaks restbase.

Hm... Make sense. I'll just revert that patch

Tue, Feb 13, 6:00 PM · Services (done), RESTBase, MediaWiki-Vagrant
Pchelolo added a comment to T187123: vagrant git-update breaks restbase.

Hm. Actually, the problem here is that the restbase-mod-table-sqlite is a dev dependency in RESTBase and git-update only installs production dependencies starting from https://gerrit.wikimedia.org/r/#/c/365076/

Tue, Feb 13, 5:42 PM · Services (done), RESTBase, MediaWiki-Vagrant
Pchelolo added a comment to T176126: Update node-rdkafka version to v2.x.

Vagrant is a bit specific. By default the node-rdkafka just clones librdkafka sources into a subfolder and then build librdkafka package from scratch, statically linking to it, so technically we don't even need to install librdkafka-dev package in Vagrant - everything works out of the box.

Tue, Feb 13, 5:29 PM · Services (doing), Analytics, EventBus, Reading-Infrastructure-Team-Backlog, ChangeProp

Mon, Feb 12

Pchelolo claimed T187123: vagrant git-update breaks restbase.
Mon, Feb 12, 10:16 PM · Services (done), RESTBase, MediaWiki-Vagrant
Pchelolo closed T179684: ChangeProp workers die if they can't connect to redis as Resolved.

After the deployment of the https://github.com/wikimedia/change-propagation/pull/229 the death spiral bug is not observed anymore, so I'm resolving the ticket.

Mon, Feb 12, 9:35 PM · Services (doing), Analytics, EventBus

Fri, Feb 9

Pchelolo added a comment to T186764: refreshLinks/jobqueue issues in wmf.20 causing MW-reported replag.

I know @Ottomata and @Pchelolo did some work earlier today on jobs/change-prop so CCing them

Fri, Feb 9, 12:40 AM · MW-1.31-release-notes (WMF-deploy-2018-02-06 (1.31.0-wmf.20)), Patch-For-Review, MediaWiki-JobQueue, MediaWiki-Database, Regression

Thu, Feb 8

Pchelolo added a comment to T181680: Allow rendering PDFs for mobile.

is it possible to host RESTBase on mobile domains.

Thu, Feb 8, 7:42 PM · Patch-For-Review, Services (watching), Readers-Web-Kanbanana-Board, Readers-Web-Backlog, Proton
Pchelolo closed T186761: Normalize messages in service-runner as Resolved.

Merged and published. We'll see new log messages as we deploy services

Thu, Feb 8, 1:05 AM · Services (done), service-runner
Pchelolo moved T186761: Normalize messages in service-runner from designing to done on the Services board.
Thu, Feb 8, 12:56 AM · Services (done), service-runner
Pchelolo moved T186761: Normalize messages in service-runner from later to designing on the Services board.
Thu, Feb 8, 12:56 AM · Services (done), service-runner
Pchelolo moved T186748: New service request: chromium-render/deploy from Backlog to watching on the Services board.
Thu, Feb 8, 12:56 AM · Patch-For-Review, Services (watching), Service-deployment-requests, Readers-Web-Kanbanana-Board, Readers-Web-Backlog, Proton, Operations, Electron-PDFs
Pchelolo moved T181680: Allow rendering PDFs for mobile from Backlog to watching on the Services board.
Thu, Feb 8, 12:56 AM · Patch-For-Review, Services (watching), Readers-Web-Kanbanana-Board, Readers-Web-Backlog, Proton
Pchelolo edited projects for T184534: Cached page previews not shown when refreshed, added: Services (watching); removed Services.

I've done a little bit more research here and Varnish docs actually confirm that age header can effectively disallow the client-side caching, see http://book.varnish-software.com/3.0/HTTP.html#age

Thu, Feb 8, 12:52 AM · Services (watching), Traffic, Operations, Readers-Web-Backlog (Tracking), RESTBase, Page-Previews
Pchelolo added a comment to T186761: Normalize messages in service-runner.

Since it was really easy and I myself was irritated by this I went ahead and implemented it https://github.com/wikimedia/service-runner/pull/180

Thu, Feb 8, 12:36 AM · Services (done), service-runner

Wed, Feb 7

Pchelolo edited projects for T186720: Etag is not returned for mobile-sections-lead or mobile-sections-remaining if a revision parameter is present, added: Services (done); removed Services.

The PR's been merged so this is basically done. Will close as soon as it's deployed.

Wed, Feb 7, 11:01 PM · Services (done), Reading-Infrastructure-Team-Backlog, Page Content Service, Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo edited projects for T186761: Normalize messages in service-runner, added: Services (later); removed Services.

We can take the PID out of the log message and put it into a separate field in the log

Wed, Feb 7, 10:59 PM · Services (done), service-runner
Pchelolo closed T162372: Trending service should consume events from both DCs as Declined.

Since the trending service has been put out of production this is not relevant any more T180384

Wed, Feb 7, 10:56 PM · Services (done), Reading-Infrastructure-Team-Backlog, Trending-Service
Pchelolo committed rMSCD5fdc03a55b26: Update change-propagation to 7bfcdd3 (authored by Pchelolo).
Update change-propagation to 7bfcdd3
Wed, Feb 7, 8:16 PM

Tue, Feb 6

Pchelolo added a comment to T186261: Support reliable delayed job execution in ChangeProp.

A very basic version of this implemented at https://github.com/wikimedia/change-propagation/pull/233

Tue, Feb 6, 11:07 PM · Services (doing), ChangeProp, Analytics, EventBus
Pchelolo closed T184751: Benchmark the new page summary API as Resolved.

@Mholloway I believe we can close this one.

Tue, Feb 6, 10:09 PM · Services (done), Performance-Team (Radar), Page Content Service, Reading-Infrastructure-Team-Backlog (Kanban), Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo closed T184751: Benchmark the new page summary API, a subtask of T177431: Develop a Summary JSON API, as Resolved.
Tue, Feb 6, 10:08 PM · Patch-For-Review, Page Content Service, Reading-Infrastructure-Team-Backlog (Kanban), Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo added a comment to T176126: Update node-rdkafka version to v2.x.

Yesterday during the deployment of JobQueue CP instance we've had a small incident because of the disparity between the librdkafka installed on the machines on SCB and node-rdkafka. So we've had to rever a commit in deploy repo that updated to node-rdkafka v2 https://gerrit.wikimedia.org/r/#/c/408441/

Tue, Feb 6, 7:19 PM · Services (doing), Analytics, EventBus, Reading-Infrastructure-Team-Backlog, ChangeProp
Pchelolo added a comment to T185052: Migrate RefreshLinks job to kafka.

I have rerun the script for 5 million events and the results are fairly similar to what was observed, so the plan is valid.

Tue, Feb 6, 5:24 PM · Patch-For-Review, Analytics, Services (next), EventBus, MediaWiki-JobQueue

Mon, Feb 5

Pchelolo closed T182023: Migrate htmlCacheUpdate job to Kafka as Resolved.

Seems like the migration is complete with no issues. Resolving

Mon, Feb 5, 11:26 PM · Patch-For-Review, Services (done), MediaWiki-JobQueue, ChangeProp, EventBus, Operations, User-Joe, User-Elukey, Analytics
Pchelolo closed T182023: Migrate htmlCacheUpdate job to Kafka, a subtask of T183744: FY17/18 Q3 Program 8 Services Goal: Migrate two high-traffic jobs over to EventBus, as Resolved.
Mon, Feb 5, 11:26 PM · EventBus, MediaWiki-JobQueue, Services (doing), Goal, Analytics

Sat, Feb 3

Pchelolo added a project to T186371: Redesign revision-related event schemas for MCR: Multi-Content-Revisions.
Sat, Feb 3, 12:04 AM · Multi-Content-Revisions, Services (doing), EventBus, Analytics
Pchelolo created T186371: Redesign revision-related event schemas for MCR.
Sat, Feb 3, 12:04 AM · Multi-Content-Revisions, Services (doing), EventBus, Analytics

Fri, Feb 2

Pchelolo added a comment to T184556: Page Preview permanently broken after title is renamed .

Hm, I can no longer reproduce this issue nor can I find anything suspicious in Cassandra storage for these pages. I'm not sure how to debug this so if nobody objects I'm inclined to close this task as no action can be taken.

Fri, Feb 2, 9:12 PM · Readers-Web-Backlog (Tracking), Services, Reading-Infrastructure-Team-Backlog, Page Content Service, Page-Previews, RESTBase
Pchelolo added a comment to T184534: Cached page previews not shown when refreshed.
Fri, Feb 2, 8:46 PM · Services (watching), Traffic, Operations, Readers-Web-Backlog (Tracking), RESTBase, Page-Previews
Pchelolo added a comment to T184534: Cached page previews not shown when refreshed.

Yep, just confirmed that locally - whenever the age header is larget the max-age header local browser HTTP caching is not happening. This is weird cause the age header is defined for the proxy caches, not the client-side cache.

Fri, Feb 2, 8:23 PM · Services (watching), Traffic, Operations, Readers-Web-Backlog (Tracking), RESTBase, Page-Previews
Pchelolo added a comment to T184534: Cached page previews not shown when refreshed.

I think I might have found a correlation here - we (Varnish actually) set the age header of the response to the number of seconds the entry has been in Varnish. Whenever the age is smaller the max-age the browser caches the response. Whenever the age is larger than max-age the browser doesn't use the cached response even though that's not the age from the disc cache.

Fri, Feb 2, 8:09 PM · Services (watching), Traffic, Operations, Readers-Web-Backlog (Tracking), RESTBase, Page-Previews
Pchelolo added a comment to T184534: Cached page previews not shown when refreshed.

hm.. I was able to reproduce this in Safari as well. I'm not sure what could be special about these articles, we set the exact same cache-control headers for all the responses, so I don't think this is a RESTBase issue. Will dig a bit more

Fri, Feb 2, 6:39 PM · Services (watching), Traffic, Operations, Readers-Web-Backlog (Tracking), RESTBase, Page-Previews
Pchelolo added a comment to T184751: Benchmark the new page summary API.

@bearND sure!

Fri, Feb 2, 12:33 AM · Services (done), Performance-Team (Radar), Page Content Service, Reading-Infrastructure-Team-Backlog (Kanban), Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo added a project to T184751: Benchmark the new page summary API: Services (watching).
Fri, Feb 2, 12:09 AM · Services (done), Performance-Team (Radar), Page Content Service, Reading-Infrastructure-Team-Backlog (Kanban), Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo added a comment to T184751: Benchmark the new page summary API.

Here's your numbers @Mholloway. Measured from within production environment from xenon.eqiad.wmnet:

Fri, Feb 2, 12:03 AM · Services (done), Performance-Team (Radar), Page Content Service, Reading-Infrastructure-Team-Backlog (Kanban), Reading Epics (Platform JS CSS and HTML consolidation)

Thu, Feb 1

Pchelolo created T186261: Support reliable delayed job execution in ChangeProp.
Thu, Feb 1, 8:18 PM · Services (doing), ChangeProp, Analytics, EventBus
Pchelolo added a comment to T179875: Update RESTBase to get summary content from MCS Summary 1.3 endpoint when development is complete.

No that's me who should be blamed it's taken so long, we've had an offsite and didn't do any real work.

Thu, Feb 1, 6:11 PM · Patch-For-Review, User-mobrovac, Services (doing), RESTBase-API, RESTBase, Page Content Service, Reading-Infrastructure-Team-Backlog (Kanban), Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo edited projects for T186228: Use RevisionRecordInserted hook for EventBus revision-create records, added: Services (doing); removed Services.
Thu, Feb 1, 5:34 PM · Patch-For-Review, Services (doing), Analytics, EventBus
Pchelolo claimed T186228: Use RevisionRecordInserted hook for EventBus revision-create records.
Thu, Feb 1, 5:34 PM · Patch-For-Review, Services (doing), Analytics, EventBus

Wed, Jan 31

Pchelolo added a comment to T184534: Cached page previews not shown when refreshed.

@Jdlrobson We're on an offsite so if it can wait till tomorrow I'll have a look tomorrow.

Wed, Jan 31, 5:52 PM · Services (watching), Traffic, Operations, Readers-Web-Backlog (Tracking), RESTBase, Page-Previews
Pchelolo updated the task description for T175210: Select candidate jobs for transferring to the new infrastucture.
Wed, Jan 31, 12:11 AM · Patch-For-Review, Services (doing), MediaWiki-JobQueue, ChangeProp, Analytics, EventBus, Operations, User-Joe, User-Elukey

Tue, Jan 30

Pchelolo updated subscribers of T185139: Popups should respect user's language variant preferences.

This is likely to impact Chinese as well and other languages with variants.
Services: Can we serve language content variants in RESTBase? Is there a ticket open relating to that if not.

Tue, Jan 30, 5:40 PM · Readers-Web-Backlog (Tracking), Services (blocked), Page-Previews

Wed, Jan 24

Pchelolo closed T182953: A page required a "dummy" edit to trigger a preview for it! as Resolved.

The fix has been deployed. Resolving.

Wed, Jan 24, 11:10 PM · Services (done), RESTBase, Readers-Web-Backlog (Tracking), Page-Previews
Pchelolo added a comment to T179684: ChangeProp workers die if they can't connect to redis.

I believe I've identified the cause of this and it's not really related to Redis. Whenever the Kafka consumer was disconnected from Kafka, we were trying to reconnect back in a tight loop that was causing workers to die. I believe the following patch will fix the issue https://github.com/wikimedia/change-propagation/pull/229

Wed, Jan 24, 7:46 PM · Services (doing), Analytics, EventBus
Pchelolo committed rMSCDe18b5b78c8fa: Only rerender summary after mobile-sections were rerendered. (authored by Pchelolo).
Only rerender summary after mobile-sections were rerendered.
Wed, Jan 24, 7:29 PM

Jan 20 2018

Pchelolo added a comment to T184753: Use cached page leads when creating page summaries to reduce MCS load.

@Jdlrobson so is my understanding correct that after we switch to fetching mobile-section-lead for summaries from RESTBase we will essentially enable pregeneration for mobile content for non-wikipedia projects since summaries will be pregenerated for them and as they require mobile content it will also be pregenerated? If that's correct, we can just get rid of the special-case for wikipedias for mobile content. I don't think the additional load will be too high, vast majority of edits are done on wikipedias anyway..

Jan 20 2018, 12:02 AM · Reading-Infrastructure-Team-Backlog, Patch-For-Review, Services (blocked), Page Content Service, Reading Epics (Platform JS CSS and HTML consolidation)

Jan 19 2018

Pchelolo added a comment to T184753: Use cached page leads when creating page summaries to reduce MCS load.

Page summaries are only available on Wikipedia.

Jan 19 2018, 11:43 PM · Reading-Infrastructure-Team-Backlog, Patch-For-Review, Services (blocked), Page Content Service, Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo added a comment to T184753: Use cached page leads when creating page summaries to reduce MCS load.

While working on this I've noticed that we only prerender MCS content for wikipedia domains, not for other projects, cause the apps only use it for wikipedia, however summary is pre-rendered for all of the projects.

Jan 19 2018, 10:23 PM · Reading-Infrastructure-Team-Backlog, Patch-For-Review, Services (blocked), Page Content Service, Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo added a comment to T184753: Use cached page leads when creating page summaries to reduce MCS load.

Pull request for change-propagation: https://github.com/wikimedia/change-propagation/pull/228

Jan 19 2018, 9:48 PM · Reading-Infrastructure-Team-Backlog, Patch-For-Review, Services (blocked), Page Content Service, Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo edited projects for T184753: Use cached page leads when creating page summaries to reduce MCS load, added: Services (doing); removed Services.
Jan 19 2018, 8:35 PM · Reading-Infrastructure-Team-Backlog, Patch-For-Review, Services (blocked), Page Content Service, Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo added a comment to T184753: Use cached page leads when creating page summaries to reduce MCS load.

@bearND You shouldn't emit any ChangeProp events in MCS, it will be done in RESTBase, let's try to avoid making MCS contact EventBus.

Jan 19 2018, 8:34 PM · Reading-Infrastructure-Team-Backlog, Patch-For-Review, Services (blocked), Page Content Service, Reading Epics (Platform JS CSS and HTML consolidation)
Pchelolo added a comment to T182840: kartotherian package repo fails to build.

I've created a PR for service-runner that will allow specifying the npm version in the deploy section of the package.json file. At least with that, a hack looks a little bit less hacky. Also, it removes the custom installed npm so that we don't really deploy npm to production.

Jan 19 2018, 7:27 PM · Services (next), service-runner, Maps-Sprint, Maps (Maps-data)
Pchelolo added a comment to T182840: kartotherian package repo fails to build.

My guess is something was failing in the build and leaving a nonempty directory, which could then not be removed. But honestly, I don't know what I'm talking about.

Jan 19 2018, 6:04 PM · Services (next), service-runner, Maps-Sprint, Maps (Maps-data)
Pchelolo added a comment to T182840: kartotherian package repo fails to build.

Ok, I've come up with a dirty workaround. We need to downgrade npm to version 2 in order to make it work.

Jan 19 2018, 12:56 AM · Services (next), service-runner, Maps-Sprint, Maps (Maps-data)
Pchelolo added a comment to T182840: kartotherian package repo fails to build.

Another piece of info: after unsuccessfully poking around different ideas I've tried to build it with node 8 and it did build successfully. That indicates that the bug is in a particular version of npm we're using.

Jan 19 2018, 12:34 AM · Services (next), service-runner, Maps-Sprint, Maps (Maps-data)

Jan 18 2018

Pchelolo added a comment to T182840: kartotherian package repo fails to build.

My initial theory that this is somehow related to the docker storage driver was incorrect. I've checked overlay, overlay2 and vfs drivers with no results - all end up the same result - build fails.

Jan 18 2018, 10:42 PM · Services (next), service-runner, Maps-Sprint, Maps (Maps-data)
Pchelolo added a comment to T182840: kartotherian package repo fails to build.

I've finally managed to reproduce the issue with an Ubuntu VM and the latest version of docker-ce (v17.12). However, I still have no idea what's happening here.

Jan 18 2018, 7:30 PM · Services (next), service-runner, Maps-Sprint, Maps (Maps-data)
Pchelolo committed rMSCPf82d46b57221: Update node-rdkafka driver to version 2 (authored by Pchelolo).
Update node-rdkafka driver to version 2
Jan 18 2018, 1:23 AM
Pchelolo created T185170: Enable EventBus on all wikis.
Jan 18 2018, 12:16 AM · MediaWiki-JobQueue, EventBus, Analytics, Services (later)

Jan 17 2018

Pchelolo committed rMSCDd64f43a9d132: Update change-propagation to 214d48e (authored by Pchelolo).
Update change-propagation to 214d48e
Jan 17 2018, 11:24 PM
Pchelolo updated subscribers of T184858: Global user pages cannot be viewed in rest api (404).

We've chatted with @ssastry as well about this and this will be needed when/if we start using Parsoid HTML for page views, so no immediate necessity to fix this.

Jan 17 2018, 10:51 PM · Parsoid, Services (later), GlobalUserPage, RESTBase-architecture, RESTBase-API, RESTBase