GWicke (Gabriel Wicke)
User

Projects (34)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Monday

  • Clear sailing ahead.

User Details

User Since
Oct 31 2014, 3:23 AM (151 w, 1 d)
Availability
Available
LDAP User
GWicke
MediaWiki User
Unknown

Recent Activity

Yesterday

GWicke updated subscribers of T176520: Pageimage property (and possibly other page properties) not updated reliably after reverts.

@Tgr added this on a related mail thread:

Fri, Sep 22, 9:16 PM · PageImages, Services (watching)
GWicke updated the task description for T176520: Pageimage property (and possibly other page properties) not updated reliably after reverts.
Fri, Sep 22, 9:13 PM · PageImages, Services (watching)
GWicke updated subscribers of T176520: Pageimage property (and possibly other page properties) not updated reliably after reverts.
Fri, Sep 22, 9:10 PM · PageImages, Services (watching)
GWicke added projects to T176520: Pageimage property (and possibly other page properties) not updated reliably after reverts: Services (watching), PageImages.
Fri, Sep 22, 8:55 PM · PageImages, Services (watching)
GWicke created T176520: Pageimage property (and possibly other page properties) not updated reliably after reverts.
Fri, Sep 22, 8:55 PM · PageImages, Services (watching)

Wed, Sep 20

GWicke updated subscribers of T173821: Investigate exposing content styles needed via API vs as HTML tags.
Wed, Sep 20, 6:23 PM · RESTBase-API, Services (watching), Reading-Infrastructure-Team-Backlog (Kanban), Page Content Service
GWicke added a comment to T173821: Investigate exposing content styles needed via API vs as HTML tags.

If I recall correctly, ResourceLoader client code on desktop already looks at a list of modules needed in a given page, checks client side caches, and fetches the remaining modules from the RL API (in a single call), and caches those modules separately in localstorage. Given that this discussion is making no reference to this, I am getting the impression that this understanding might be wrong. Could you clarify?

Wed, Sep 20, 6:22 PM · RESTBase-API, Services (watching), Reading-Infrastructure-Team-Backlog (Kanban), Page Content Service
GWicke added a comment to T173821: Investigate exposing content styles needed via API vs as HTML tags.

@Fjalapeno, that comment touches on 1), but as I said to me it looks like the API focused discussion has moved to 2). Either way, I am not sure we need a new API for either 1) or 2).

Wed, Sep 20, 5:50 PM · RESTBase-API, Services (watching), Reading-Infrastructure-Team-Backlog (Kanban), Page Content Service
GWicke added a comment to T123854: Set up action API latency / error rate metrics & alerts.

FTR, this is the graph with the alert I mentioned: https://grafana.wikimedia.org/dashboard/db/restbase?panelId=12&fullscreen&orgId=1

Wed, Sep 20, 4:12 PM · Services (watching), MediaWiki-API, Traffic, Operations, monitoring

Tue, Sep 19

GWicke added a comment to T172224: Consider partitioning local_group_wikipedia even more.

At today's team sync we agreed with @Pchelolo's proposal:

Tue, Sep 19, 8:06 PM · User-mobrovac, Services (done), Cassandra, RESTBase
GWicke added a comment to T175210: Select candidate jobs for transferring to the new infrastucture.

I honestly don't have a strong preference between the other "hearted" tasks. Given that all of them are fairly low volume, would it make sense to just deploy all of the hearted ones in the next wave?

Tue, Sep 19, 6:10 PM · Services (doing), MediaWiki-JobQueue, ChangeProp, Analytics, EventBus, Operations, User-Joe, User-Elukey

Mon, Sep 18

GWicke added a comment to T173821: Investigate exposing content styles needed via API vs as HTML tags.

It sounds like there are two separate questions:

Mon, Sep 18, 8:01 PM · RESTBase-API, Services (watching), Reading-Infrastructure-Team-Backlog (Kanban), Page Content Service
GWicke added a comment to T175882: Requesting access to scb* and pdfrender-admin for tgr.

I strongly support @Tgr's access request as well.

Mon, Sep 18, 3:23 PM · Operations, Ops-Access-Requests
GWicke added a comment to T175957: Options for implementing JobQueue statistics methods.

Added the "fetch metrics from graphite / prometheus" option.

Mon, Sep 18, 2:54 PM · Services (designing), MediaWiki-JobQueue, User-mobrovac, Analytics, ChangeProp, EventBus
GWicke updated the task description for T175957: Options for implementing JobQueue statistics methods.
Mon, Sep 18, 2:53 PM · Services (designing), MediaWiki-JobQueue, User-mobrovac, Analytics, ChangeProp, EventBus

Thu, Sep 14

GitHub <noreply@github.com> committed rMSCP1a5e44f940a9: Merge pull request #205 from Pchelolo/job_tests (authored by GWicke).
Merge pull request #205 from Pchelolo/job_tests
Thu, Sep 14, 7:56 PM
GitHub <noreply@github.com> committed rMSCPe4d63f6de369: Merge pull request #207 from Pchelolo/more_frequent_stats (authored by GWicke).
Merge pull request #207 from Pchelolo/more_frequent_stats
Thu, Sep 14, 7:56 PM
GWicke added a comment to T175316: Very large jobs posted by Wikidata.

Looks like adding the JSON_UNESCAPED_UNICODE flag should do it: http://php.net/manual/en/function.json-encode.php

Thu, Sep 14, 7:14 PM · MW-1.30-release-notes (WMF-deploy-2017-09-19 (1.30.0-wmf.19)), Patch-For-Review, EventBus, Analytics, Services (designing), Wikidata

Wed, Sep 13

GWicke added a comment to T175210: Select candidate jobs for transferring to the new infrastucture.

Given the useful information we have in this task, I am proposing to widen the scope beyond the first job, towards generally coordinating the order of migrating individual jobs. @mobrovac, does that sound reasonable to you?

Wed, Sep 13, 5:57 PM · Services (doing), MediaWiki-JobQueue, ChangeProp, Analytics, EventBus, Operations, User-Joe, User-Elukey
GWicke lowered the priority of T175800: Allow easy tuning of the jobqueue concurrency. from Normal to Low.

We briefly discussed this during today's sync meeting. While there are ways to set up targeted processing priorities for specific jobs (by wiki, type, or other criteria), we realized that there will likely be less of a need for this in the new setup. The Redis job queue divides processing throughput evenly between projects. This makes it relatively likely for individual projects to accumulate large backlogs, which would then need manual intervention (re-prioritization) to address.

Wed, Sep 13, 5:46 PM · Services (designing), MediaWiki-JobQueue, User-mobrovac, Analytics, ChangeProp, EventBus
GWicke added a comment to T175316: Very large jobs posted by Wikidata.

Raised priority, as this is a) blocking the migration to the Kafka job queue backend (T157088), and b) is likely already causing performance and possibly reliability issues in the current job queue.

Wed, Sep 13, 5:30 PM · MW-1.30-release-notes (WMF-deploy-2017-09-19 (1.30.0-wmf.19)), Patch-For-Review, EventBus, Analytics, Services (designing), Wikidata
GWicke raised the priority of T175316: Very large jobs posted by Wikidata from Normal to High.
Wed, Sep 13, 5:11 PM · MW-1.30-release-notes (WMF-deploy-2017-09-19 (1.30.0-wmf.19)), Patch-For-Review, EventBus, Analytics, Services (designing), Wikidata
GitHub <noreply@github.com> committed rMSCPbabbc777cf62: Merge pull request #204 from Pchelolo/redis_key_prefix (authored by GWicke).
Merge pull request #204 from Pchelolo/redis_key_prefix
Wed, Sep 13, 12:11 PM
GitHub <noreply@github.com> committed rMSCP22cda1868f4f: Merge pull request #201 from Pchelolo/node_v8 (authored by GWicke).
Merge pull request #201 from Pchelolo/node_v8
Wed, Sep 13, 12:11 PM

Tue, Sep 12

GWicke updated subscribers of T174993: Vandalism in "In the news" articles persisting in the app' ?.

As far as I can tell, the page image(s) are handled as part of deferred linksUpdate processing. This means that the updates would be executed after the main web request, but on the same PHP thread that handled the original edit request.

Tue, Sep 12, 6:49 PM · Reading-Infrastructure-Team-Backlog, Services (watching), Mobile, Wikipedia-iOS-App-Backlog, Wikipedia-Android-App-Backlog, iOS-app-Bugs, Android-app-Bugs
GWicke added a comment to T172224: Consider partitioning local_group_wikipedia even more.

Considering the scalability limits of Cassandra's schema synchronization we see in production, I think it would be good to reduce the number of storage groups more aggressively. Perhaps something like this?

Tue, Sep 12, 5:20 PM · User-mobrovac, Services (done), Cassandra, RESTBase

Mon, Sep 11

GWicke closed T170319: End of August milestone: Begin migrating job queue processing to multi-DC enabled eventbus infrastructure. as Resolved.

Update from our month-end check-in:

Mon, Sep 11, 10:01 PM · Services (doing), User-Joe, User-Elukey, Goal, Epic
GWicke closed T170319: End of August milestone: Begin migrating job queue processing to multi-DC enabled eventbus infrastructure., a subtask of T169937: Services Q1 2017/18 goal: Begin migrating job queue processing to multi-DC enabled eventbus infrastructure., as Resolved.
Mon, Sep 11, 10:01 PM · Operations, User-Joe, User-Elukey, Goal, Epic, Services (doing)
GWicke created T175637: End of September milestone: Migrate first production use case.
Mon, Sep 11, 9:59 PM · MW-1.30-release-notes (WMF-deploy-2017-09-12_(1.30.0-wmf.18)), Operations, User-Joe, User-Elukey, Goal, Epic, Services (doing)
GWicke added a comment to T175210: Select candidate jobs for transferring to the new infrastucture.

IMHO, updateBetaFeaturesUserCounts is the perfect candidate here. It's very lightweight (one SELECT, one UPDATE), it's idempotent and low-volume.

Mon, Sep 11, 7:16 PM · Services (doing), MediaWiki-JobQueue, ChangeProp, Analytics, EventBus, Operations, User-Joe, User-Elukey
GWicke added a comment to T175305: mobile-sections: Cannot read property 'indexOf' of undefined in markReferenceSections.

@bearND, MediaWiki's section edit feature is implemented without knowledge of a DOM, so <div> wrappers do not suppress edit sections. Example: https://en.wikipedia.org/wiki/User:GWicke/TestSections with source

Mon, Sep 11, 7:12 PM · Patch-For-Review, Mobile-Content-Service, Reading-Infrastructure-Team-Backlog (Kanban)
GWicke added a comment to T174993: Vandalism in "In the news" articles persisting in the app' ?.

I believe it was the pageimages designation for those articles I mentioned above. Not exactly sure what happened on wiki since the revisions have been deleted from public archives (and I don't have the permission to view it).

Mon, Sep 11, 5:51 PM · Reading-Infrastructure-Team-Backlog, Services (watching), Mobile, Wikipedia-iOS-App-Backlog, Wikipedia-Android-App-Backlog, iOS-app-Bugs, Android-app-Bugs
GWicke added a comment to T174993: Vandalism in "In the news" articles persisting in the app' ?.

Just to clarify what exactly happened here: The offending edits were adding an image to the featured page itself, and also nominated that image to be the pageimage?

Mon, Sep 11, 4:59 PM · Reading-Infrastructure-Team-Backlog, Services (watching), Mobile, Wikipedia-iOS-App-Backlog, Wikipedia-Android-App-Backlog, iOS-app-Bugs, Android-app-Bugs
GWicke added a comment to T175281: Separate off ChangePropagation for JobQueue as a new deployment.

Yay! 🎆

Mon, Sep 11, 4:01 PM · Services (done), User-mobrovac, Service-deployment-requests, Operations, MediaWiki-JobQueue, EventBus, ChangeProp, Analytics
GWicke added a comment to T120242: Reliable (atomic) MediaWiki event production.

@Ottomata, from a cursory look at those connectors, it looks like they all aim to capture all SQL updates (update, insert, delete). They don't seem to be targeted at emitting specific semantic events, such as the ones we are interested in for EventBus. This is where the SQL comment idea could help, by letting us essentially embed the events we want to have emitted in the statement, rather than trying to reverse-engineer an event from raw SQL statement(s).

Mon, Sep 11, 3:58 PM · Services (later)
GWicke updated subscribers of T175316: Very large jobs posted by Wikidata.
Mon, Sep 11, 3:49 PM · MW-1.30-release-notes (WMF-deploy-2017-09-19 (1.30.0-wmf.19)), Patch-For-Review, EventBus, Analytics, Services (designing), Wikidata
GWicke updated subscribers of T174982: Sourcemap is incorrect in RESTBase help page.
Mon, Sep 11, 3:40 PM · Services (later), RESTBase-API, RESTBase
GWicke added a comment to T174982: Sourcemap is incorrect in RESTBase help page.

Looking at the three custom changes we did on top of upstream in https://github.com/wikimedia/swagger-ui/commits/master, it seems that the build process we ran after each did not update the source map. However, the gulpfile defineds "dist" to be part of the default task (see https://github.com/wikimedia/swagger-ui/blob/master/gulpfile.js#L188). Perhaps we "just" forgot to check in the updated source maps?

Mon, Sep 11, 3:40 PM · Services (later), RESTBase-API, RESTBase
GWicke added a comment to T175305: mobile-sections: Cannot read property 'indexOf' of undefined in markReferenceSections.

In terms of document structure, the behavior in line two (add section around <div>-wrapped heading) seems to make sense. I think it also matches edit section behavior, which should ignore the <div> completely (as it is not DOM-based).

Mon, Sep 11, 3:13 PM · Patch-For-Review, Mobile-Content-Service, Reading-Infrastructure-Team-Backlog (Kanban)

Fri, Sep 8

GWicke updated the task description for T169116: Support flagged revisions in RESTBase.
Fri, Sep 8, 9:23 PM · RESTBase-API, RESTBase, VisualEditor, Reading-Infrastructure-Team-Backlog, Mobile-Content-Service, Services (designing)
GWicke added a comment to T156829: Remove must-revalidate cache-control header from mobile-sections-lead and mobile-sections-remaining responses.

From a practical perspective, I think the biggest question is how common clients behave these days when must-revalidate is omitted, and the client cache timeout expires. My memory on this is rather foggy, but I *think* in the dark ages behavior in that area was inconsistent, with early IE versions not re-validating even when they were online. If we can verify that all browsers we care about do the right thing (check as if must-revalidate was set when connected), then dropping must-revalidate in the headers would be harmless.

Fri, Sep 8, 6:48 PM · Reading-Infrastructure-Team-Backlog, RESTBase, Mobile-Content-Service, Android-app-Bugs, Wikipedia-Android-App-Backlog
GWicke added a comment to T76541: Somehow(?) load document-level data (e.g. internal lists, meta lists) for the whole document but only the content for a part of the document.

We already support fetching specific HTML sections by ID in the REST API (see https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_html_title), but until consistent <section> wrapping with a sensible granularity & perhaps a predictable section ID for the lead section are implemented in Parsoid (T114072), this is not as useful in practice as it could be.

Fri, Sep 8, 6:36 PM · VisualEditor-MediaWiki, VisualEditor, VisualEditor-Performance
GWicke updated the task description for T169116: Support flagged revisions in RESTBase.
Fri, Sep 8, 6:12 PM · RESTBase-API, RESTBase, VisualEditor, Reading-Infrastructure-Team-Backlog, Mobile-Content-Service, Services (designing)
GWicke updated the task description for T175316: Very large jobs posted by Wikidata.
Fri, Sep 8, 5:07 PM · MW-1.30-release-notes (WMF-deploy-2017-09-19 (1.30.0-wmf.19)), Patch-For-Review, EventBus, Analytics, Services (designing), Wikidata
GWicke added a comment to T175318: Optimize SVGs in data URIs.

This proposed optimization is similar to something I implemented in Parsoid's HTML5 serializer. In that case, we switch between single & double quotes for HTML attributes depending on whether the attribute value contains more single quotes or double quotes. This had a very significant impact on Parsoid HTML size, mainly because it has many JSON values embedded in attributes.

Fri, Sep 8, 3:52 PM · MW-1.31-release-notes (WMF-deploy-2017-09-26 (1.31.0-wmf.1)), MW-1.30-release-notes (WMF-deploy-2017-09-19 (1.30.0-wmf.19)), Front-end-Standards-Group, Performance-Team (Radar), MediaWiki-ResourceLoader
GWicke added a comment to T175316: Very large jobs posted by Wikidata.

@Pchelolo, based on our previous conversation about this I am assuming that the bulk of the task is a very large list of pages. Is this correct?

Fri, Sep 8, 3:43 PM · MW-1.30-release-notes (WMF-deploy-2017-09-19 (1.30.0-wmf.19)), Patch-For-Review, EventBus, Analytics, Services (designing), Wikidata

Thu, Sep 7

GWicke added a comment to T120242: Reliable (atomic) MediaWiki event production.

Facebook actually heavily relies on SQL comments to pass event information to binlog tailer daemons (see the TAO paper). We currently use those SQL comments only to mark the source of a SQL query (PHP function), but could potentially add some annotations that would make it easy to generically extract & export such events into individual Kafka topics.

Thu, Sep 7, 10:37 PM · Services (later)
GWicke added a comment to T175305: mobile-sections: Cannot read property 'indexOf' of undefined in markReferenceSections.

Starting a new section when encountering a new heading of the same level is expected behavior, in line with MediaWiki section edit behavior. When encountering a heading of a higher level (higher number, lower prominence), the sectioning code I wrote in parsoid-utils creates a nested section. This is in line with typical HTML5 section and page outline semantics: https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Using_HTML_sections_and_outlines.

Thu, Sep 7, 10:04 PM · Patch-For-Review, Mobile-Content-Service, Reading-Infrastructure-Team-Backlog (Kanban)
GWicke added a comment to T151934: Check & possibly improve request id handling in changeprop.

Rebased PR now ready at https://github.com/wikimedia/change-propagation/pull/203.

Thu, Sep 7, 8:51 PM · Services (done), ChangeProp
GWicke updated subscribers of T175213: 2017/18 Annual Plan Program 8: Multi-datacenter support, Q2 goals.
Thu, Sep 7, 4:29 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke updated subscribers of T175206: 2017/18 Annual Plan Program 8: Multi-datacenter support.
Thu, Sep 7, 4:29 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke renamed T175206: 2017/18 Annual Plan Program 8: Multi-datacenter support from 2017/18 annual plan program 8: Multi-datacenter support to 2017/18 Annual Plan Program 8: Multi-datacenter support.
Thu, Sep 7, 4:20 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke added a comment to T170120: Standardize on the "default" pod setup.

I don't have strong views on how to scale metrics and log collection. In any case, we have been doing this remotely for a while now (using standard formats like gelf for logs), so whether things are aggregated per pod or more centrally doesn't make a big difference to the services themselves.

Thu, Sep 7, 3:17 PM · User-Joe, Services (watching), Kubernetes, Operations, Goal

Wed, Sep 6

GWicke added a subtask for T175213: 2017/18 Annual Plan Program 8: Multi-datacenter support, Q2 goals: T175212: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure..
Wed, Sep 6, 8:19 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke added a parent task for T175212: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure.: T175213: 2017/18 Annual Plan Program 8: Multi-datacenter support, Q2 goals.
Wed, Sep 6, 8:19 PM · MediaWiki-JobQueue, ChangeProp, EventBus, Analytics, Goal, Services (next)
GWicke updated the task description for T175213: 2017/18 Annual Plan Program 8: Multi-datacenter support, Q2 goals.
Wed, Sep 6, 8:14 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke created T175213: 2017/18 Annual Plan Program 8: Multi-datacenter support, Q2 goals.
Wed, Sep 6, 8:06 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke renamed T175212: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure. from Services Q1 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure. to Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure..
Wed, Sep 6, 8:06 PM · MediaWiki-JobQueue, ChangeProp, EventBus, Analytics, Goal, Services (next)
GWicke renamed T175212: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure. from Services Q1 2017/18 goal: Migrate most jobs to multi-DC enabled event processing infrastructure. to Services Q1 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure..
Wed, Sep 6, 8:04 PM · MediaWiki-JobQueue, ChangeProp, EventBus, Analytics, Goal, Services (next)
GWicke created T175212: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure..
Wed, Sep 6, 8:04 PM · MediaWiki-JobQueue, ChangeProp, EventBus, Analytics, Goal, Services (next)
GWicke updated the task description for T175206: 2017/18 Annual Plan Program 8: Multi-datacenter support.
Wed, Sep 6, 7:55 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke created T175206: 2017/18 Annual Plan Program 8: Multi-datacenter support.
Wed, Sep 6, 7:51 PM · Performance-Team (Radar), Epic, MediaWiki-Platform-Team, Operations, Services (watching)
GWicke updated subscribers of T174993: Vandalism in "In the news" articles persisting in the app' ?.

@bearND: Okay, nevermind then. @Pchelolo or @mobrovac, is there any other caching for the explore feed?

Wed, Sep 6, 6:25 PM · Reading-Infrastructure-Team-Backlog, Services (watching), Mobile, Wikipedia-iOS-App-Backlog, Wikipedia-Android-App-Backlog, iOS-app-Bugs, Android-app-Bugs
GWicke updated subscribers of T174993: Vandalism in "In the news" articles persisting in the app' ?.
Wed, Sep 6, 5:40 PM · Reading-Infrastructure-Team-Backlog, Services (watching), Mobile, Wikipedia-iOS-App-Backlog, Wikipedia-Android-App-Backlog, iOS-app-Bugs, Android-app-Bugs
GWicke added a comment to T174993: Vandalism in "In the news" articles persisting in the app' ?.

@bearND, until recently the cache timeout for the trending response was 30 minutes. It was lowered to five minutes in https://gerrit.wikimedia.org/r/#/c/374648/.

Wed, Sep 6, 5:39 PM · Reading-Infrastructure-Team-Backlog, Services (watching), Mobile, Wikipedia-iOS-App-Backlog, Wikipedia-Android-App-Backlog, iOS-app-Bugs, Android-app-Bugs
GWicke added a comment to T171965: [Spike - 8 hours] How should the PDF post-processing script be exposed for use by Extension:Collection.

This service would replace the current electron pdf renderer as well on the medium/long run, right?

Wed, Sep 6, 2:16 PM · Proton, Readers-Web-Kanban-Board, Electron-PDFs, Readers-Web-Backlog (Tracking), Spike
GWicke added a comment to T174982: Sourcemap is incorrect in RESTBase help page.

cause an error by putting the wrong URL in the input field

Could you elaborate on this? Which input field are you referring to?

Wed, Sep 6, 12:48 AM · Services (later), RESTBase-API, RESTBase
GWicke added a comment to T150871: [EPIC] (Proposal) Replicate core OCG features and sunset OCG service.

Thanks for the update & clarity on the timeline, @ovasileva! It is much appreciated.

Wed, Sep 6, 12:47 AM · Readers-Web-Backlog (Tracking), Services (watching), OCG-General, User-mobrovac, Epic, Reading-Community-Engagement, Operations

Tue, Sep 5

GWicke updated the task description for T171965: [Spike - 8 hours] How should the PDF post-processing script be exposed for use by Extension:Collection.
Tue, Sep 5, 5:01 PM · Proton, Readers-Web-Kanban-Board, Electron-PDFs, Readers-Web-Backlog (Tracking), Spike
GWicke edited projects for T170319: End of August milestone: Begin migrating job queue processing to multi-DC enabled eventbus infrastructure., added: Services (doing); removed Services (later).
Tue, Sep 5, 2:48 PM · Services (doing), User-Joe, User-Elukey, Goal, Epic
GWicke edited projects for T174992: Update zotero translators , added: Services (watching); removed Services.
Tue, Sep 5, 2:47 PM · User-Ryasmeen, Services (watching), Citoid, VisualEditor

Thu, Aug 31

GWicke added a comment to T171965: [Spike - 8 hours] How should the PDF post-processing script be exposed for use by Extension:Collection.

@Tgr, at first sight it looks like there are reasonable python bindings for headless Chrome as well. Combined with the PDF post-processing library you have been testing, I could see a simple python service doing both pre/postprocessing and actual rendering work well. The service portion of either option is trivial in any case, and all the heavy lifting is in the libraries & Chrome.

Thu, Aug 31, 10:26 PM · Proton, Readers-Web-Kanban-Board, Electron-PDFs, Readers-Web-Backlog (Tracking), Spike
GWicke added a comment to T173710: Job queue is increasing non-stop.

I updated https://gerrit.wikimedia.org/r/#/c/295027/ to apply on current master. This removes CDN purges from HTMLCacheUpdate, and only performs them after RefreshLinks, and only if nothing else caused a re-render since.

Thu, Aug 31, 7:36 PM · Patch-For-Review, Services (watching), Performance-Team (Radar), Discovery-Search, Discovery, CirrusSearch, Wikidata-Sprint, Wikidata, Operations, MediaWiki-JobQueue
GWicke added a comment to T174392: Disk errors: restbase1010.eqiad.wmnet.

No Samsung spares would be surprising, given our last conversation on the topic in April, and from what I remember about the stock back then.

Thu, Aug 31, 6:33 PM · Services (watching), Operations
GWicke updated the task description for T173821: Investigate exposing content styles needed via API vs as HTML tags.
Thu, Aug 31, 5:56 PM · RESTBase-API, Services (watching), Reading-Infrastructure-Team-Backlog (Kanban), Page Content Service
GWicke added a comment to T162179: Extract HTML Compatibility Layer from MCS Mobile Sections API.

Since you asked for bikeshedding.. How about

Thu, Aug 31, 5:54 PM · Services (watching), Page Content Service, Patch-For-Review, Mobile-Content-Service, Reading-Infrastructure-Team-Backlog (Kanban), Reading Epics (Platform JS CSS and HTML consolidation), RESTBase-API
GWicke added a comment to T73853: Retry counts not working / jobs re-executed beyond retry limits.

The replication issues discussed in T163337 could play a role in duplication / keeping old jobs alive.

Thu, Aug 31, 2:33 PM · WMF-deploy-2015-08-25_(1.26wmf20), WMF-deploy-2015-07-28_(1.26wmf16), Patch-For-Review, MediaWiki-JobQueue
Thibaut120094 awarded T152540: Migrate to HTML5 section ids a Like token.
Thu, Aug 31, 9:48 AM · Patch-For-Review, Readers-Web-Backlog (Tracking), User-notice, Community-Tech-Sprint, Mobile-Content-Service, Reading-Infrastructure-Team-Backlog, TechCom-RfC, Community-Wishlist-Survey-2016, Community-Tech, ContentTranslation, MediaWiki-Parser, Parsoid, Parsing-Team

Wed, Aug 30

GWicke added a comment to T73853: Retry counts not working / jobs re-executed beyond retry limits.

I just looked into HTMLCacheUpdate jobs executed in the last 15 hours, and the number of really old jobs still being executed (presumably retried or respawned) is greater than I would expect with a retry limit of 3 (or 2?):

Wed, Aug 30, 11:30 PM · WMF-deploy-2015-08-25_(1.26wmf20), WMF-deploy-2015-07-28_(1.26wmf16), Patch-For-Review, MediaWiki-JobQueue
GWicke added a comment to T173710: Job queue is increasing non-stop.

HTMLCacheUpdate root job timestamp distribution, jobs executed within the last 15 hours:

Wed, Aug 30, 11:12 PM · Patch-For-Review, Services (watching), Performance-Team (Radar), Discovery-Search, Discovery, CirrusSearch, Wikidata-Sprint, Wikidata, Operations, MediaWiki-JobQueue
GWicke added a comment to T173710: Job queue is increasing non-stop.

A possible contribution to the backlog building could be the infinite retry / immortal job problem described in T73853. Looking for old htmlCacheUpdate root jobs from April still executing over four months later (!) via grep htmlCacheUpdate runJobs.log | grep -c 'rootJobTimestamp=201704' in mwlog1001:/srv/mw-log yields 9208 executions, just today. Interestingly, jobs from May, June, and July are much less common (hundreds). Considering that HTMLCacheUpdateJob basically only updates touched timestamps in the DB, and then quickly fires off CDN purges, seeing anything but zero ancient jobs might mean that T73853 is not actually resolved yet. To actually establish whether this significantly contributes to the current backlog, we would need to look at the distribution of rootJobTimestamp values for htmlCacheUpdates from July, especially for the period since the backlog growth really started around the 8th.

Wed, Aug 30, 10:56 PM · Patch-For-Review, Services (watching), Performance-Team (Radar), Discovery-Search, Discovery, CirrusSearch, Wikidata-Sprint, Wikidata, Operations, MediaWiki-JobQueue
GWicke added a project to T173710: Job queue is increasing non-stop: Services (watching).
Wed, Aug 30, 10:07 PM · Patch-For-Review, Services (watching), Performance-Team (Radar), Discovery-Search, Discovery, CirrusSearch, Wikidata-Sprint, Wikidata, Operations, MediaWiki-JobQueue
GWicke added a comment to T73853: Retry counts not working / jobs re-executed beyond retry limits.

@Krinkle, are you saying that we are confident that jobs are no longer retried for more times than the retry limit would nominally allow?

Wed, Aug 30, 9:34 PM · WMF-deploy-2015-08-25_(1.26wmf20), WMF-deploy-2015-07-28_(1.26wmf16), Patch-For-Review, MediaWiki-JobQueue
GWicke added a comment to T174600: Add the ability to sign and verify jobs.

Signed JSON blobs are kind of what JWTs are designed for. There are good libraries for validation.

Wed, Aug 30, 7:19 PM · MediaWiki-JobQueue, Epic, Services (doing), User-mobrovac, Analytics, ChangeProp, EventBus
GWicke added a comment to T159922: pdfrender fails to serve requests since Mar 8 00:30:32 UTC on scb1003.

I personally am not sure whether the startup issues are caused by the same underlying issue as the hangs, or not. I would imagine that an electron worker process restarting could run into similar hangs as on service startup.

Wed, Aug 30, 6:23 PM · Services (done), Readers-Web-Backlog (Tracking), Operations, Electron-PDFs

Tue, Aug 29

GitHub <noreply@github.com> committed rMSCP803222f5b139: Merge pull request #197 from Pchelolo/redis_dedupe (authored by GWicke).
Merge pull request #197 from Pchelolo/redis_dedupe
Tue, Aug 29, 11:10 PM
GWicke triaged T174486: Research: Reduce Cassandra memory usage by avoiding deserialization of expired data as Normal priority.
Tue, Aug 29, 7:32 PM · Services (later), Cassandra
GWicke created T174486: Research: Reduce Cassandra memory usage by avoiding deserialization of expired data.
Tue, Aug 29, 7:32 PM · Services (later), Cassandra
GWicke added a comment to T169116: Support flagged revisions in RESTBase.

Bumped priority, as support for flagged revisions is important for serious reading use cases. There is also an opportunity to piggy-back on current storage schema migration efforts.

Tue, Aug 29, 6:41 PM · RESTBase-API, RESTBase, VisualEditor, Reading-Infrastructure-Team-Backlog, Mobile-Content-Service, Services (designing)
GWicke raised the priority of T169116: Support flagged revisions in RESTBase from Normal to High.
Tue, Aug 29, 6:39 PM · RESTBase-API, RESTBase, VisualEditor, Reading-Infrastructure-Team-Backlog, Mobile-Content-Service, Services (designing)
GWicke claimed T169140: ChangeProp: Enhance if-modified-since support; possibly integrate into rate limiting / deduplication logic.
Tue, Aug 29, 6:37 PM · ChangeProp, RESTBase, Services (designing)
GWicke moved T142090: Add hover-card like summary (og:description) to open graph meta data printing plain text extract from Backlog to watching on the Services board.
Tue, Aug 29, 3:52 PM · Services (watching), Design (RW-Design-Debt), TextExtracts, Readers-Web-Backlog, New-Readers, MediaWiki-General-or-Unknown
GWicke added a comment to T142090: Add hover-card like summary (og:description) to open graph meta data printing plain text extract.

@Jdlrobson, in general MW can (and does) certainly fetch data from the REST API. However, there are some potential issues if we wanted to fetch the summary on each parse or skin render:

Tue, Aug 29, 3:51 PM · Services (watching), Design (RW-Design-Debt), TextExtracts, Readers-Web-Backlog, New-Readers, MediaWiki-General-or-Unknown

Mon, Aug 28

GWicke added a comment to T150871: [EPIC] (Proposal) Replicate core OCG features and sunset OCG service.

@ovasileva, thank you for the update. Does this mean that OCG will be switched off by the end of September, or end of October?

Mon, Aug 28, 7:05 PM · Readers-Web-Backlog (Tracking), Services (watching), OCG-General, User-mobrovac, Epic, Reading-Community-Engagement, Operations
GWicke updated the task description for T159922: pdfrender fails to serve requests since Mar 8 00:30:32 UTC on scb1003.
Mon, Aug 28, 6:56 PM · Services (done), Readers-Web-Backlog (Tracking), Operations, Electron-PDFs
GWicke added a comment to T172832: Investigate use-cases for delayed job executions.

There are three considerations relevant to Echo:

  1. Delayed notifications (T156808: Back-end infrastructure for timed notifications in Echo) 1a. Article reminder notifications (T2582: Remind me of this article in X days) 1b. User group expiry notifications (T153817: Notify users when their user group membership is about to expire, or has expired)
  2. "Batching & rate limiting for Echo notifications".
Mon, Aug 28, 6:35 PM · Wikidata, Analytics, MediaWiki-JobQueue, Epic, Services (doing), User-mobrovac, ChangeProp, EventBus
GWicke added a comment to T73966: Show categories of a page in the Android app.

Categories are page metadata, and the default desktop rendering is done by the skin. Other experiences will want to display categories in different ways, which is also facilitated by separating the category data from its formatting.

Mon, Aug 28, 3:48 PM · Community-Wishlist-Survey-2016, Wikipedia-Android-App-Backlog
GWicke added a project to T174184: Configure phlogiston for the Services team: Services (watching).
Mon, Aug 28, 2:53 PM · Phlogiston (Reporting), Services (watching)
GWicke added a comment to T174184: Configure phlogiston for the Services team.

Description from the original mail for slightly more detail:

Mon, Aug 28, 2:53 PM · Phlogiston (Reporting), Services (watching)

Aug 21 2017

GWicke updated subscribers of T172815: Improve stability and maintainability of our browser-based PDF render service.

The Electron render service currently requires manual attention every few days, so we should address the reliability issues sooner rather than later.

Aug 21 2017, 5:44 PM · Electron-PDFs, OfflineContentGenerator, Operations, Services (designing)