Pchelolo
User

Projects

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.
User Since
Jun 24 2015, 10:23 AM (87 w, 23 h)
Availability
Available
LDAP User
Unknown
MediaWiki User
Pchelolo

Recent Activity

Yesterday

Pchelolo claimed T158822: RESTBase/Parsoid responding with 404 in Vagrant for Main Page.

Ok, the reason is that we've move the CSP headers generation out into a filter and forgot to update RESTBase config in Vagrant. The above patch should take care of that and fix your problem.

Wed, Feb 22, 11:55 PM · Patch-For-Review, Services (doing), Parsoid, RESTBase, MediaWiki-Vagrant
Pchelolo added a project to T158822: RESTBase/Parsoid responding with 404 in Vagrant for Main Page: Services (doing).

Ha! Interesting! I can reproduce the problem only if I go to localhost:8080 and try to do a visual edit. Then the browser sends a request to dev.wiki.local.wmftest.net, it's a cross-origin request, so a preflight is sent. For some reason RESTBase responds to a preflight OPTIONS with a 404. Investigation continues.

Wed, Feb 22, 11:48 PM · Patch-For-Review, Services (doing), Parsoid, RESTBase, MediaWiki-Vagrant
Pchelolo added a comment to T156662: Drop old mobile apps tables.

Cool. I will do that early next week. Wanna have some buffer time in case we'd need to rollback the switch

Wed, Feb 22, 11:32 PM · RESTBase, Services (later)
Pchelolo added a comment to T156662: Drop old mobile apps tables.

RESTBase was switched to using new tables, so old tables could be dropped from Cassandra now. Let's wait a bit to make sure all's fine before actually dropping the tables.

Wed, Feb 22, 11:14 PM · RESTBase, Services (later)
Pchelolo added a comment to T112648: enable restbase syslog/file logging.

So, I've tested the syslog-over-udp solution too, and as we've expected, the service doesn't even notice the filled-up partition.

Wed, Feb 22, 10:04 PM · service-runner, Services (doing), Operations, RESTBase
Pchelolo added a comment to T112648: enable restbase syslog/file logging.

@Pchelolo awesome! Thank you for testing. Could you also test the scenario where logs are sent to syslog via udp and then fill the partition again? So that we can be informed of the outcomes, make a decision and proceed accordingly.

Wed, Feb 22, 8:28 PM · service-runner, Services (doing), Operations, RESTBase
Pchelolo added a comment to T112648: enable restbase syslog/file logging.

I've done some testing on what's happening if the partition where we log fills up. I've created a VM with a 100 meg partition and started trace-logging there. When the partition fills up, an unhandled exception is raised and a worker is bing killed. This makes the master process attempt to log that a worker has died, and the master is also being killed.

Wed, Feb 22, 7:06 PM · service-runner, Services (doing), Operations, RESTBase

Tue, Feb 21

Pchelolo added a comment to T112648: enable restbase syslog/file logging.

@Pchelolo, logging directly to a file is synchronous, which is bad for performance & can cause outages. See the earlier discussion for an example of such an outage.

Tue, Feb 21, 9:51 PM · service-runner, Services (doing), Operations, RESTBase
Pchelolo edited projects for T112648: enable restbase syslog/file logging, added: Services (doing); removed Services (later).

So, we've discussed this on the team meeting and decided to move forward on this.

Tue, Feb 21, 9:30 PM · service-runner, Services (doing), Operations, RESTBase
Pchelolo added a comment to T158702: Page properties-change event is rejected if page was deleted.

What if the page is deleted, the properties are changed, and then later the page is restored?

Yes, that is my question too: will the page pros event still affect it in MW or not? We don't seem to know for sure, so let's investigate?

Tue, Feb 21, 9:03 PM · Patch-For-Review, EventBus, Services (next), Analytics
Pchelolo added a comment to T158702: Page properties-change event is rejected if page was deleted.

@mobrovac So, I've found the way to get that original page_id for the page (see Gerrit) patch. On one hand, I agree that firing an event for a deleted page might not be useful, but just discarding it seems a bit scary. We are not really sure what properties might've been changed and whether it's 100% safe to discard it.. I'm on the edge here..

Tue, Feb 21, 8:54 PM · Patch-For-Review, EventBus, Services (next), Analytics
Pchelolo created T158702: Page properties-change event is rejected if page was deleted.
Tue, Feb 21, 8:34 PM · Patch-For-Review, EventBus, Services (next), Analytics
Pchelolo added a comment to T158338: Set up DNS caching for node services.

So, seems we need to redeploy all the services once before the switchover happens. Do if think it worths doing that explicitly or rely on normal deployment flow for that?

Tue, Feb 21, 8:05 PM · service-runner, Services (doing), codfw-rollout, codfw-rollout-Jan-Mar-2016
Pchelolo closed T157693: Use minified JSON format in ChangeProp as "Resolved".

Change deployed. Resolving.

Tue, Feb 21, 6:41 PM · Services (blocked), Patch-For-Review, ChangeProp, ORES, Revision-Scoring-As-A-Service, User-Ladsgroup
Gerrit Code Review <gerrit@wikimedia.org> committed rMSCD4706f9dafc12: Merge "Make ORES return minified responses" (authored by Pchelolo).
Merge "Make ORES return minified responses"
Tue, Feb 21, 6:36 PM

Mon, Feb 20

Pchelolo edited projects for T158338: Set up DNS caching for node services, added: Services (doing); removed Services (next).

I'll need to switch back statsd to eqiad in the next few days, let me know what services have been switched so we can test it in production too.

Mon, Feb 20, 9:37 PM · service-runner, Services (doing), codfw-rollout, codfw-rollout-Jan-Mar-2016
Pchelolo added a comment to T112648: enable restbase syslog/file logging.

I think we have to proceed on this. Right now, without local logging, if something breaks with Logstash (see T158602) we're losing all the logs completely, which is very not good. For example, in the latest 24 hours there were some spikes in 500 errors in RESTBase which I would really like to look at, but simultaneously logstash logging is somehow broken, so we've lost some potentially important info.

Mon, Feb 20, 8:50 PM · service-runner, Services (doing), Operations, RESTBase
Pchelolo created T158602: RESTBase logs disappeared from logstash.
Mon, Feb 20, 8:30 PM · Wikimedia-Logstash, Services (watching)
Pchelolo committed rMSCP8c499448a947: Use default DNS cache parameters (authored by Pchelolo).
Use default DNS cache parameters
Mon, Feb 20, 5:19 PM
Pchelolo committed rMSCPb8ef149af458: Add DNS caching (authored by Pchelolo).
Add DNS caching
Mon, Feb 20, 5:19 PM

Fri, Feb 17

Pchelolo added a comment to T158338: Set up DNS caching for node services.

So, now when the PR is merged we probably need to go over services setting up the caching configuration and updating the service-runner version..

Fri, Feb 17, 10:12 PM · service-runner, Services (doing), codfw-rollout, codfw-rollout-Jan-Mar-2016
Pchelolo edited projects for T158451: Kafka logs are not pruned on vagrant, added: Services (watching), EventBus; removed Services.
Fri, Feb 17, 9:35 PM · EventBus, Services (watching), Analytics, MediaWiki-Vagrant
Pchelolo added a comment to T158338: Set up DNS caching for node services.

I've made a PR for service-runner to stop caching statsd IPs indefinitely and to introduce the global DNS caching: https://github.com/wikimedia/service-runner/pull/152

Fri, Feb 17, 9:06 PM · service-runner, Services (doing), codfw-rollout, codfw-rollout-Jan-Mar-2016
Pchelolo lowered the priority of T157845: Updates to mobile content might not be working from "Normal" to "Low".

Ok, I've looked at this and right now everything seems to work fine. I will repeat the test after we switch the backends, but lowering the priority to low for now.

Fri, Feb 17, 8:25 PM · Services (next)
Pchelolo moved T158437: Change ORES rules to send all events to new "/precache" endpoint from Backlog to watching on the Services board.
Fri, Feb 17, 7:08 PM · Services (watching), Revision-Scoring-As-A-Service-Backlog, ORES, ChangeProp
Pchelolo added a comment to T148714: Create generalized "precache" endpoint for ORES.

@Halfak Hm... So, in the context of Edit-Review-Improvements-ReviewStream project we might want to reuse the response to integrate it into the new stream. Why wouldn't we just return the normal response for that? Basically treat a precaching request like a request for scores with 'all models'?

Fri, Feb 17, 6:35 PM · Revision-Scoring-As-A-Service, Analytics, Services (watching), User-mobrovac, EventBus, ChangeProp
Pchelolo added a comment to T146944: Mobile-sections replacement.

In T146944#3035658, @Niedzielski wrote:
For fallback to kick in, the endpoint must return a status other than 404 (e.g., https://en.wikipedia.org/api/rest_v1/foobar returns 404 which would be considered insignificant)
Right. Thank you. 404 wouldn't work since we need that for titles that don't exist. We could try other codes, like 410, 400, or 501.

Fri, Feb 17, 5:48 PM · Reading Epics (Platform JS CSS and HTML consolidation), User-mobrovac, RESTBase-API, Services (watching), Wikipedia-Android-App-Backlog, Reading-Web-Backlog, Mobile-Content-Service
Pchelolo added a comment to T148714: Create generalized "precache" endpoint for ORES.

@Halfak One moment we need to clarify is what to return in case the wiki is not supported.

Fri, Feb 17, 5:43 PM · Revision-Scoring-As-A-Service, Analytics, Services (watching), User-mobrovac, EventBus, ChangeProp
Pchelolo added a comment to T157693: Use minified JSON format in ChangeProp.

Oh, sorry, I've made a new one, didn't look that you've already done it. Anyway, will deploy on Mon.

Fri, Feb 17, 5:39 PM · Services (blocked), Patch-For-Review, ChangeProp, ORES, Revision-Scoring-As-A-Service, User-Ladsgroup
Pchelolo committed rMSCDadb7e21f0e08: Config: use minified JSON in ORES response (authored by Pchelolo).
Config: use minified JSON in ORES response
Fri, Feb 17, 5:24 PM
Pchelolo added a comment to T148714: Create generalized "precache" endpoint for ORES.

E.g. send something like {"wiki": "enwiki", "revision": {"new": 123456}, "bot": ""} as data to https://ores.wikimedia.org/v2/scores/precache/ and ORES will internally work out that (1) it has models for 'enwiki' and (2) which models should be precached for this bot edit.

Fri, Feb 17, 5:14 PM · Revision-Scoring-As-A-Service, Analytics, Services (watching), User-mobrovac, EventBus, ChangeProp
Pchelolo added a comment to T157693: Use minified JSON format in ChangeProp.

Done. Will deploy Monday.

Fri, Feb 17, 5:12 PM · Services (blocked), Patch-For-Review, ChangeProp, ORES, Revision-Scoring-As-A-Service, User-Ladsgroup
Pchelolo added a comment to T158265: runner.stop() only calls worker.stop() if num_workers === 0.

@GWicke Hm, good point. But we do know when disconnect is done - when the disconnected event is fired. Current implementation would indeed shut down stuff under the feet of the running requests - that's true. I'll try to overturn the order of execution.

Fri, Feb 17, 1:40 AM · Services (next), service-runner
Pchelolo added a comment to T158265: runner.stop() only calls worker.stop() if num_workers === 0.

@Arlolra If you have time, could you test if my PR would solve your issue?

Fri, Feb 17, 1:34 AM · Services (next), service-runner
Pchelolo added a comment to T158265: runner.stop() only calls worker.stop() if num_workers === 0.

This should fix it: https://github.com/wikimedia/service-runner/pull/151

Fri, Feb 17, 1:17 AM · Services (next), service-runner
Pchelolo closed T158248: Rate limiter timers prevent process from exiting cleanly as "Resolved".

PR merged and published. Should be all good now, resolving. I've tested it with Parsoid tests.

Fri, Feb 17, 12:30 AM · Services (done), service-runner

Thu, Feb 16

Pchelolo added a comment to T125254: Stabilize API end points.

The PR stabilizing endpoints: https://github.com/wikimedia/restbase/pull/764

Thu, Feb 16, 11:40 PM · Services (next), Services-next, RESTBase-API, RESTBase
Pchelolo added a comment to T155804: log-events topic emitted in EventBus.

Do we know the approximate rate of events here?

Thu, Feb 16, 10:35 PM · Services (watching), Analytics, EventBus
Pchelolo added a project to T155804: log-events topic emitted in EventBus: Services (watching).
Thu, Feb 16, 10:30 PM · Services (watching), Analytics, EventBus
Pchelolo added a comment to T146944: Mobile-sections replacement.

It's actually possible to stream JSON, there's a bunch of libraries for doing that, for example this one: http://oboejs.com

Thu, Feb 16, 10:19 PM · Reading Epics (Platform JS CSS and HTML consolidation), User-mobrovac, RESTBase-API, Services (watching), Wikipedia-Android-App-Backlog, Reading-Web-Backlog, Mobile-Content-Service
Pchelolo added a comment to T158265: runner.stop() only calls worker.stop() if num_workers === 0.

@Pchelolo, the disconnect() call is part of the graceful shutdown sequence. It causes the worker to stop accepting new requests, but lets existing requests finish processing.

Thu, Feb 16, 9:42 PM · Services (next), service-runner
Pchelolo added a comment to T158265: runner.stop() only calls worker.stop() if num_workers === 0.

Hm, actually, this ticket made me look into the whole termination procedure that we have, and we're doing it all wrong.

Thu, Feb 16, 9:27 PM · Services (next), service-runner
Pchelolo added a comment to T158338: Set up DNS caching for node services.

I've made a simple benchmark for htcp-purge package. Without DNS caching we're doing 2932 purges/sec, using the dnscache package we're able to do 35663 purges per second. So for UDP traffic the performance improvement is tremendous, more then 10 times.

Thu, Feb 16, 7:35 PM · service-runner, Services (doing), codfw-rollout, codfw-rollout-Jan-Mar-2016
Pchelolo added a comment to T158338: Set up DNS caching for node services.

At some point I've looked into the dnscache package and tried to apply it to several services and measure performance. I didn't find any major performance improvements for smallish cache periods, but that's because I've been mostly testing it for TCP traffic between services, where a DNS lookup is just a small fraction of the request latency. I can try again with enabling all of our UDP traffic and check whether performance boost would be more significant.

Thu, Feb 16, 7:05 PM · service-runner, Services (doing), codfw-rollout, codfw-rollout-Jan-Mar-2016
Pchelolo added a project to T158265: runner.stop() only calls worker.stop() if num_workers === 0: Services (next).

We've made the runner.stop() for tests which never run with > 0 workers, thus the bug is there. I can fix it tomorrow unless you wanna make a PR yourself.

Thu, Feb 16, 12:55 AM · Services (next), service-runner

Wed, Feb 15

Pchelolo renamed T158100: Deprecate and remove the public title/{title} endpoint from "Decide on the future of the public title/{title} endpoint" to "Deprecate and remove the public title/{title} endpoint".
Wed, Feb 15, 12:43 AM · RESTBase-API, RESTBase, Services (doing)
Pchelolo added a comment to T156734: Create Location Search APIs in RESTBase.

Current known use cases:
Provide a location and radius (or bounding box) and return articles sorted by page views
Provide a search query, location and radius (or bounding box) and return articles sorted by relevance/page views
Provide "an object type", location and radius (or bounding box) and return articles sorted by relevance/page views (e.g. buildings, cities, monuments, etc…)

Wed, Feb 15, 12:36 AM · Services (designing), Reading Epics (Location Search), Mobile-Content-Service
Pchelolo updated subscribers of T158139: Randomise the checker script delay.

@mobrovac That works too. I think we don't have this issue on other endpoints since there's no expiration for them. I'll create a PR to get rid of monitoring for this endpoint in RESTBase as soon as @bearND adds monitoring for it on the MCS side.

Wed, Feb 15, 12:09 AM · Services (later)
Pchelolo created T158139: Randomise the checker script delay.
Wed, Feb 15, 12:00 AM · Services (later)

Tue, Feb 14

Pchelolo added a comment to T158100: Deprecate and remove the public title/{title} endpoint.

Drafted a deprecation notice: https://docs.google.com/document/d/1lOsQRsH7aJfnyf_UGL31gHoqJZCO_XrYEUZ1EYq47hU/edit#

Tue, Feb 14, 11:32 PM · RESTBase-API, RESTBase, Services (doing)
Pchelolo added a comment to T158128: Deprecate and delete mobile-text endpoint.

Here's a draft for the deprecation email: https://docs.google.com/document/d/1R2qmPZ7HfitKcZoqhnQ1BMvtqwX2FU7Q6vdHpnF9y9Y/edit#

Tue, Feb 14, 10:39 PM · Mobile-Content-Service, Services (doing)
Pchelolo added a comment to T158100: Deprecate and remove the public title/{title} endpoint.

So, the overall agreement is to delete (hide) the following endpoints:

  1. /page/title/
  2. /page/title/{title}
  3. /page/title/{title}/
  4. /page/revision/
  5. /page/revision/{revision}
Tue, Feb 14, 10:39 PM · RESTBase-API, RESTBase, Services (doing)
Pchelolo added a comment to T158128: Deprecate and delete mobile-text endpoint.

I propose to set the deletion date to 2017-05-01.

Tue, Feb 14, 10:24 PM · Mobile-Content-Service, Services (doing)
Pchelolo created T158128: Deprecate and delete mobile-text endpoint.
Tue, Feb 14, 10:16 PM · Mobile-Content-Service, Services (doing)
Pchelolo added a comment to T158100: Deprecate and remove the public title/{title} endpoint.

Okey, lemme try to prototype this.. I'll do it in a separate PR from the general stability bump PR (https://github.com/wikimedia/restbase/pull/764) for possibly of easy revert.

Tue, Feb 14, 9:55 PM · RESTBase-API, RESTBase, Services (doing)
Pchelolo added a comment to T158100: Deprecate and remove the public title/{title} endpoint.

As for the secret endpoint, instead of playing with IP address ranges, I think we should just set up a hidden domain in RESTBase, i.e. one that cannot be accessed from outside of production, and use the endpoint there. Something like restbase.svc.eqiad.wmnet:7231/{domain:private}/v1/{target-domain}/page/title/{title}.

Tue, Feb 14, 9:49 PM · RESTBase-API, RESTBase, Services (doing)
Pchelolo added a comment to T158100: Deprecate and remove the public title/{title} endpoint.

I think it's useful to have the title -> (pageid, revision) connection. It is true that it is not used right now and it somewhat incomplete, but I don't think that's a good reason to remove it. Not being used now, does not mean people will not leverage it in the future, does it? What is your concern with keeping it? You mention purely implementation details, but I think that the conversation should be about the API endpoint, not its underlying implementation.

Tue, Feb 14, 8:47 PM · RESTBase-API, RESTBase, Services (doing)
Pchelolo added a comment to T158100: Deprecate and remove the public title/{title} endpoint.

One more option is to leave the endpoints for updates just hide it from the public merged spec.

Tue, Feb 14, 7:19 PM · RESTBase-API, RESTBase, Services (doing)
Pchelolo created T158100: Deprecate and remove the public title/{title} endpoint.
Tue, Feb 14, 6:58 PM · RESTBase-API, RESTBase, Services (doing)

Mon, Feb 13

Pchelolo closed T143420: Make public "Anniversaries" endpoint as "Resolved".

Deployment done: https://en.wikipedia.org/api/rest_v1/#!/Feed/aggregatedFeed_0
The endpoint is completely operational!

Mon, Feb 13, 11:55 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo closed T143420: Make public "Anniversaries" endpoint, a subtask of T143408: Create endpoint for "Anniversaries", as "Resolved".
Mon, Feb 13, 11:55 PM · Services (watching), Patch-For-Review, Reading Epics (Anniversaries), Mobile-Content-Service (Kanban), Wikipedia-iOS-App-Backlog
Pchelolo closed T157647: Anniversaries crash for unsupported wikis as "Resolved".

It's been deployed, so we're good. Resolving.

Mon, Feb 13, 10:41 PM · Mobile-Content-Service (Kanban), Patch-For-Review, Services (watching)
Pchelolo closed T157647: Anniversaries crash for unsupported wikis, a subtask of T143420: Make public "Anniversaries" endpoint, as "Resolved".
Mon, Feb 13, 10:41 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo added a comment to T156199: Low-latency current revision storage.

What was the reason for using tid as a cluster key here (I mean as opposed to another component of the partition key)? In other words, is there any reason not to use the following instead?

Mon, Feb 13, 9:48 PM · Services (next), RfC, RESTBase
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

The PR for RESTBase has been updated. https://github.com/wikimedia/restbase/pull/761

Mon, Feb 13, 7:54 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

Cool. I will update the RESTBase PR after @bearND makes changes to MCS and we can deploy RB portion today/tomorrow

Mon, Feb 13, 6:32 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

I don't really care too much, let's just get to a conclusion and deploy this. Looks like we are all leaning to onthisday, so I will agree with that. 3. 2. 1. Sold?

Mon, Feb 13, 6:29 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog

Sat, Feb 11

Pchelolo added a comment to T152131: "geojson" plugin is missing in Mapnik package.

In the meantime I've made some tweaks to the docker build script and to service-runner and hopefully with @MaxSem patch and this gerrit https://gerrit.wikimedia.org/r/#/c/337200/ we will get a working build.

Sat, Feb 11, 1:40 AM · Interactive-Sprint, Discovery, Maps (Kartotherian)

Fri, Feb 10

Pchelolo edited the description of T157845: Updates to mobile content might not be working.
Fri, Feb 10, 11:15 PM · Services (next)
Pchelolo created T157845: Updates to mobile content might not be working.
Fri, Feb 10, 10:50 PM · Services (next)
Pchelolo added a comment to T155070: Clicking on references is broken..

Hm... I'm gonna create a subtask and assign to myself to investigate, but won't test it today, since there's a pretty big change coming to RESTBase regarding MCS content early next week (https://github.com/wikimedia/restbase/pull/753), so even if there's a bug, we need to test after that's merged and deployed.

Fri, Feb 10, 10:49 PM · ChangeProp, RESTBase, Mobile-Content-Service (Kanban), Android-app-Bugs, Patch-For-Review, Wikipedia-Android-App-Backlog (Kanban)
Pchelolo added a comment to T155070: Clicking on references is broken..

Em, sorry, not quite clear to me what happened here from the comments. Do I understand correctly that a fix to MCS was deployed, but after that even a very popular-emitted page wasn't updated to the newer content?

Fri, Feb 10, 10:41 PM · ChangeProp, RESTBase, Mobile-Content-Service (Kanban), Android-app-Bugs, Patch-For-Review, Wikipedia-Android-App-Backlog (Kanban)
Pchelolo closed T107966: Establish a way for services to get the path of the config file as "Resolved".

It's possible to provide the path to the config using the -c option. Resolving.

Fri, Feb 10, 10:35 PM · Services (next), service-template-node, service-runner
Pchelolo triaged T156315: Optimisation for revision policies: use DELETE if grace_ttl is 0 as "Low" priority.

Setting low priority since if the new storage model would not need this optimization, because we're removing the revision retention policies completely.

Fri, Feb 10, 10:12 PM · Services (next)
Pchelolo added a comment to T157095: Create a special spec file in RESTBase to hold common schemas.

PR here: https://github.com/wikimedia/restbase/pull/762

Fri, Feb 10, 8:17 PM · Services (done), RESTBase
Pchelolo added a comment to T157693: Use minified JSON format in ChangeProp.

Cool! I'll schedule some work on our end to leverage that.

Fri, Feb 10, 7:55 PM · Services (blocked), Patch-For-Review, ChangeProp, ORES, Revision-Scoring-As-A-Service, User-Ladsgroup
Pchelolo added a comment to T157822: Support multiple partitions per topic in EventBus.

It's not a high priority, I just keep filing tasks with ideas I have for the upcoming JobQueue migration project. It will be way more relevant a bit later.

Fri, Feb 10, 6:32 PM · ChangeProp, Services (later)
Pchelolo added a comment to T157822: Support multiple partitions per topic in EventBus.

Per topic schema-field-key. We'd have to produce each message to Kafka using a key, and this would have to be configured per topic.

Fri, Feb 10, 6:30 PM · ChangeProp, Services (later)
Pchelolo created T157822: Support multiple partitions per topic in EventBus.
Fri, Feb 10, 6:09 PM · ChangeProp, Services (later)
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

Hm, but if we plan to extend it to include specific years and/or months as we discussed today in the meeting, neither of these really work :(

Fri, Feb 10, 5:58 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo added a comment to T157693: Use minified JSON format in ChangeProp.

@Ladsgroup Kk, then a question is whether we want to have pretty JSON at all? Normally in APIs pretty JSON is not the default, at least I've never seen any APIs that return pretty JSON by default. Even special parameters to return pretty JSON are quite rear, since presentation is the client's responsibility.

Fri, Feb 10, 5:16 PM · Services (blocked), Patch-For-Review, ChangeProp, ORES, Revision-Scoring-As-A-Service, User-Ladsgroup
Pchelolo added a comment to T157693: Use minified JSON format in ChangeProp.

@Ladsgroup Em, there's literally no difference between the responses. They're byte-to-byte identical.

Fri, Feb 10, 5:06 PM · Services (blocked), Patch-For-Review, ChangeProp, ORES, Revision-Scoring-As-A-Service, User-Ladsgroup
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

I've been trying to look something up in the dictionary, but didn't really find anything good. I'm ok with onthisday or anniversaries

Fri, Feb 10, 1:02 AM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog

Thu, Feb 9

Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

/feed/events/byday/{type}/{mm}/{dd} is a bit unclear to me. What events? "Event" is like a super-broad concept, it could mean almost anything. In the API it's even more confusing, because in computer science 'event' have a pretty specific meaning.

Thu, Feb 9, 10:36 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo added a comment to T156680: Allow API consumer to express a timeframe in hours.

Ping me once it's deployed, I'll modify the RESTBase part

Thu, Feb 9, 8:37 PM · Reading-Web-Sprint-91-🐔🍔, Unplanned-Sprint-Work, Reading-Web-Backlog, User-mobrovac, Services (watching), Reading-Web-Trending-Service
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

Created the primary version of the PR here: https://github.com/wikimedia/restbase/pull/761

Thu, Feb 9, 8:31 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo added a comment to T157647: Anniversaries crash for unsupported wikis.

For the public endpoint, though, we probably want to expose the endpoint only for enwiki, and not for the others at this time.

Thu, Feb 9, 7:02 PM · Mobile-Content-Service (Kanban), Patch-For-Review, Services (watching)
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

Why can't we purge when we get the change prop event for the page being edited? Is this purely because of it being under the "feed" route?

Thu, Feb 9, 4:51 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo added projects to T157649: Partition the transclusions topic in ChangeProp: Services (later), ChangeProp.
Thu, Feb 9, 1:59 AM · ChangeProp, Services (later)
Pchelolo created T157649: Partition the transclusions topic in ChangeProp.
Thu, Feb 9, 1:58 AM · ChangeProp, Services (later)
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

Thank you @Mhurd. I think after your explanation we'd all agree not to include yyyy parameter. In case we ever want to implement the functionality to return stuff that happened exactly on the day of the particular year we can add one more endpoint for that.

Thu, Feb 9, 1:48 AM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo assigned T157647: Anniversaries crash for unsupported wikis to bearND.

We can't deploy the public endpoints until this is fixed, so setting the priority to High.

Thu, Feb 9, 1:31 AM · Mobile-Content-Service (Kanban), Patch-For-Review, Services (watching)
Pchelolo created T157647: Anniversaries crash for unsupported wikis.
Thu, Feb 9, 1:29 AM · Mobile-Content-Service (Kanban), Patch-For-Review, Services (watching)
Pchelolo created T157645: Maximum call stack size exceeded error in Parsoid.
Thu, Feb 9, 1:23 AM · Parsoid
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

Ye, I'm on the edge here as well.

Thu, Feb 9, 12:05 AM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog

Wed, Feb 8

Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.

Hm, there are also births/deaths/events per year too, e.g. 2016. In the same vein, there are also per-month pages, e.g. deaths in March 2016. Is this something we should consider in this discussion as well? Or at least keep the room for it?

Wed, Feb 8, 10:27 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo created T157620: Support providing source sha1 to the deploy build script.
Wed, Feb 8, 9:28 PM · service-runner, Services (later)
Pchelolo added a comment to T143420: Make public "Anniversaries" endpoint.
  1. From what I've understood the all anniversaries endpoint would be used in the app after the user clicks a 'more' button on the selected anniversaries card. How do you expect the other individual types of anniversaries be used? Should we expose individual anniversary types at all?
Wed, Feb 8, 9:22 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo edited projects for T143420: Make public "Anniversaries" endpoint, added: Services (doing); removed Services (blocked).

@Fjalapeno From the discussions we've had about this, the public API should look as follows:

Wed, Feb 8, 8:55 PM · Reading Epics (Anniversaries), Mobile-Content-Service, Wikipedia-iOS-App-Backlog
Pchelolo closed T157061: Add page view data to summaries? as "Declined".

I think we've reached an agreement here that the page views data will not be added to the summary endpoint. I'm closing the task as Declined

Wed, Feb 8, 8:22 PM · Services (designing), Reading Epics, RESTBase-API, Mobile-Content-Service