Fri, May 26
The HTML <strong> wrapper in display_title is somewhat surprising to me. Would all titles have the <strong> wrapper? If so, could we instead return just the human-readable title, and let clients add the wrappers / formatting they want?
Thank you @Joe for getting to this!
Was that due to moves, or due to undeletions?
Not sure if that's related, but today we've noticed then when some templates were renamed on ZH and ES wikis, the RevisionInsertComplete hook was run for every single revision in the history of that template page as if they were all inserted in the DB again. It created quite an unusual load pattern in ChangeProp, so I think that was not happening before.
Thu, May 25
Unassigning this from myself since Parsoid is the cause of this.
I've tracked it down to a problem in Parsoid and title validation. Perhaps some weird link or template was included so right now Parsoid is not able to parse this page. Investigation continues.
Tue, May 23
@GWicke: When will Services be able to pick this up? Is it blocked on the iOS team checking whether the app is sending the correct Content-Type header?
Can we create an list of summary usages and see what happens with them if we start returning the html version by default?
On the discussion on the hackathon with @JAllemandou we've decided to reuse spark infrastructure for this. @Pchelolo will develop a Scala-based router for that would take a spec and match request URIs to RESTBase endpoints and @JAllemandou will integrate it on the analytics side.
The schema is merged and live in the EventBus
Same thing happened today for one of the mobile-apps checks:
Fri, May 19
Ok, we've figured it out. It's happening because it's a private Wiki that's not supported in RESTBase currently. Support is tracked in T88016
Thu, May 18
While template edits should only enqueue a constant number of small jobs directly, I suspect that the expansion into "leaf" sub-jobs will initially still happen during job processing itself. This means that those would hit eventbus, rather than following the changeprop model of producing directly. @Pchelolo, is this correct?
Wed, May 3
Tue, May 2
Alright, I think we need to hold our horses a bit, people. There have been a number of tickets lately about adding information and/or functionality to the summary end point, which, quite frankly, I believe don't belong there at all. Instead of adding things to summary, I think we should list/assess what information will Reading need in the short- to mid-term and find the appropriate ways to provide/expose/surface the needed information.
To support this effort, there must be a way to let users know how large a download will be. This is primarily important when downloading a "set" of articles. As in the upcoming Reading List Service.
@Fjalapeno We would eventually support Accept header content-type negotiation, but even with that it's a disruptive change, so we're definitely not doing it until we absolutely must. This change is not an absolute must, but I've created a task to not forget about it when something more important appears.
Seems good @Niedzielski. I've created this to not forget we wanted to do it, but most likely it's not gonna happen very soon, the changes are backwards-incompatible and there's lots of clients using the old format..
So the page previews uses the RESTBase summary endpoint which emits the plain-text extract. Potentially we can change that to the HTML extract, but need to verify with all the known clients of the endpoint. Under the hood it uses the TextExtracts extension, but I don't see it emitting the classes in the HTML. @kaldari how do I fetch the html with the nopopup class from the TextExtracts API?
But it seems like this isn't a good idea? I am seeing some resistance to this notion (also in the Reading List discussion). Normally having an unique, numeric id to access a resource is a pretty welcome thing - but maybe there are complications here that make this not as good as I thought?
Mon, May 1
@Pchelolo I thought all RESTBase summaries used the same keys/structure? Are we overriding something here?
Apr 28 2017
As I've noted in some other ticket:
I guess we need a lint rule for just the output of our services
Created a PR to add the data anyway: https://github.com/wikimedia/restbase/pull/810 but it would be nice to get naming resolving plan before adding one more property for title.
So, we could port this Android code to RESTBase and strip it while generating the summary response. The question is whether there's any client that need this information? If not - the solution is quite straightforward.
Would it make sense to strip it in RESTBase for the summary endpoint since it seems like none of the clients actually need it.
So, we're already making the call to the query info endpoint for constructing the summaries, so adding the displaytitle property is easy.
Hm.. In RESTBase for example we don't really allow snake_case variable names, only property names. We use snake_case for configuration parameters while normal vars use camelCase
We have one in a form of eslint config: https://github.com/wikimedia/eslint-config-node-services
There's multiple levels of reasoning why this will probably not happen:
And here we are: https://github.com/wikimedia/restbase/pull/809
Easy! Will make a PR today
Apr 26 2017
Apr 25 2017
@Halfak Added a subtask.
Deployed to production, resolving.
I'm not sure whether it makes sense to add this to the REST API. It wouldn't benefit from caching at all, and although we support logged-in calls, the actual authorization is based on the mediawiki API, so calling action API for this data would most likely be more efficient then wrapping it to the REST API
Apr 21 2017
Created a subtask for that work @Jdforrester-WMF will implement next week.
@Jdforrester-WMF Hm... Good idea. Gateway timeout might be a more appropriate error here I guess. What do you think?
Parsoid is dying on this page with an exception from T163330 so should get resolved after the next Parsoid deploy
Apr 20 2017
The patch was merged, to be deployed on Monday
Apr 19 2017
To enable topic deletion we need to put delete.topic.enable=true to the kafka server.properties and restart instances.
Apr 18 2017
Actually, that has been already done and announced with a blog post. Resolving.
All right @Ottomata, I'll try to remember python which I didn't use in like 5 years, let's see where it gets us :)
@Ottomata would you have any spare cycles to work on this or to guide me a bit through the event logging code to see how to do that? I'm slowly progressing on the general task and this seems to be a logical next preparation step..
I guess the patch was deployed and I haven't seen these errors in the RB logs for a while, so the issue can be closed.
Apr 13 2017
Created a PR for RESTBase to allow 5 minutes of client-side caching: https://github.com/wikimedia/restbase/pull/795
After moving to scap3 this ticket is obsolete, resolving.
I guess after move to scap3 this ticket is obsolete, so I'm closing it as resolved. If disagree please reopen.
After switching to scap3 this is not relevant any more
@Pchelolo I don't know if you had in mind some programmatic means of looking at these, but I sampled a number of these at random and found them to have less VE usage than commons (or none at all).
At that point - yes, just tried to rerun it - same thing. I have vagrant settings nfs_shares off because of the T158617 , otherwise it's a standard vagrant setup
Apr 12 2017
The prefacing rule is now updating ORES in both datacenters. Although there's still room for improvement, this issue can be resolved now.
Here's the list of wikis that will be affected by the truncate:
Apr 11 2017
After a bit of hassle the newest version of the service has been deployed.
Although the patch was merged, the situation didn't change - the exact same log is produced on server restart. This blocks deployment of the new version T160764 because, although for the old version the error was there, it didn't prevent the service from starting up. The new version just hangs after the AssertionError and doesn't accept connections. In beta cluster though this doesn't happen.
The same thing has just happened when I've tried to update the service to a newer version (see T160764). Will put the patch for puppet SWAT and attempt to deploy after it's merged.