Page MenuHomePhabricator
Create Task
Maniphest T120257

Investigate RESTBase as a possible storage solution for wikitext "errors" and issues that are found by Parsoid
Closed, DeclinedPublic
Actions

Description

T48705 is a proposal to expose information available in Parsoid to enable editors to start fixing up pages. As part of the GSoC project, Hardik used a mongodb based store in labs to prototype the solution. We had initial conversations with WikiProject Check Wikipedia folks about how to integrate this with their workflows. But, we didn't get very far at that time and this project has languished since waiting for someone to resolve this.

On the Parsoid end, the linting code has been around for a long time now and has not been enabled because of this last mile problem of where to dump this info. One possibility is for Parsoid to dump this information in RESTBase and let bots and other Check Wikpedia tools use this information to start fixing things. This ticket is to explore that possibility.

If this seems feasible, we need to develop an API to add / fetch / purge / de-dupe these entries. We also need to figure out consistency requirements and a schema for storage that enables this API. We might be able to pick this up based on what Hardik had used for the mongodb instance.

Related Objects

Event Timeline

ssastry created this task.Dec 3 2015, 6:49 PM
ssastry raised the priority of this task from to Medium.
ssastry updated the task description. (Show Details)
ssastry added projects: Parsoid, RESTBase.
ssastry added subscribers: ssastry, tstarling.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 3 2015, 6:49 PM
ssastry added a comment.Dec 3 2015, 6:52 PM

In the context of T120256: Add tracking category to pages that generate empty <li> elements, something like this might be a possibility for doing a similar thing on the Parsoid end instead of adding a tracking category in the mysql table. But, this wouldn't be as immediately usable by editors that have a workflow based on tracking categories.So, that part needs thinking through.

• mobrovac subscribed.Dec 3 2015, 8:47 PM

It would be really handy to see this concept extended to a check-as-you-type-kind-of feature, so that possible errors and issues can be corrected on the spot by the user editing the article.

• GWicke subscribed.EditedDec 3 2015, 11:43 PM

@ssastry, I think the main question for RB would be the number of dimensions by which this needs to be queried. If this number is fairly low & fits a hierarchical model, then RB with the table storage backend could be a good fit. If it's a random combination of a large number of criteria, then elasticsearch indexing would likely be better.

I have been wondering about integrating elasticsearch indexing with RESTBase before, but haven't actually written the code or drafted an actual design. Lots of handwaving, basically.

Hardikj subscribed.Dec 4 2015, 11:25 AM
Hardikj added a comment.Dec 4 2015, 1:55 PM

The web service for linter lives here, if it helps - https://github.com/hardikj/lintbridge/blob/master/server.js#L127

ssastry mentioned this in T120256: Add tracking category to pages that generate empty <li> elements.Jan 20 2016, 1:14 AM
• GWicke added a project: Services.Oct 12 2016, 11:19 PM
• Pchelolo moved this task from Backlog to later on the Services board.Oct 12 2016, 11:27 PM
• Pchelolo edited projects, added Services (later); removed Services.
• GWicke lowered the priority of this task from Medium to Lowest.Oct 12 2016, 11:29 PM
• GWicke edited projects, added Services (watching); removed Services (later).

I actually don't think this has seen any movement since 2015, and we don't currently plan to work on this.

@ssastry, should we close this?

Legoktm closed this task as Declined.Oct 16 2016, 3:53 AM
Legoktm subscribed.

The current plan is to store these in a database table, so closing this as declined.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL