Page MenuHomePhabricator
Create Task
Maniphest T206048

Eliminate Parsoid section.offsets table from Cassandra
Closed, ResolvedPublic
Actions

Description

Section offsets are a subpropery of the data-parsoid but we're storing it both in data-parsoid and separately, in sections.offsets table in Cassandra. The original idea was to optimize the reading of the sections so that the full data-parsoid structure shouldn't be read from storage and parsed to extract the offsets. However, I believe that was a very premature optimization. The sections endpoints are by far not the most popular, fetching and parsing the data-parsoid will probably take some negligible amount of time compared to everything else that should be done. On the other hand, removing it will simplify the code and given that we're short on IO in Cassandra, saving 1/5th of the writes for Parsoid content could be a decent win.

What do you think?

Related Objects
Search...

Event Timeline

• Pchelolo created this task.Oct 2 2018, 9:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 2 2018, 9:39 PM
• mobrovac subscribed.Oct 3 2018, 11:18 AM

I agree. On the one hand, section offsets are not nearly enough requested to warrant their own table, and on the other, RB workers are currently under-utilised so we can handle the extra CPU needed to extract them from data-parsoid.

Eevans awarded a token.Oct 3 2018, 1:06 PM
• Pchelolo added a project: good first task.Oct 3 2018, 2:47 PM

This might be a bit too advanced to award the good first task tag, but at least it's very straightforward and makes good exposure to the convoluted storage semantics™ and the hell hole of parsoid.js module, so I will tag it.

Stashbot mentioned this in T207904: RESTBase doesn't check title normalization for wiktionary.Nov 5 2018, 8:58 PM
Stashbot subscribed.

Mentioned in SAL (#wikimedia-operations) [2018-11-05T20:58:22Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@5b8ad3c]: Update deps, removed sections table, T207904 T206048 T207324

Stashbot mentioned this in T207324: RESTBase adds duplicated `accept-language` entry in Vary header.Nov 5 2018, 8:58 PM
Stashbot added a comment.Nov 5 2018, 9:10 PM

Mentioned in SAL (#wikimedia-operations) [2018-11-05T21:10:37Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@5b8ad3c]: Update deps, removed sections table, T207904 T206048 T207324 (duration: 12m 15s)

Stashbot added a comment.Nov 5 2018, 9:14 PM

Mentioned in SAL (#wikimedia-operations) [2018-11-05T21:14:12Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@5b8ad3c]: Update deps, removed sections table, T207904 T206048 T207324 take 2

Stashbot added a comment.Nov 5 2018, 9:23 PM

Mentioned in SAL (#wikimedia-operations) [2018-11-05T21:23:30Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@5b8ad3c]: Update deps, removed sections table, T207904 T206048 T207324 take 2 (duration: 09m 18s)

• Clarakosi added a project: User-Clarakosi.Nov 8 2018, 6:12 PM
Eevans closed subtask T208616: Drop section-offsets keyspaces as Resolved.Nov 15 2018, 11:38 PM
• Pchelolo closed this task as Resolved.Nov 16 2018, 12:13 AM
• Pchelolo claimed this task.
• Pchelolo edited projects, added Services (done); removed Services (later).
• Pchelolo added a subscriber: • Clarakosi.

Yay!!! Since the keyspaces have been actually deleted, we can close the task now. Congrats @Clarakosi

• Clarakosi added a comment.Nov 16 2018, 2:38 PM

@Pchelolo Thanks!

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL