Page MenuHomePhabricator
Create Task
Maniphest T258836

[EPIC] Ensure Parsoid meets performance criteria for serving read views on the Parsoid cluster
Open, MediumPublic
Actions

Description

This is just a tracker task for tracking the work needed around Parsoid performance. Subtasks will capture the specific work that needs doing.

Broadly, Performance should meet acceptance criteria wrt (a) memory usage and OOMs (b) parse latency and timeouts (c) compressed HTML size relative to core parser.

https://www.mediawiki.org/wiki/Parsing/Parser_Unification/Performance has more detailed information.

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenReleaseNoneT84936 Release VisualEditor-MediaWiki as "1.0"
 OpenNoneT50429 [Epic] Support editing parts of a page in VisualEditor-MediaWiki
 OpenNoneT54365 Explore performance gains from progressive (JIT?) de-alienation in VisualEditor
 OpenNoneT396539 Introduce a way to edit a span of text from within read mode
 OpenNoneT385558 Improve copy and paste from read mode to VE
 OpenNoneT174303 Copy-pasting linked ISBN numbers from view mode HTML into VisualEditor inserts wikitext links to Special:BookSources (it should turn them into magic links?)
 OpenFeatureNoneT54091 The read HTML should have hinting to allow full DOM copying (as opposed to just rich copying) from read mode into VE surfaces
 OpenNoneT55784 [EPIC] Use Parsoid HTML for all page views
 OpenNoneT258836 [EPIC] Ensure Parsoid meets performance criteria for serving read views on the Parsoid cluster
 OpenNoneT254508 Reconcile Parsoid memory usage w/ legacy parser and mediawiki core
 ResolvedssastryT254509 Turn on garbage collection for Parsoid cluster
 ResolvedJoeT230861 PHP 7.2 is very slow on an allocation-intensive benchmark
 DeclinedNoneT257462 Parsoid should not disable GC unless a config var is set
 ResolvedNoneT254519 Remove array cast workaround from Parsoid
 OpenNoneT254523 Lint wikitext against Parsoid and legacy parser wikitext metrics
 OpenNoneT254522 Set appropriate wikitext limits for Parsoid to ensure it doesn't OOM
 OpenNoneT254521 Collect data and correlate Parsoid memory usage with legacy parser memory usage
 DuplicateBUG REPORTNoneT286355 Templates expanding to mapframe extensions with large bodies exceed wikitext limits
 ResolvedssastryT392261 Investigate crashers (out of memory, timeouts)
Restricted Task
 DuplicateNoneT385716 Parsoid's "include size" accounting differs from legacy
 OpenNoneT78676 Store & load data-mw separately
 Resolved• marcoilT88290 Update data-mw encoding for <ref> tags to point to the HTML content in the <references /> output rather than duplicating it
 ResolvedCatropeT88650 Support data-mw.body.id for reference contents
 Resolved• marcoilT88660 Parsoid Cite: Render missing reflists
 ResolvedArlolraT76518 Errors in Parsoid v2 entry point
 ResolvedArlolraT54936 Move data-parsoid and possibly data-mw out of the DOM, add uids
 Resolved• GWickeT50483 Expose a parsoid public web API end point in the MediaWiki API
 Resolved• GWickeT89481 RESTBase beta release (revision storage / content API)
 Resolved• GWickeT1228 RESTbase deployment
 ResolvedfgiunchediT76986 RESTBase production hardware
 ResolvedfgiunchediT88805 rack and setup restbase production cluster in eqiad
 Resolved• CmjohnsonT83551 Split memcached in eqiad across multiple racks/rows
 Resolved• CmjohnsonT82259 mc1016 mgmt not working
 ResolvedJoeT89345 Missing memcached logs
 ResolvedfgiunchediT89639 restbase1006 faulty disk controller
 Resolved• GWickeT75955 RESTBase / Parsoid integration - waiting for parsoid deploy
 Resolved• JdouglasT88456 On-demand generation of HTML and data-parsoid
 Resolved• GWickeT88457 html2wt conversion support
 Resolved• GWickeT88588 Formalize the RESTBase v1 specification
 ResolvedArlolraT89411 Parsoid v2 API selser might not be working correctly
 Resolved• GWickeT76370 RESTBase performance testing
 Resolved• mobrovacT76165 Handle revision deletion and suppression in RESTBase
 Resolved• mobrovacT87520 Set up update jobs for RESTBase: code done and deployed, to be configured
 Resolved• mobrovacT88652 Integrate revisions into /page structure. Was: Create a /page/revision/{revision} entry point
 Resolved• mobrovacT87392 Add property & index on namespace in page_revisions
 Duplicate• JdouglasT78214 Documentation location and deployment process
 ResolvedAndrewT88850 Import Cassandra packages for Jessie
 Resolved• GWickeT89461 trebuchet puppet provider broken on systems without upstart
 ResolvedfgiunchediT89636 setup internal LVS for restbase eqiad servers
 ResolvedfgiunchediT78514 Detailed cassandra monitoring: metrics and dashboards done, need to set up alerts
 ResolvedfgiunchediT93886 Cassandra/CQL query interface monitoring
 ResolvedEevansT93884 graphs for Cassandra metrics
 ResolvedfgiunchediT97024 some cassandra metrics sent with invalid values
 Resolved• GWickeT78194 Public entry point for RESTBase
 Resolved• GWickeT89567 Load test production cluster
 ResolvedfgiunchediT89366 Access to restbase / cassandra cluster
 Resolved• mobrovacT91102 Update / maintain Beta Cluster restbase cluster: Up & working with VE
 Resolved• GWickeT91188 Need to change data model for >~300 wikis in single cassandra cluster
 Resolved• GWickeT128231 All known clients of Parsoid HTML (either directly hitting Parsoid or via RESTBase) should provide Accept: headers in their requests
 Resolved• AlexMonk-WMFT128233 VisualEditor: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedsanthoshT128235 CX Server: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedssastryT128236 Flow: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 Resolved• MhollowayT128237 Mobile Content Service: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedJackmcbarnT128238 EPH Gadget: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedRenxiaoyiT128239 Google: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedcscottT130685 OCG: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 ResolvedKelsonT130686 Kiwix: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
 OpenNoneT114413 Support various conversions in Parsoid's pb2pb endpoint
 OpenNoneT130639 All known clients of Parsoid HTML that require data-mw should fetch data-mw separately (if using RESTBase) or process the data-mw blob in Parsoid's pagebundle API response (if using Parsoid directly)
 OpenNoneT88623 On Parsoid HTML read pages, VisualEditor should be able to load data-mw from a separate API call and zip it into the DOM
 ResolvedArlolraT130638 Add data-mw as a separate JSON blob in the pagebundle output of Parsoid's API
 ResolvedArlolraT91700 Store and unpack data-* attributes when serializing/parsing document fragments throughout the pipeline
 DeclinedNoneT130643 Content Translation should load data-mw from a separate API call alongside the body content
 DeclinedNoneT130687 OCG should load data-mw from a separate API call alongside the body content
 OpenNoneT130689 Google's services should load data-mw from a separate API call to RESTBase
 OpenNoneT124837 Update Flow for Parsoid changes re data-mw
 DeclinedNoneT125885 Switch Flow storage to store a page bundle blob instead of HTML
 DeclinedNoneT125886 Run migration script (HTML => wikitext => page bundle) in production
 DeclinedNoneT125888 Write migration script (HTML => wikitext => page bundle) in production
 OpenNoneT125889 Update code (extractors, converters) that uses data-mw to deal with separate components
 OpenNoneT125890 Figure out how to get page bundle from Flow VE
 InvalidNoneT194980 Triage
 InvalidNoneT130641 Allocate a storage bucket for Parsoid's data-mw attribute
ResolvedABreault-WMFT270150 Selectors in content.media.less need improvement in terms of performance and stability
 OpenNoneT309305 Ensure Parsoid HTML served for read views doesn't degrade performance in low-resource contexts
 OpenNoneT272331 Evaluate and recommend strategies for ensuring Parsoid HTML payload doesn't degrade performance in low-resource contexts.
 ResolvedJgiannelosT309064 Analyze rt-testing data and identify pages whose Parsoid HTML sizes are "outside limits"
 ResolvedJgiannelosT309304 Collect per-attribute stats during Parsoid HTML attribute stripping

Event Timeline

ssastry created this task.Jul 24 2020, 9:05 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 24 2020, 9:05 PM
ssastry triaged this task as Medium priority.Jul 24 2020, 9:05 PM
ssastry added a subtask: T254508: Reconcile Parsoid memory usage w/ legacy parser and mediawiki core.Jul 24 2020, 9:23 PM
ssastry added a parent task: T55784: [EPIC] Use Parsoid HTML for all page views.Jul 24 2020, 9:28 PM
ssastry added a subtask: T78676: Store & load data-mw separately.Jul 24 2020, 9:36 PM
ssastry moved this task from Needs Triage to Following Quarter on the Parsoid board.Jul 24 2020, 9:58 PM
Arlolra mentioned this in T51097: Use figure and figcaption HTML5 elements when possible.Dec 15 2020, 1:14 AM
Krinkle edited projects, added Performance-Team (Radar), Platform Engineering; removed Performance Issue.Dec 15 2020, 1:31 AM
Krinkle removed subscribers: Platform Engineering, Performance-Team.
Krinkle subscribed.

Moved team tags from Subscribers to Projects as otherwise they remain invisible.

Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.Dec 15 2020, 1:32 AM
Arlolra added a subtask: T270150: Selectors in content.media.less need improvement in terms of performance and stability.Dec 15 2020, 2:17 AM
ssastry added a comment.Dec 15 2020, 3:03 AM

I / we should probably break the task chain from some of those old restbase epics since it makes the tree very noisy and cluttery. I dropped a couple but need to figure out what the "best practices" are around editing dependencies on old resolved task chains.

• holger.knust moved this task from Inbox to Tracking/Watching on the Platform Engineering board.Dec 15 2020, 2:24 PM
ssastry added a subtask: T272331: Evaluate and recommend strategies for ensuring Parsoid HTML payload doesn't degrade performance in low-resource contexts..Jan 19 2021, 12:40 AM
Arlolra moved this task from Following Quarter to Performance on the Parsoid board.Aug 3 2021, 5:55 PM
ssastry added a project: Parsoid-Read-Views (Phase 3 - Main namespace of officewiki / mediawiki.org renders with Parsoid).Oct 4 2021, 7:58 PM
ssastry removed a subtask: T272331: Evaluate and recommend strategies for ensuring Parsoid HTML payload doesn't degrade performance in low-resource contexts..May 26 2022, 12:37 PM
TheresNoTime reopened subtask T309305: Ensure Parsoid HTML served for read views doesn't degrade performance in low-resource contexts as Open.May 26 2022, 9:42 PM
Krinkle removed a project: Performance-Team (Radar).Aug 18 2023, 8:13 PM
MSantos closed subtask T270150: Selectors in content.media.less need improvement in terms of performance and stability as Resolved.Nov 13 2025, 3:32 PM
Restricted Application added a project: Epic. · View Herald TranscriptNov 13 2025, 3:32 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits