[EPIC] Ensure Parsoid meets performance criteria for serving read views on the Parsoid cluster
Open, MediumPublic
Actions

Assigned To

None

Authored By

	ssastry
	Jul 24 2020, 9:05 PM

Description

This is just a tracker task for tracking the work needed around Parsoid performance. Subtasks will capture the specific work that needs doing.

Broadly, Performance should meet acceptance criteria wrt (a) memory usage and OOMs (b) parse latency and timeouts (c) compressed HTML size relative to core parser.

https://www.mediawiki.org/wiki/Parsing/Parser_Unification/Performance has more detailed information.

Related Objects
Search...

Status	Subtype	Assigned	Task
Open	Release	None	T84936 Release VisualEditor-MediaWiki as "1.0"
Open		None	T50429 [Epic] Support editing parts of a page in VisualEditor-MediaWiki
Open		None	T54365 Explore performance gains from progressive (JIT?) de-alienation in VisualEditor
Open		None	T174303 Copy-pasting linked ISBN numbers from view mode HTML into VisualEditor inserts wikitext links to Special:BookSources (it should turn them into magic links?)
Open	Feature	None	T54091 The read HTML should have hinting to allow full DOM copying (as opposed to just rich copying) from read mode into VE surfaces
Open		None	T55784 [EPIC] Use Parsoid HTML for all page views
Open		None	T258836 [EPIC] Ensure Parsoid meets performance criteria for serving read views on the Parsoid cluster
Open		None	T254508 Reconcile Parsoid memory usage w/ legacy parser and mediawiki core
Resolved		ssastry	T254509 Turn on garbage collection for Parsoid cluster
Resolved		Joe	T230861 PHP 7.2 is very slow on an allocation-intensive benchmark
Declined		None	T257462 Parsoid should not disable GC unless a config var is set
Resolved		None	T254519 Remove array cast workaround from Parsoid
Open		None	T254523 Lint wikitext against Parsoid and legacy parser wikitext metrics
Open		None	T254522 Set appropriate wikitext limits for Parsoid to ensure it doesn't OOM
Open		None	T254521 Collect data and correlate Parsoid memory usage with legacy parser memory usage
Open	BUG REPORT	None	T286355 Templates expanding to mapframe extensions with large bodies exceed wikitext limits
			Restricted Task
Open		None	T78676 Store & load data-mw separately
Resolved		• marcoil	T88290 Update data-mw encoding for <ref> tags to point to the HTML content in the <references /> output rather than duplicating it
Resolved		Catrope	T88650 Support data-mw.body.id for reference contents
Resolved		• marcoil	T88660 Parsoid Cite: Render missing reflists
Resolved		Arlolra	T76518 Errors in Parsoid v2 entry point
Resolved		Arlolra	T54936 Move data-parsoid and possibly data-mw out of the DOM, add uids
Resolved		• GWicke	T50483 Expose a parsoid public web API end point in the MediaWiki API
Resolved		• GWicke	T89481 RESTBase beta release (revision storage / content API)
Resolved		• GWicke	T1228 RESTbase deployment
Resolved		fgiunchedi	T76986 RESTBase production hardware
Resolved		fgiunchedi	T88805 rack and setup restbase production cluster in eqiad
Resolved		• Cmjohnson	T83551 Split memcached in eqiad across multiple racks/rows
Resolved		• Cmjohnson	T82259 mc1016 mgmt not working
Resolved		Joe	T89345 Missing memcached logs
Resolved		fgiunchedi	T89639 restbase1006 faulty disk controller
Resolved		• GWicke	T75955 RESTBase / Parsoid integration - waiting for parsoid deploy
Resolved		• Jdouglas	T88456 On-demand generation of HTML and data-parsoid
Resolved		• GWicke	T88457 html2wt conversion support
Resolved		• GWicke	T88588 Formalize the RESTBase v1 specification
Resolved		Arlolra	T89411 Parsoid v2 API selser might not be working correctly
Resolved		• GWicke	T76370 RESTBase performance testing
Resolved		• mobrovac	T76165 Handle revision deletion and suppression in RESTBase
Resolved		• mobrovac	T87520 Set up update jobs for RESTBase: code done and deployed, to be configured
Resolved		• mobrovac	T88652 Integrate revisions into /page structure. Was: Create a /page/revision/{revision} entry point
Resolved		• mobrovac	T87392 Add property & index on namespace in page_revisions
Duplicate		• Jdouglas	T78214 Documentation location and deployment process
Resolved		Andrew	T88850 Import Cassandra packages for Jessie
Resolved		• GWicke	T89461 trebuchet puppet provider broken on systems without upstart
Resolved		fgiunchedi	T89636 setup internal LVS for restbase eqiad servers
Resolved		fgiunchedi	T78514 Detailed cassandra monitoring: metrics and dashboards done, need to set up alerts
Resolved		fgiunchedi	T93886 Cassandra/CQL query interface monitoring
Resolved		Eevans	T93884 graphs for Cassandra metrics
Resolved		fgiunchedi	T97024 some cassandra metrics sent with invalid values
Resolved		• GWicke	T78194 Public entry point for RESTBase
Resolved		• GWicke	T89567 Load test production cluster
Resolved		fgiunchedi	T89366 Access to restbase / cassandra cluster
Resolved		• mobrovac	T91102 Update / maintain Beta Cluster restbase cluster: Up & working with VE
Resolved		• GWicke	T91188 Need to change data model for >~300 wikis in single cassandra cluster
Resolved		• GWicke	T128231 All known clients of Parsoid HTML (either directly hitting Parsoid or via RESTBase) should provide Accept: headers in their requests
Resolved		• AlexMonk-WMF	T128233 VisualEditor: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Resolved		• santhosh	T128235 CX Server: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Resolved		ssastry	T128236 Flow: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Resolved		• Mholloway	T128237 Mobile Content Service: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Resolved		Jackmcbarn	T128238 EPH Gadget: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Resolved		Renxiaoyi	T128239 Google: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Resolved		cscott	T130685 OCG: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Resolved		Kelson	T130686 Kiwix: Add Accept: header to your HTML requests to RESTBase/Parsoid as per T124365
Open		None	T114413 Support various conversions in Parsoid's pb2pb endpoint
Open		None	T130639 All known clients of Parsoid HTML that require data-mw should fetch data-mw separately (if using RESTBase) or process the data-mw blob in Parsoid's pagebundle API response (if using Parsoid directly)
Open		None	T88623 On Parsoid HTML read pages, VisualEditor should be able to load data-mw from a separate API call and zip it into the DOM
Resolved		Arlolra	T130638 Add data-mw as a separate JSON blob in the pagebundle output of Parsoid's API
Resolved		Arlolra	T91700 Store and unpack data-* attributes when serializing/parsing document fragments throughout the pipeline
Open		None	T130643 Content Translation should load data-mw from a separate API call alongside the body content
Declined		None	T130687 OCG should load data-mw from a separate API call alongside the body content
Open		None	T130689 Google's services should load data-mw from a separate API call to RESTBase
Open		None	T124837 Update Flow for Parsoid changes re data-mw
Declined		None	T125885 Switch Flow storage to store a page bundle blob instead of HTML
Declined		None	T125886 Run migration script (HTML => wikitext => page bundle) in production
Declined		None	T125888 Write migration script (HTML => wikitext => page bundle) in production
Open		None	T125889 Update code (extractors, converters) that uses data-mw to deal with separate components
Open		None	T125890 Figure out how to get page bundle from Flow VE
Invalid		None	T194980 Triage
Invalid		None	T130641 Allocate a storage bucket for Parsoid's data-mw attribute
Open		Arlolra	T270150 Selectors in content.media.less need improvement in terms of performance and stability
Open		None	T309305 Ensure Parsoid HTML served for read views doesn't degrade performance in low-resource contexts
Open		Jgiannelos	T272331 Evaluate and recommend strategies for ensuring Parsoid HTML payload doesn't degrade performance in low-resource contexts.
Resolved		Jgiannelos	T309064 Analyze rt-testing data and identify pages whose Parsoid HTML sizes are "outside limits"
Resolved		Jgiannelos	T309304 Collect per-attribute stats during Parsoid HTML attribute stripping

Event Timeline

ssastry created this task.Jul 24 2020, 9:05 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 24 2020, 9:05 PM

ssastry triaged this task as Medium priority.Jul 24 2020, 9:05 PM

ssastry added a subtask: T254508: Reconcile Parsoid memory usage w/ legacy parser and mediawiki core.Jul 24 2020, 9:23 PM

ssastry added a parent task: T55784: [EPIC] Use Parsoid HTML for all page views.Jul 24 2020, 9:28 PM

ssastry added a subtask: T78676: Store & load data-mw separately.Jul 24 2020, 9:36 PM

ssastry moved this task from Needs Triage to Following Quarter on the Parsoid board.Jul 24 2020, 9:58 PM

Arlolra mentioned this in T51097: Use figure and figcaption HTML5 elements when possible.Dec 15 2020, 1:14 AM

Moved team tags from Subscribers to Projects as otherwise they remain invisible.

Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.Dec 15 2020, 1:32 AM

Arlolra added a subtask: T270150: Selectors in content.media.less need improvement in terms of performance and stability.Dec 15 2020, 2:17 AM

I / we should probably break the task chain from some of those old restbase epics since it makes the tree very noisy and cluttery. I dropped a couple but need to figure out what the "best practices" are around editing dependencies on old resolved task chains.

• holger.knust moved this task from Inbox to Tracking/Watching on the Platform Engineering board.Dec 15 2020, 2:24 PM

ssastry added a subtask: T272331: Evaluate and recommend strategies for ensuring Parsoid HTML payload doesn't degrade performance in low-resource contexts..Jan 19 2021, 12:40 AM

Arlolra moved this task from Following Quarter to Performance on the Parsoid board.Aug 3 2021, 5:55 PM

ssastry added a project: Parsoid-Read-Views (Phase 3 - Main namespace of officewiki / mediawiki.org renders with Parsoid).Oct 4 2021, 7:58 PM

ssastry removed a subtask: T272331: Evaluate and recommend strategies for ensuring Parsoid HTML payload doesn't degrade performance in low-resource contexts..May 26 2022, 12:37 PM

TheresNoTime reopened subtask T309305: Ensure Parsoid HTML served for read views doesn't degrade performance in low-resource contexts as Open.May 26 2022, 9:42 PM

Krinkle removed a project: Performance-Team (Radar).Aug 18 2023, 8:13 PM