Page MenuHomePhabricator
Create Task
Maniphest T333528

Increase memory_limit for jobrunners to $wmgMemoryLimitParsoid
Closed, ResolvedPublic
Actions

Description

In preparation for the anticipated increased load on the jobrunner cluster when WarmParsoidParserCache is enabled, raise jobrunner cluster memory_limit to match settings for parsoid cluster (1400MiB)

Details

SubjectRepoBranchLines +/-
jobrunners: Raise memory_limit to match parsoidoperations/mediawiki-configmaster+3 -1
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 StalledNoneT324931 Clean up open RESTBase related tickets
 In ProgressNoneT262315 <CORE TECHNOLOGY> API Migration & RESTBase Sunset
 OpenNoneT264669 Move VE API from RESTBase into core.
 OpenNoneT265846 Turn RESTBase tests for /html and /transform endpoints into api-testing integration tests
 OpenNoneT267980 Benchmark cached Parsoid HTML endpoints RESTBase vs Core
 ResolveddanielT267981 Implement /revision/{id}/html endpoint in core
 ResolvedWDoranWMFT262597 MediaWiki Developer stores revision HTML in parser cache
 Resolved• PcheloloT269663 Implement parser output caching for /revision/{id}/html endpoint
 ResolvedNoneT267990 Support stashing in page/html and revision/html endpoints in MW core
 ResolvedNoneT267991 Simple per-IP ratelimit for HTML stashing
 DuplicateNoneT308684 Determine directory to place ParsoidStash logic
 ResolveddanielT308743 PageHTMLHandler: add support for getting fully annotated parsoid HTML.
 ResolveddanielT308744 PageHTMLHandler should return different eTag for different output modes
 ResolveddanielT309016 Determine storage requirements for stashing parsoid output for VE edits
 ResolvedDAlangi_WMFT309017 Collect stats on stash and cache usage in the page/html and revision/html endpoints
 InvalidNoneT262572 Use Parsoid-specific Parser Cache for Visual Editor
 InvalidNoneT262605 Parsoid REST API Client gets a speedy result
 OpenNoneT262593 MediaWiki Developer stores different HTML output for different language variants in the parser cache
 DuplicateNoneT262586 MediaWiki Developer stores parse metadata in parser cache
 Resolved• eprodromouT262604 Reader quickly reads a wiki page
 Resolved• PcheloloT268234 Evaluate ETags in core HTML endpoints
 In ProgressNoneT301370 Move transform endpoints from RESTBase to MediaWiki
 ResolveddanielT310467 Remove implementation of the transform endpoint from the parsoid extension
 ResolvedNoneT311477 Remove pagebundle support from public transform endpoint
 OpenNoneT311645 Split the TransformHandler class
 ResolveddanielT311819 Make the transform endpoint match ETags emitted by the page endpoint
 ResolveddanielT311867 Support redirects to private endpoints
 OpenNoneT301371 Preemptively warm caches for Parsoid output
 ResolvedDAlangi_WMFT308588 Do not cache fast parsoid output in ParserCache (exploration)
 ResolvedppelbergT308511 [SPIKE] Determine necessity of edit session continuity during data center switchovers
ResolveddanielT310710 Do not ignore HTTP conditionals in transform REST endpoint.
 ResolvedNoneT310904 Find out if Varnish is messing with ETags, and what to do about it.
 ResolveddanielT310878 Consolidate ParsoidHandler base class with HtmlOutputRendererHelper
 OpenNoneT312916 Parsoid: Migrate 3rd party clients from RESTbase to the MW core API
 OpenNoneT312917 Parsoid: Proxy old RESTbase URLs to the core endpoints
 ResolvedMSantosT314244 Parsoid: make roundtrip-test.js run as a Mocha test in CI
 ResolveddanielT317017 Make the core page/html endpoint return system message content when appropriate
 ResolvedNoneT318393 Parsoid: Maintain context for selser on the server side
 ResolvedNoneT310464 Make use of stashed data-parsoid mapping for html-to-wikitext transformation
 ResolveddanielT310489 Collect statistics on stash read failures
 ResolvedNoneT318395 Parsoid: use rendering from ParserCache for selser if the render ID matches.
 ResolvedNoneT318398 Parsoid: use current revision rendering to enable selser, when no stash key/etag is provided.
 ResolveddanielT318399 HtmlInputTransformHelper: collect statistics on selser context
OpenNoneT344944 Move Parsoid endpoints out of RESTBase
 ResolvedJgiannelosT344945 Disable storage of Parsoid content in RESTbase
 In ProgressNoneT328559 Replace usage of RESTbase parsoid endpoints
 ResolveddanielT320529 Configure VE backend to use Parsoid directly, instead of calling RESTbase
 ResolvedNoneT310376 Make VE Action API use Parsoid directly
 ResolvedNoneT310377 Make parsoid transformation API available internally
 ResolvedmatmarexT314565 ApiVisualEditor: Convert ApiParsoidTrait into a helper object.
 ResolvedPRODUCTION ERRORdanielT320703 VisualEditor can't save pages on de.wikipedia.beta
 ResolveddanielT320704 When VisualEditor saves an edit, it should use the same backend that the HTML was originally loaded from
 ResolvedMSantosT333536 Survey RESTBase services and find which ones accesses Parsoid via RESTBase
 ResolvedDAlangi_WMFT339228 Remove VRS code from ContentTranslation
 ResolvedDAlangi_WMFT339227 Remove VRS code from VisualEditor
 Resolved• R_RanaT338922 Introduce maintenance script for warming cache with Parsoid Outputs
 ResolvedJgiannelosT339865 PCS should use parsoid endpoints in MediaWiki, not RESTbase
 ResolvedPRODUCTION ERRORcscottT356368 Revision endpoint: InvalidArgumentException: ParserOutput does not have a render ID
 ResolvedPRODUCTION ERRORJgiannelosT356369 Missing content-language header on PCS responses
 ResolvedClement_GoubertT356497 Raise mw-api-int replicas for increased load from mobileapps
 ResolvedKartikMistryT344982 Make cxserver call parsoid endpoints on MediaWiki, instead of going through RESTbase
 ResolveddanielT350661 Expose parsoid transformation API from MediaWiki core.
 ResolvedBUG REPORTngkountasT357769 cxserver "fetch segmented page content" API endpoint doesn't work for space-separated multi-word titles
 ResolveddanielT359426 REST: endpoints that return rendered page content should support all content models
 ResolvedcscottT311648 Allow WikitextContentHandler to use Parsoid for rendering wikitext
 ResolvedcscottT300190 Ensure that if 'useParsoid' flag is present in ParserOptions that we use Parsoid to parse the page.
 OpenNoneT310520 Parsoid content not compatible with `index.php?title=` URLs
 ResolvedDAlangi_WMFT327769 ParserOutputAccess should use a separate ParserCache instance for Parsoid output
 ResolvedDAlangi_WMFT311728 Gracefully handle non-wikitext pages when returning HTML from the page endpoints
 ResolveddanielT320534 Put Parsoid output into the ParserCache on every edit
 ResolveddanielT322427 Job for pre-emptive population of the ParserCache with Parsoid output
 ResolveddanielT329366 Enable WarmParsoidParserCache on all wikis
 ResolvedClement_GoubertT333528 Increase memory_limit for jobrunners to $wmgMemoryLimitParsoid
 Resolved• PcheloloT267982 Extract cached Parsoid access to an internal service object
 Resolved• PcheloloT263587 CAPEX for ParserCache for Parsoid
 ResolveddanielT309549 Create end-to-end tests for the page/html and revision/html endpoints.
 ResolveddanielT268205 Store data-parsoid in parsoid ParserCache
 OpenDAlangi_WMFT308424 Determine http cache control and active purging for REST endpoints serving parsoid output

Event Timeline

Clement_Goubert triaged this task as High priority.Mar 30 2023, 8:39 AM
Clement_Goubert created this task.
Clement_Goubert moved this task from Incoming 🐫 to this.quarter 🍕 on the serviceops board.
gerritbot added a comment.Mar 30 2023, 9:05 AM

Change 904463 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/mediawiki-config@master] jobrunners: Raise memory_limit to match parsoid

https://gerrit.wikimedia.org/r/904463

gerritbot added a project: Patch-For-Review.Mar 30 2023, 9:05 AM
Clement_Goubert changed the task status from Open to In Progress.Mar 30 2023, 3:10 PM
Clement_Goubert moved this task from this.quarter 🍕 to Doing 😎 on the serviceops board.
gerritbot added a comment.Apr 6 2023, 9:12 AM

Change 904463 merged by jenkins-bot:

[operations/mediawiki-config@master] jobrunners: Raise memory_limit to match parsoid

https://gerrit.wikimedia.org/r/904463

Stashbot added a comment.Apr 6 2023, 9:12 AM

Mentioned in SAL (#wikimedia-operations) [2023-04-06T09:12:16Z] <cgoubert@deploy2002> Started scap: Backport for [[gerrit:904463|jobrunners: Raise memory_limit to match parsoid (T333528)]]

Stashbot added a comment.Apr 6 2023, 9:13 AM

Mentioned in SAL (#wikimedia-operations) [2023-04-06T09:13:37Z] <cgoubert@deploy2002> cgoubert: Backport for [[gerrit:904463|jobrunners: Raise memory_limit to match parsoid (T333528)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet

Stashbot added a comment.Apr 6 2023, 9:19 AM

Mentioned in SAL (#wikimedia-operations) [2023-04-06T09:19:27Z] <cgoubert@deploy2002> Finished scap: Backport for [[gerrit:904463|jobrunners: Raise memory_limit to match parsoid (T333528)]] (duration: 07m 11s)

Clement_Goubert closed this task as Resolved.Apr 6 2023, 9:19 AM
Maintenance_bot removed a project: Patch-For-Review.Apr 6 2023, 9:30 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits