Change Details

**Introduction** During Wikibase local development we noticed something odd and slow 1) Create some new wikibase items 2) create a final wikibase item that refers to all of the other items you just wrote 3) `derivedDataUpdater->prepareUpdate` and `derivedDataUpdater->doUpdates` are called, triggering parser output generation at the end of the same request edit request (which in wikibase ends up doing some amount of work) **Problem** Generation of HTML ends up being the "expensive" part of ParserOutput, as it needs to load data from a whole collection of other entities. There is secondary storage and caching etc in place, but ideally, when not needed, we would not do this extra work. And if it is needed for some reason, we would not do it pre send in the API edit call. In many cases this HTML for the ParserOutput is not used immediately after the API request, as most edits on Wikidata.org are made by bots. Even for edits made by users, post edit they will not normally reload the page, as editing generally happens in JS with on page elements changing instead. For Wikibase 3rd party users that want to do large bulk imports of data this is an even more pressing issue, as they will often have less resource, speed, caching etc, and may not even have parser caching enabled, but post API request to edit a wikidata entity, they application will still go and generate this possible not needed html parser output. **Other details** For search indexing we already generate parser output with the `generate-html => false` hint as part of T239931. We already have a split parser cache, but the canonical parser output is used for en page views, for example: - canonical `wikidatawiki:pcache:idhash:3369-0!termboxVersion=1!wb=3` - other `wikidatawiki:pcache:idhash:3369-0!termboxVersion=1!userlang=pt!wb=3` **Possible solution** If MediaWiki could ask the content type if html should be generated for it or not post edit, then we would be able to say that a `generate-html => false` hint should be used for Wikibase content in most cases. When such a hint is used to generate content, we could add a similar hack to EntityContent::getParserOutput for when no html is generated https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/c566765494f366c06b56a4ae3a2257d378b93222/repo/includes/Content/EntityContent.php#168 We already have a "hack" or 2 to change when parser output is or isn't cached, such as https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/c566765494f366c06b56a4ae3a2257d378b93222/repo/includes/Content/EntityContent.php#180 ``` if($generateHtml = true){ $out->updateCacheExpiry( 0 ); } ``` We could then either: 1) Just do not generate this ParserOutput and wait for a user to trigger the render on page view? 2) Send a job to the job queue to generate it if we do want it? We could also do other things such as only pre render very expensive pages (100+ statements etc) **Predicted impact** All Wikibase content edits would do less work and be faster. Memcached (for terms caching) would likely have a slightly lower load, as would s8 terms related tables, which are used during parser output generation. Currently per https://grafana.wikimedia.org/d/FxKUKqUik/wikibase-parseroutputgenerator?orgId=1 item parser output it generated roughly 2.5-3k times per minuite. Roughly 700-1k of those per minute likely come from edits, **Acceptance Criteria 🏕️🌟** [] Don't generate HTML parser output (and cache) on edit [] Monitor 1) Wikibase Parser Cache Generation 2) Wikibase save times 3) Wikibase edge entity load times [] Consider backporting to the next wikibase release (REL1_36 branch)

**Introduction** During Wikibase local development we noticed something odd and slow 1) Create some new wikibase items 2) create a final wikibase item that refers to all of the other items you just wrote 3) `derivedDataUpdater->prepareUpdate` and `derivedDataUpdater->doUpdates` are called, triggering parser output generation at the end of the same request edit request (which in wikibase ends up doing some amount of work) **Problem** Generation of HTML ends up being the "expensive" part of ParserOutput, as it needs to load data from a whole collection of other entities. There is secondary storage and caching etc in place, but ideally, when not needed, we would not do this extra work. And if it is needed for some reason, we would not do it pre send in the API edit call. In many cases this HTML for the ParserOutput is not used immediately after the API request, as most edits on Wikidata.org are made by bots. Even for edits made by users, post edit they will not normally reload the page, as editing generally happens in JS with on page elements changing instead. For Wikibase 3rd party users that want to do large bulk imports of data this is an even more pressing issue, as they will often have less resource, speed, caching etc, and may not even have parser caching enabled, but post API request to edit a wikidata entity, they application will still go and generate this possible not needed html parser output. **Other details** For search indexing we already generate parser output with the `generate-html => false` hint as part of T239931. We already have a split parser cache, but the canonical parser output is used for en page views, for example: - canonical `wikidatawiki:pcache:idhash:3369-0!termboxVersion=1!wb=3` - other `wikidatawiki:pcache:idhash:3369-0!termboxVersion=1!userlang=pt!wb=3` **Possible solution** If MediaWiki could ask the content type if html should be generated for it or not post edit, then we would be able to say that a `generate-html => false` hint should be used for Wikibase content in most cases. When such a hint is used to generate content, we could add a similar hack to EntityContent::getParserOutput for when no html is generated https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/c566765494f366c06b56a4ae3a2257d378b93222/repo/includes/Content/EntityContent.php#168 We already have a "hack" or 2 to change when parser output is or isn't cached, such as https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/c566765494f366c06b56a4ae3a2257d378b93222/repo/includes/Content/EntityContent.php#180 ``` if($generateHtml = true){ $out->updateCacheExpiry( 0 ); } ``` We could then either: 1) Just do not generate this ParserOutput and wait for a user to trigger the render on page view? 2) Send a job to the job queue to generate it if we do want it? We could also do other things such as only pre render very expensive pages (100+ statements etc) **Predicted impact** All Wikibase content edits would do less work and be faster. Memcached (for terms caching) would likely have a slightly lower load, as would s8 terms related tables, which are used during parser output generation. Currently per https://grafana.wikimedia.org/d/FxKUKqUik/wikibase-parseroutputgenerator?orgId=1 item parser output it generated roughly 2.5-3k times per minuite. Roughly 700-1k of those per minute likely come from edits, **Acceptance Criteria 🏕️🌟** [] HTML parser output is not generated or cached during an edit [] While the change is being deployed monitor 1) Wikibase Parser Cache Generation 2) Wikibase save times 3) Wikibase edge entity load times [] Consider backporting the change to the next Wikibase release (REL1_36 branch)

**Introduction** During Wikibase local development we noticed something odd and slow 1) Create some new wikibase items 2) create a final wikibase item that refers to all of the other items you just wrote 3) `derivedDataUpdater->prepareUpdate` and `derivedDataUpdater->doUpdates` are called, triggering parser output generation at the end of the same request edit request (which in wikibase ends up doing some amount of work) **Problem** Generation of HTML ends up being the "expensive" part of ParserOutput, as it needs to load data from a whole collection of other entities. There is secondary storage and caching etc in place, but ideally, when not needed, we would not do this extra work. And if it is needed for some reason, we would not do it pre send in the API edit call. In many cases this HTML for the ParserOutput is not used immediately after the API request, as most edits on Wikidata.org are made by bots. Even for edits made by users, post edit they will not normally reload the page, as editing generally happens in JS with on page elements changing instead. For Wikibase 3rd party users that want to do large bulk imports of data this is an even more pressing issue, as they will often have less resource, speed, caching etc, and may not even have parser caching enabled, but post API request to edit a wikidata entity, they application will still go and generate this possible not needed html parser output. **Other details** For search indexing we already generate parser output with the `generate-html => false` hint as part of T239931. We already have a split parser cache, but the canonical parser output is used for en page views, for example: - canonical `wikidatawiki:pcache:idhash:3369-0!termboxVersion=1!wb=3` - other `wikidatawiki:pcache:idhash:3369-0!termboxVersion=1!userlang=pt!wb=3` **Possible solution** If MediaWiki could ask the content type if html should be generated for it or not post edit, then we would be able to say that a `generate-html => false` hint should be used for Wikibase content in most cases. When such a hint is used to generate content, we could add a similar hack to EntityContent::getParserOutput for when no html is generated https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/c566765494f366c06b56a4ae3a2257d378b93222/repo/includes/Content/EntityContent.php#168 We already have a "hack" or 2 to change when parser output is or isn't cached, such as https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/c566765494f366c06b56a4ae3a2257d378b93222/repo/includes/Content/EntityContent.php#180 ``` if($generateHtml = true){ $out->updateCacheExpiry( 0 ); } ``` We could then either: 1) Just do not generate this ParserOutput and wait for a user to trigger the render on page view? 2) Send a job to the job queue to generate it if we do want it? We could also do other things such as only pre render very expensive pages (100+ statements etc) **Predicted impact** All Wikibase content edits would do less work and be faster. Memcached (for terms caching) would likely have a slightly lower load, as would s8 terms related tables, which are used during parser output generation. Currently per https://grafana.wikimedia.org/d/FxKUKqUik/wikibase-parseroutputgenerator?orgId=1 item parser output it generated roughly 2.5-3k times per minuite. Roughly 700-1k of those per minute likely come from edits, **Acceptance Criteria 🏕️🌟** [] Don't generate HTML parser output (andt is not generated or cache) od during an edit [] MWhile the change is being deployed monitor 1) Wikibase Parser Cache Generation 2) Wikibase save times 3) Wikibase edge entity load times [] Consider backporting the change to the next wWikibase release (REL1_36 branch)