Change Details

RevisionStore does not provide a way to access page content (or slot meta-data) in bulk. Code that has been using the 'text' for getQueryInfo() for bulk access to page content can no longer be used, because now there can be multiple slots for each revision. As a replacement, we need a way to construct a list if RevisionRecord objects in a way that optimizes the query for the associated information from the slots and content tables. **Design draft:** (rewritten 2019-08-20 after discussion with Anomie on this ticket) Introduce a new method in RevisionStore: `RevisionStore::newRevisionsFromBatch( $rows, array $options = [] ): RevisionRecord[]`. Contract: Construct a RevisionRecord instance for each row in $rows, and return them as an associative array indexed by revision ID. Signature: * `$rows` is a Traversable of anonymous objects representing rows from a query result. Each row is expected to conform to the requirements of `RevisionStore::newRevisionFromRow()`. In practice, `$rows` is typically an IResultWrapper containing the result of a query that was based on `RevisionStore::getQueryInfo`. * `$options` is an associative array supporting the following options: ** `'slots'`: whether meta-data about slots (and slot content) should be loaded immediately, instead of relying on the lazy loading mechanism in RevisionStoreRecord. This allows all the meta-data for all slots to be loaded in a single query. The value is either //truthy// to indicate that all slot should be preloaded, or //falsy// to indicate that no slot meta data should be preloaded, or a list (array) or slot role names, to indicate for which slots the meta data should be preloaded. The default is //false//. ** `'content'`: whether the actual content blobs should be preloaded, instead of relying on lazy loading as usual. The value is //truthy// or //falsy//, default is //false//. Only applies to slots which have been selected for preloading via the `'slots'` option. This option has no effect until `BlobStore` also gets a batch interface (TBD: file a ticket for that). * returns an associative array of Revision(Store)Records, indexed by revision ID. Implementation idea: * At first, just loop over `$rows` and call `RevisionStore::newRevisionFromRow` on each row. This would be the baseline version of the method. This already satisfies the contract, but passes on the opportunity to optimize by preloading. * In the next iteration, if preloading of slot meta-data is requested: ** first pull out the revision ID from each row in `"$rows"`. Remember the row in an associative array indexed by revision ID ** Run a query that selects all the requested slot (and slot content) meta-data; let's call the result of that query `$slots`. ** Loop through the result set, grouping together slot meta-data belonging to the same revision. Use `RevisionStore::newRevisionFromRowAndSlots` to construct a RevisionStoreRecord from that plus the respective row from $rows. * In a following iteration, use a batch query to also preload the Content objects for each slot, by content ID. Details TBD. * Old stale patch implementing something similar for use in `WikiExporter` https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513073 * WikiExporter should be changed to use the new system as well. Doing that should validate the approach and the interface.

RevisionStore does not provide a way to access page content (or slot meta-data) in bulk. Code that has been using the 'text' for getQueryInfo() for bulk access to page content can no longer be used, because now there can be multiple slots for each revision. As a replacement, we need a way to construct a list if RevisionRecord objects in a way that optimizes the query for the associated information from the slots and content tables. **Design draft:** (rewritten 2019-08-20 after discussion with Anomie on this ticket) Introduce a new method in RevisionStore: `RevisionStore::newRevisionsFromBatch( $rows, array $options = [] ): RevisionRecord[]`. Contract: Construct a RevisionRecord instance for each row in $rows, and return them as an associative array indexed by revision ID. Signature: * `$rows` is a Traversable of anonymous objects representing rows from a query result. Each row is expected to conform to the requirements of `RevisionStore::newRevisionFromRow()`. In practice, `$rows` is typically an IResultWrapper containing the result of a query that was based on `RevisionStore::getQueryInfo`. * `$options` is an associative array supporting the following options: ** `'slots'`: whether meta-data about slots (and slot content) should be loaded immediately, instead of relying on the lazy loading mechanism in RevisionStoreRecord. This allows all the meta-data for all slots to be loaded in a single query. The value is either //truthy// to indicate that all slot should be preloaded, or //falsy// to indicate that no slot meta data should be preloaded, or a list (array) or slot role names, to indicate for which slots the meta data should be preloaded. The default is //false//. ** `'content'`: whether the actual content blobs should be preloaded, instead of relying on lazy loading as usual. The value is //truthy// or //falsy//, default is //false//. Only applies to slots which have been selected for preloading via the `'slots'` option. This option has no effect until `BlobStore` also gets a batch interface (see {T230834}). * returns an associative array of Revision(Store)Records, indexed by revision ID. Implementation idea: * At first, just loop over `$rows` and call `RevisionStore::newRevisionFromRow` on each row. This would be the baseline version of the method. This already satisfies the contract, but passes on the opportunity to optimize by preloading. * In the next iteration, if preloading of slot meta-data is requested: ** first pull out the revision ID from each row in `"$rows"`. Remember the row in an associative array indexed by revision ID ** Run a query that selects all the requested slot (and slot content) meta-data; let's call the result of that query `$slots`. ** Loop through the result set, grouping together slot meta-data belonging to the same revision. Use `RevisionStore::newRevisionFromRowAndSlots` to construct a RevisionStoreRecord from that plus the respective row from $rows. * In a following iteration, use a batch query to also preload the Content objects for each slot, by content ID. Details TBD, see T230834. * Old stale patch implementing something similar for use in `WikiExporter` https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513073 * WikiExporter should be changed to use the new system as well. Doing that should validate the approach and the interface.

RevisionStore does not provide a way to access page content (or slot meta-data) in bulk. Code that has been using the 'text' for getQueryInfo() for bulk access to page content can no longer be used, because now there can be multiple slots for each revision. As a replacement, we need a way to construct a list if RevisionRecord objects in a way that optimizes the query for the associated information from the slots and content tables. **Design draft:** (rewritten 2019-08-20 after discussion with Anomie on this ticket) Introduce a new method in RevisionStore: `RevisionStore::newRevisionsFromBatch( $rows, array $options = [] ): RevisionRecord[]`. Contract: Construct a RevisionRecord instance for each row in $rows, and return them as an associative array indexed by revision ID. Signature: * `$rows` is a Traversable of anonymous objects representing rows from a query result. Each row is expected to conform to the requirements of `RevisionStore::newRevisionFromRow()`. In practice, `$rows` is typically an IResultWrapper containing the result of a query that was based on `RevisionStore::getQueryInfo`. * `$options` is an associative array supporting the following options: ** `'slots'`: whether meta-data about slots (and slot content) should be loaded immediately, instead of relying on the lazy loading mechanism in RevisionStoreRecord. This allows all the meta-data for all slots to be loaded in a single query. The value is either //truthy// to indicate that all slot should be preloaded, or //falsy// to indicate that no slot meta data should be preloaded, or a list (array) or slot role names, to indicate for which slots the meta data should be preloaded. The default is //false//. ** `'content'`: whether the actual content blobs should be preloaded, instead of relying on lazy loading as usual. The value is //truthy// or //falsy//, default is //false//. Only applies to slots which have been selected for preloading via the `'slots'` option. This option has no effect until `BlobStore` also gets a batch interface (TBD: file a ticket for thatsee {T230834}). * returns an associative array of Revision(Store)Records, indexed by revision ID. Implementation idea: * At first, just loop over `$rows` and call `RevisionStore::newRevisionFromRow` on each row. This would be the baseline version of the method. This already satisfies the contract, but passes on the opportunity to optimize by preloading. * In the next iteration, if preloading of slot meta-data is requested: ** first pull out the revision ID from each row in `"$rows"`. Remember the row in an associative array indexed by revision ID ** Run a query that selects all the requested slot (and slot content) meta-data; let's call the result of that query `$slots`. ** Loop through the result set, grouping together slot meta-data belonging to the same revision. Use `RevisionStore::newRevisionFromRowAndSlots` to construct a RevisionStoreRecord from that plus the respective row from $rows. * In a following iteration, use a batch query to also preload the Content objects for each slot, by content ID. Details TBD, see T230834. * Old stale patch implementing something similar for use in `WikiExporter` https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513073 * WikiExporter should be changed to use the new system as well. Doing that should validate the approach and the interface.