Pinging some folks who have in the past expressed interest in this idea: @Fjalapeno @WMDE-leszek @Halfak.

It should probably have a mass fetch/store interface too, to avoid the problems every similar service has predictably run into.

In T209044#6088746, @Tgr wrote:

It should probably have a mass fetch/store interface too, to avoid the problems every similar service has predictably run into.

True. Though for the blob store, we have so far not encountered a use case for mass write. Batch reads are needed by some maintenance scripts.

Experimental patch (old, linking in the hope it might be useful for discussion): https://gerrit.wikimedia.org/r/c/mediawiki/core/+/631180

To allow other storage mechanisms to be applied, a wrapper could be used that delegates to the correct ContentStore based on the content model (when writing) and address prefix (when reading).

How would this look in practice? At first I assumed that it could make sense to have each ContentHandler know how to store its corresponding Content (i.e. choosing a ContentStore implementation), so it would have a method that returns either a ContentStore instance, or a class name. But then I wondered if ContentHandler is the right place for that: ContentStore already depends on ContentHandler, plus ContentHandler is already pretty big and might as well be broken down into separate interfaces. Also, we need to handle custom prefixes for read operations.

An alternative could be a ContentStoreRegistry, where custom stores could be registered similar to custom slots, i.e.

$services->addServiceManipulator(
	'ContentStoreRegistry',
	function ( ContentStoreRegistry $registry ) {
		$registry->registerStore(
                    CONTENT_MODEL_MYCONTENT,
                    MY_CONTENT_BLOB_PREFIX,
                    $services->getMyContentContentStore()
                );
	}
);

In T209044#7707580, @Daimona wrote:

To allow other storage mechanisms to be applied, a wrapper could be used that delegates to the correct ContentStore based on the content model (when writing) and address prefix (when reading).

How would this look in practice? At first I assumed that it could make sense to have each ContentHandler know how to store its corresponding Content (i.e. choosing a ContentStore implementation), so it would have a method that returns either a ContentStore instance, or a class name. But then I wondered if ContentHandler is the right place for that: ContentStore already depends on ContentHandler, plus ContentHandler is already pretty big and might as well be broken down into separate interfaces. Also, we need to handle custom prefixes for read operations.

My intention was to have a DispatchingContentStore which can delegate to different implementations based on the content model, or based on the role hint.

In T209044#7707817, @daniel wrote:

My intention was to have a DispatchingContentStore which can delegate to different implementations based on the content model, or based on the role hint.

That would make sense, but how would the DispatchingContentStore know how to associate a model (or prefix) to a specific ContentStore implementation? Which is equivalent to asking: how would an extension register another ContentStore? I assume that the DispatchingContentStore would either act as a registry itself (and perhaps run some hooks to allow customizations), or have a registry injected (and then such a registry would be the one I mentioned in T209044#7707580).

In T209044#7707885, @Daimona wrote:

That would make sense, but how would the DispatchingContentStore know how to associate a model (or prefix) to a specific ContentStore implementation? Which is equivalent to asking: how would an extension register another ContentStore?

I'd suggest an extension.js attribute like we do for REST routes. Alternatively, a factory hook could be used. Either way, I suppose we'd cache a ContentStore instance per combination of slot and model.

(I originally thought "service manipulators" where a good way to do that, but I now think that was a mistake.)

I assume that the DispatchingContentStore would either act as a registry itself (and perhaps run some hooks to allow customizations), or have a registry injected (and then such a registry would be the one I mentioned in T209044#7707580).

Correct.

In T209044#7708856, @daniel wrote:

In T209044#7707885, @Daimona wrote:

That would make sense, but how would the DispatchingContentStore know how to associate a model (or prefix) to a specific ContentStore implementation? Which is equivalent to asking: how would an extension register another ContentStore?

I'd suggest an extension.js attribute like we do for REST routes.

That seems reasonable. One thing I don't like about extension.json (in general) is that you can't use PHP constants, which is probably fine-ish if it's just for the content model, a bit less so if it's also for the prefix. I was also thinking about a more long-term idea of having an extension.json entry for defining new content types that would allow you to register ContentHandlers, ContentStores and any other thing we may want to add in the future. Something like:

"Contents": {
    "MyContent": {
        handler: "MyContentHandler",
        store: "MyContentStore",
        blobprefix: "my-prefix",
        // More things could be added here in the future
    }
}

but this really is beyond the scope of this task.

In T209044#7712531, @Daimona wrote:

I was also thinking about a more long-term idea of having an extension.json entry for defining new content types that would allow you to register ContentHandlers, ContentStores and any other thing we may want to add in the future.

That sounds quite nice!

I've created T301891 about the generic extension.json proposal. In the meanwhile, I'd like to try and finish the experimental ContentStore patch.

Change 631180 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[mediawiki/core@master] EXPERIMENTAL: ContentStore

https://gerrit.wikimedia.org/r/631180