I guess there is no harm in writing this down but it happens even without a policy. Why would anyone not use service abstraction layers?
RedisConnectionPool and BagOStuff are at very different levels of abstraction, though.

What about higher abstraction layers like the XxxStore classes? Do we generally expect non-legacy code to use that?

In T215465#4970630, @Tgr wrote:

I guess there is no harm in writing this down but it happens even without a policy. Why would anyone not use service abstraction layers?
RedisConnectionPool and BagOStuff are at very different levels of abstraction, though.

What about higher abstraction layers like the XxxStore classes? Do we generally expect non-legacy code to use that?

Yes, I would very much encourage all new code to bind to the narrowest possible interface with the most concrete possible contract. Using BagOStuff directly is bad, because the different implementations have very different contracts, and most calling code does not use most of the functionality it exposes. To go even further, directly querying database tables is bad, because it spreads knowledge about the schema through the code base.

But that's beyond the scope of this proposal, and it's hard to clearly state in a way that is not ambiguous.

The point of this proposal, as far as I understand it, is to ensure that application logic does not bind to the specifics of a storage mechanism in terms of connecting and authenticating. The only way to make this simple and reliable is to enumerate the available storage mechanisms that application logic and extensions are allowed to use; if there is need to use another, an appropriate abstraction layer would have to be added first, to the code base and to the policy.

I don't think this kind of recommendation ("require use of storage abstraction layers") fits in a development policy, it seems more like a guideline.

The stuff that Daniel said about:

...encourage all new code to bind to the narrowest possible interface with the most concrete possible contract.

That seems more like something that fits in the development policy.

Though I don't really follow the part about:

Using BagOStuff directly is bad, because the different implementations have very different contracts, and most calling code does not use most of the functionality it exposes

BagOStuff is broken in so many ways (different implementations have different behavior for lookup misses, atomicity, consistency, even the allowed key space used to be different although maybe that's fixed by now) but that needs to be fixed in core, not policy-ed around. People will use what they have.

In T215465#4970901, @daniel wrote:

The point of this proposal, as far as I understand it, is to ensure that application logic does not bind to the specifics of a storage mechanism in terms of connecting and authenticating.

Which never happened so far, as far as I know. I imagine this was inspired by extensions using low-level abstractions (e.g. GettingStarted using RedisConnectionPool instead of BagOStuff or whatever) but that wouldn't be prevented by the amendment as stated.

Nothing against this, my only suggestion would be to maybe rephrase or stress the idea of "do not duplicate abstraction layers". Fix existing ones, or replace them, but don't maintain 2 in parallel, as otherwise they are not really "abstraction". If you want to replace one, you have to demonstrate that it is absolutely required, and the deprecation and migration work is worth it.

Triaging into the backlog as the initial stage is to gain more information from stakeholders on whether this is desirable and what the motivations for doing so would be (Define a problem statement and objective).

Does it cover schema and schema changes? We are currently at middle of migrating schema and schema changes in core (T191231) but I doubt extensions be able to pick that up any time soon, making them use the abstract schema right now doesn't make much sense to me but it can be part of the policy in two full releases (for example)
Also, is schema and schema change needs to follow the same abstractions as others? I'm working on a Doctrine-DBAL-based solution because mediawiki's DBAL doesn't give me "ALTER TABLE" abstractions.

I think what would be desirable, from the point of view of a developer, to know they can use some specific abstractions, and which ones they should use.

So for example:

• If you need a relational datastore which supports ACID transactions and SQL syntax, the right interface to do so in MediaWiki is Foo
• If you need to be able to write key-value data and you don't need persistence or strict consistency, you should use interface Bar

These interfaces should:

• clearly offer specific guarantees in terms of consistency, availability, durability and multi-master write ability (among other things)
• not be tied to a specific technology (so, as @Tgr correctly noticed, I don't think RedisConnectionPool qualifies)

This would allow the developers to write code being sure of what promises they're expecting, and the actual implementation of the interface can change technology in a deployment without huge issues.

I think the main question we need to answer first is - what classes of storage patterns are currently supported by MediaWiki? How do we map the current abstractions (at any level) to such patterns?

IRC meeting hosted on this on October 23rd:

• Minutes:

https://tools.wmflabs.org/meetbot/wikimedia-office/2019/wikimedia-office.2019-10-23-21.00.html

• Log:

https://tools.wmflabs.org/meetbot/wikimedia-office/2019/wikimedia-office.2019-10-23-21.00.log.html

This task has no active project tags apart from one "radar" tag, so it is lingering. Should this remain open? If yes, please add an active project tag - thanks!

It is blocked on T283277.