To be more concrete, the scope of this ticket is the Reading Lists functionality. As such, we are not asking whether a Node.JS service can directly read/write from/to the main DB/tables, but whether it can be done to a separate DB.

MediaWiki does have a lot of features for working with MySQL at scale. However, not all of those features seem to be needed for the concrete reading list use case:

• Data set size and query volume / complexity are projected to be very modest (< ~10/s). Based on this, sharding and read scaling using slaves should not be necessary.
• All reads & writes are tied to a user account / session, which means that they will currently hit the primary DC only. Cross-DC replication consistency is not a concern for now.
• Logging slow queries at the app level is easy on any platform that supports some form of concurrent timers.

That said, I agree that it is very much worth considering those questions for larger use cases, and for future active-active operation. If we use MySQL more widely as a backing store for services, then we should ensure that common DB management functionality is consistently available to those services. This could either be a port of needed features to a library, or the use of an off-the-shelf SQL proxy along the lines of proxysql or haproxy managing sharding, slave lag, and logging concerns.

/cc @aaron @jcrespo

For a "new" service with modest number of writes and no current concerns for cross-dc consistency, we can implement clustering on MySQL- that is not currently possible on mediawiki due to its dependency on async replication and other legacy code limitations. That will provide both load balancing and HA with configurable consistency (sync or asnync reads).

I have setup already some test clusters within our infrastructure and many production ones in the past, and is probably the future for mediawiki masters HA (once mediawiki and legacy limitations are fixed)- so it will certainly be supported in the future. Load balancing can be provided by the connector or transparently by redundant proxies (HAProxy or ProxySQL).

The above is storage-wise. Architecture wise (and I have no power here), using storage as a shared mechanism seems like an anti-pattern to me and encapsulation violation. Shouldn't a piece of code be in between so only one codepath controls what is going on, exposing and api, and consumers use that api? Shouldn't probably the platform team weight in here, because even if this is for an extension, it is potentially a model shift?

@jcrespo: I think we are all on the same page that a single service should exclusively own and access its storage. The concrete question is whether creating a stand-alone service managing reading lists & storing its data in MySQL is feasible, and how the costs & benefits of doing so compare to an implementation as a MediaWiki extension.

That is the part that I would ask to the platform team 0:-)

Come to think of it, MySQL access in RESTBase will be necessary anyway for push notifications, right?

In T164805#3263061, @Tgr wrote:

Come to think of it, MySQL access in RESTBase will be necessary anyway for push notifications, right?

Not in RESTBase, but in the Push Notification service, which will be a Node.JS service. So yes, we will need a way to connect to MySQL from Node anyway.

@mobrovac @Tgr not to muddy the waters, but I have been doing some research in regards to the persistence for Push: T163116#3263679

MySQL may be the best fit here, but I am not sure yet that we should commit to the Push service also using MySQL until we do some more investigation.