Page MenuHomePhabricator

Librarize MediaWiki database layer and drop official support for non-MySQL engines
Closed, DeclinedPublic

Description

MediaWiki users overwhelmingly choose MySQL/MariaDB:
Pingback:

mysql:research@analytics-store.eqiad.wmnet [log]> select count(*) as num, event_database from MediaWikiPingback_15781718 group by event_database order by num desc;
| 8566 | mysql          |
|  265 | sqlite         |
|  149 | postgres       |
|    3 | mssql          |

WikiApiary: mysql: 21214, postgres: 283, sqlite: 144, oracle: 3, mssql: 1

PostgreSQL and SQLite usage is 1-2% and Oracle/MSSQL is zero within margin of error.

So there is little reason to spend much effort on supporting alternative backends. And de facto they are not supported. CI tests only run for MySQL, almost all developers only test on MySQL, and most DB-using extensions don't go out of their way to support others either. The policy says DB patches must support all five backends but that's fiction:

~/Wikimedia/grep/extensions$ find . -name '*.sql' | cut -d/ -f2 | uniq | wc -l
161
~/Wikimedia/grep/extensions$ for db in postgres sqlite oracle mssql; do echo -n "$db: "; find . -name '*.sql' | ack $db | cut -d/ -f2 | uniq | wc -l; done
postgres: 14
sqlite: 11
oracle: 4
mssql: 3
~/Wikimedia/grep/extensions$ find . -name '*.sql' | ack 'postgres|sqlite|oracle|mssql' | cut -d/ -f2 | uniq | wc -l
22

so about 10% of our extensions with SQL patches have non-MySQL patchfiles for at least some of their DB changes.


It will be better for everyone if we stop misleading users and clarify what database we support, while empowering more those who want to volunteer to support an alternative DB engine. That would mean:

  • update the policy and related pages
  • port non-MySQL Database subclasses into external libraries (RfC: Moving database abstractions out of MediaWiki core)
  • provide a way to test MediaWiki (and maybe extension) patches with alternative configuration, without forcing the RelEng team to maintain the necessary infrastructure. The ongoing work to make MediaWiki tests run on Travis CI (T75176) provides a possible path for that.

Event Timeline

Tgr updated the task description. (Show Details)

Someone suggested dropping mssql but it carnt happen as there are volunteers who have spent a ton of hours maintaining the support.

Volunteers have spent a lot of there time maintaining these dbs. We would have wasted there time if we remove the support.

Aklapper removed a subscriber: MediaWiki-libs-Rdbms.

This task says "drop official support". It does not say "drop support". (The reference might be to T113831.)

It says "drop official support" which would imply that dbs such as mssql or postgresql maintained by volunteers would be un trust worthy as users normally use official and supported dbs.

port non-MySQL Database subclasses into external libraries (RfC: Moving database abstractions out of MediaWiki core)

I would propose to go further, and move all of libs/rdbms/ into one or more external libraries, including the MySQL part as well. This way, components that are used by MediaWiki (or an extension) can be developed as separate projects that only require the database layer, without pulling in the rest of MediaWiki. For example, we would have liked to do this with the new Wikibase term store we’re building in Wikidata wb_terms Trailblazing, but as MediaWiki’s DBAL is not something you can composer require yet, we’re currently adding our code to Wikibase directly.

Tgr added a subscriber: aaron.

I would propose to go further, and move all of libs/rdbms/

I think that's the intention with everything under libs (as the README sort of states). @aaron is probably the best person to ask about the status of that process.

Yes, even Oracle can go under /libs once the foreign key mess is dealt with.

Things like BagOStuff would have to be done first, though nothing is really blocking doing those now.

Note that the need for this (but not for librarizing the RDBMS component in general) would probably be obviated by T191231: RFC: Abstract schemas and schema changes if that gets resourced.

The Rdbms code is well on its way to becoming a separate library. At this time there isn't yet a strong reason for us to publish it on Packagist and take on the additional release and update overhead between it and mediawiki-core however.

The conversation of dropping support for all non-MySQL backends I think is best held elsewhere, and certainly out of scope for this task and not limited to the Rdbms library itself. The Rdbms library will serve to support the needs of MediaWiki core.

Tgr added a subscriber: Ladsgroup.

Note that the need for this (but not for librarizing the RDBMS component in general) would probably be obviated by T191231: RFC: Abstract schemas and schema changes if that gets resourced.

Thanks a ton to @Ladsgroup for getting that done!