Hadoop requires a java connector for MariaDB, mariadb-connector-j.
We need to investigate how we want to fetch, install and configure the service, and develop a process to monitor for vulnerabilities and handle software updates.
http://downloads.mariadb.com/Connectors/java
https://github.com/mariadb-corporation/mariadb-connector-j
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| In Progress | greg | T377362 EPIC: Trino/MinIO/Hive-Standalone-Metaserver/Dagster/Metabase/Superset Implementation | |||
| Resolved | Jgreen | T377368 EPIC: Hive (hive-standalone-metastore) Implementation | |||
| Resolved | Jgreen | T378087 [mariadb-connector-j] Investigate packaging, install, security monitoring. |
I installed v3.4.1 by hand at fransc2001:/usr/lib/mariadb-connector-j to unblock Arnis on prototyping while I go back and deal with packaging/install.
After some discussion, we're thinking we'll make a deb package named i.e. hive-mariadb-driver-3-4-1 (with the versioned package name), which installs the three needed jar files to /usr/lib/hadoop/lib. The jar filenames will retain their version numbers per upstream.
This way we should be able to install multiple releases of the driver, so we can test and roll forward/back from hadoop config.
This is done. New package hive-mariadb-client is added to package_getter, and versions 3.4.1 and 3.5.0 are installed on fransc2001.
We ended up going with mysql-connector-j which is available from mysql's deb repository. This also takes care of the ongoing maintenance since we automatically check for repository changes daily on all attached repositories.