Refinery contains several Python modules that duplicate or complement functionality in Wmfdata-Python. For example, refinery.util.get_dbnames_from_mw_config and wmfdata.utils.get_dblist do the same thing, as do refinery.hive.query and wmfdata.hive.run. Obviously, it's preferable to avoid this code duplication.
Additionally some folks in Data Engineering have suggested that other Python functionality that exists only in Refinery should be moved into Wmfdata-Python: T293700#7438485, T292479#7482513.
Important points to consider:
- Should there be a general consolidation or just a partial one?
- Would adding modules aimed at data engineers to Wmfdata-Python make it harder for analysts to learn to use the package?
- How would updates be distributed? Both Refinery and Wmfdata-Python have different homegrown update methods.
- How would ownership of a combined package be handled?