Maniphest T316336

Upgrade DataHub to v0.8.43
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	BTullis
	Aug 26 2022, 10:04 AM

Tags

Referenced Files

None

Subscribers

• odimitrijevic

Description

We need to upgrade DataHub in order to benefit from a new feature in the Superset source driver.

Version 0.8.43 is the latest stable release, so we should aim to use this.

https://datahubproject.io/docs/releases/#v0-8-43

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Update the packaged environment for datahub clients	analytics/refinery	master	+3 -3
	Fix the datahub 0.8.43 deployment	operations/deployment-charts	master	+3 -1
	Add an empty directory for GMS plugin auth resources	analytics/datahub	wmf	+1 -0
	Upgrade datahub to version 0.8.43	operations/deployment-charts	master	+11 -11
	Build containers for version 0.8.43 of DataHub	analytics/datahub	wmf	+6 -4

Customize query in gerrit

Related Changes in GitLab:

	Title	Reference	Author	Source Branch	Dest Branch
	Bump the datahub components to version 0.8.43	repos/data-engineering/airflow-dags!129	btullis	bump_datahub_version	main

Customize query in GitLab

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T369756 [EPIC] Datahub Improvements
		Resolved		brouberol	T309622 Create Airflow Pipeline for Ingesting/Updating Superset data into DataHub
		Resolved		BTullis	T316336 Upgrade DataHub to v0.8.43

Event Timeline

BTullis created this task.Aug 26 2022, 10:04 AM

BTullis moved this task from Backlog to Data Catalog on the Data-Engineering-Planning board.

Change 826811 had a related patch set uploaded (by Btullis; author: Btullis):

[analytics/datahub@wmf] Build containers for version 0.8.43 of DataHub

https://gerrit.wikimedia.org/r/826811

gerritbot added a project: Patch-For-Review.Aug 26 2022, 10:19 AM

Change 826811 merged by jenkins-bot:

[analytics/datahub@wmf] Build containers for version 0.8.43 of DataHub

https://gerrit.wikimedia.org/r/826811

BTullis mentioned this in R3101:15d85d818460: Build containers for version 0.8.43 of DataHub.Aug 26 2022, 11:55 AM

Maintenance_bot removed a project: Patch-For-Review.Aug 26 2022, 12:30 PM

Change 826844 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] We wish to upgrade datahub to version 0.8.43

https://gerrit.wikimedia.org/r/826844

gerritbot added a project: Patch-For-Review.Aug 26 2022, 12:31 PM

Change 826844 merged by jenkins-bot:

[operations/deployment-charts@master] Upgrade datahub to version 0.8.43

https://gerrit.wikimedia.org/r/826844

Change 827987 had a related patch set uploaded (by Btullis; author: Btullis):

[analytics/refinery@master] Update the packaged environment for datahub clients

https://gerrit.wikimedia.org/r/827987

The rollout of the new version to staging failed with jetty reporting: ERROR: No such classes directory file:///etc/datahub/plugins/auth/resources

I will look to address this with another patch to the datahub/wmf branch.

Change 827999 had a related patch set uploaded (by Btullis; author: Btullis):

[analytics/datahub@wmf] Add an empty directory for GMS plugin auth resources

https://gerrit.wikimedia.org/r/827999

Change 828001 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] Fix the datahub 0.8.43 deployment

https://gerrit.wikimedia.org/r/828001

Change 827999 merged by jenkins-bot:

[analytics/datahub@wmf] Add an empty directory for GMS plugin auth resources

https://gerrit.wikimedia.org/r/827999

BTullis mentioned this in R3101:74e3f000b303: Add an empty directory for GMS plugin auth resources.Aug 30 2022, 1:42 PM

Change 828001 merged by jenkins-bot:

[operations/deployment-charts@master] Fix the datahub 0.8.43 deployment

https://gerrit.wikimedia.org/r/828001

I have now deployed version 0.8.43 of DataHub.
Now working on updatnig the client versions and the packaged environment for refinery to use.

Change 827987 merged by Joal:

[analytics/refinery@master] Update the packaged environment for datahub clients

https://gerrit.wikimedia.org/r/827987

BTullis mentioned this in R3101:2ea96f11bf64: Build containers for version 0.8.43 of DataHub.Oct 31 2022, 5:23 PM

BTullis mentioned this in R3101:bee1c69c48e3: Add an empty directory for GMS plugin auth resources.

This is now resolved. We have upgraded to version 0.9.0 in T321907: Upgrade Datahub to v0.9

• Stevemunene mentioned this in R3101:cd4a1d2330fa: Build containers for version 0.8.43 of DataHub.Mar 8 2023, 9:54 AM

• Stevemunene mentioned this in R3101:64aa1b8d43da: Add an empty directory for GMS plugin auth resources.

Maintenance_bot removed a project: Patch-For-Review.Mar 8 2023, 10:10 AM

• Stevemunene mentioned this in R3101:a1e09980006a: Add an empty directory for GMS plugin auth resources.Mar 8 2023, 11:27 AM

• Stevemunene mentioned this in R3101:65bd07f05c4f: Build containers for version 0.8.43 of DataHub.

BTullis mentioned this in R3101:c4b1928dc491: Build containers for version 0.8.43 of DataHub.Jul 5 2023, 11:14 AM

BTullis mentioned this in R3101:370b9e27b7ce: Add an empty directory for GMS plugin auth resources.