Maniphest T370710

Upgrade to Numpy ≥ 1.24 in Conda-Analytics
Open, MediumPublic
Actions

Assigned To

None

Authored By

	nshahquinn-wmf
	Jul 22 2024, 7:48 PM

Tags

Referenced Files

None

Subscribers

Description

Numpy 1.24 removes np.bool, which is accessed by Pyspark when collecting a boolean field to a Pandas dataframe.

This is fixed in Pyspark 3.4, according to this StackOverflow answer.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T370710 Upgrade to Numpy ≥ 1.24 in Conda-Analytics
		Open		None	T370712 Upgrade to Pyspark ≥ 3.4

Event Timeline

nshahquinn-wmf created this task.Jul 22 2024, 7:48 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 22 2024, 7:48 PM

nshahquinn-wmf mentioned this in T370712: Upgrade to Pyspark ≥ 3.4.Jul 22 2024, 7:54 PM

nshahquinn-wmf added a subtask: T370712: Upgrade to Pyspark ≥ 3.4.

nshahquinn-wmf mentioned this in T356230: Conda-Analytics packages incompatible with latest versions of Pandas and Numpy.Jul 22 2024, 7:57 PM

BTullis triaged this task as Medium priority.Aug 8 2024, 2:26 PM

BTullis moved this task from Incoming to Software Upgrades on the Data-Platform-SRE board.