Page MenuHomePhabricator

Upgrade to Superset 0.35
Closed, ResolvedPublic8 Estimated Story Points

Description

A new minor version is about to be released!

Related Objects

Event Timeline

fdans moved this task from Incoming to Operational Excellence on the Analytics board.

https://pypi.org/project/apache-superset/ is the new name of the Pypi package, that still shows 0.34.1. It would be really great to start using the pypi package rather than our own repository.. I can see in the superset-dev mailing list that somebody already asked when it will be released, so I'd wait a bit more before testing.

Version 0.35.0 was not released due to some bugs, 0.35.1 is still under voting in the dev mailing list. It seems that they should be able to release on pypi next week.

Version 0.35.1 was released on pypi, I tested it with https://gerrit.wikimedia.org/r/#/c/analytics/superset/deploy/+/552238/ on an-tool1005

Good things:

  • building from pypi is incredibly quick and easy, compared to build from our branch

Bad things:

Worth noticing:

I changed a bit our build script: https://gerrit.wikimedia.org/r/#/c/analytics/superset/deploy/+/552238/7/build_wheels.sh
The reason is that with pip 0.19 it was introduced the manylinux_2010 platform tag, that should replace (eventually) manylinux1. The idea is to have platform tags for C-based wheels that need to indicate what glib version they support. On Debian Buster pip is 0.18, and it doesn't recognize the new platform tag, ending up in the following error (pyarrow is a new dep) when executing create_virtualenv.sh:

pyarrow-0.14.1-cp37-cp37m-manylinux2010_x86_64.whl is not a supported wheel on this platform.

For some reason (that might be avoided but didn't find a way) pip tries to upgrade itself to the latest version when creating a venv, so the error came because the current build_script.sh creates a new venv (hence using pip 0.19) meanwhile the create_virtualenv.sh uses the system one (0.18).

The fix for https://github.com/apache/incubator-superset/issues/8676 has been merged to upstream, so it will likely be included in the next release. I would skip 0.35.1 if nobody opposes to avoid building again from our own branch etc.. Otherwise we can work on it :)

Change 554541 had a related patch set uploaded (by Elukey; owner: Elukey):
[analytics/superset/deploy@master] build_wheels.sh: use the system pip instead of the virtualenv's one

https://gerrit.wikimedia.org/r/554541

Change 554541 merged by Elukey:
[analytics/superset/deploy@master] build_wheels.sh: use the system pip instead of the virtualenv's one

https://gerrit.wikimedia.org/r/554541

elukey set the point value for this task to 8.Dec 4 2019, 4:08 PM