I tried to deploy superset to the staging box, but it failed with

aiohttp-3.7.3-cp37-cp37m-manylinux2014_x86_64.whl is not a supported wheel on this platform.

Furthermore the rollback failed with

Rollback all deployed groups? [Y/n]: y
17:38:42
== DEFAULT ==
:* an-tool1005.eqiad.wmnet
17:40:45 ['/usr/bin/scap', 'deploy-local', '-v', '--repo', 'analytics/superset/deploy', '--force', '-g', 'default', 'rollback', '--refresh-config'] on an-tool1005.eqiad.wmnet returned [70]: Rolling back from revision c7147389288b54b0a25c3b3e9d64b9ebed32e638 to 828ef03a8b0bc1b8a59c63f50b4c7ff131183bbb
Removing old revision /srv/deployment/analytics/superset/deploy-cache/revs/5d4f5ac8530e10c468ab9156f52daf6b96ed29ee
Restarting service 'superset'
Port 9080 not up. Waiting 3.00s
...
Port 9080 not up. Waiting 3.00s
Unhandled error:
deploy-local failed: <OSError> {}

analytics/superset/deploy: rollback stage(s): 100% (ok: 0; fail: 1; left: 0)
17:40:45 1 targets had deploy errors
17:40:45 1 targets failed

So staging superset is currently unavailable.

Ok, the problem was that I had upgraded the pip version in the docker container when building the wheels, which made the wheels incompatible with the staging server.

I was able to keep the old pip version and build all packages by using apt to install cargo, which was needed by the python cryptography package.

Now I'm running into an error when following the instructions to upgrade superset (https://wikitech.wikimedia.org/wiki/Analytics/Systems/Superset#Upgrading):

(venv) superset@an-tool1005:/home/razzi$ superset db upgrade
Loaded your LOCAL configuration at [/etc/superset/superset_config.py]
logging was configured successfully
...
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1709, 'Index column size too large. The maximum column size is 767 bytes.')
[SQL: CREATE INDEX ix_row_level_security_filters_filter_type ON row_level_security_filters (filter_type)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)

I tried creating the mysql database with

create database superset_staging DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci

rather than the documented

create database superset_staging DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

but got the same error. I'm not a mysql expert, @elukey @Ottomata any ideas?

Not sure what the 'right' thing to do is, but a quick search for that error brought me to https://stackoverflow.com/questions/30761867/mysql-error-the-maximum-column-size-is-767-byteswith some suggestions. Been a while since I did MySQL stuff too, do we use innodb file format barracuda?

I have something in my notes: https://wikitech.wikimedia.org/wiki/User:Elukey/Analytics/Superset#Upgrade_DB

We already have innodb_file_format=Barracuda in the config, can you check if doing ALTER TABLE row_level_security_filters ROW_FORMAT=DYNAMIC works? (Assuming that row_level_security_filters is a table of course)

MariaDB [(none)]> show global variables like 'innodb_file_format';
+--------------------+-----------+
| Variable_name      | Value     |
+--------------------+-----------+
| innodb_file_format | Barracuda |
+--------------------+-----------+
1 row in set (0.00 sec)

@razzi remember also to drop/re-create the staging database as we did the last time so we'll have a more up to date version to do testing of dashboards etc..

ALTER TABLE row_level_security_filters ROW_FORMAT=DYNAMIC; fixed it, thanks! Here's the full procedure so the order is clear:

On an-coord1001:

$ sudo mysql
> drop database superset_staging;
> create database superset_staging DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
> exit
$ sudo sh -c 'mysqldump superset_production > superset_production_$(date +%s).sql'
$ sudo mysql superset_staging < superset_production_1613751453.sql
$ sudo mysql superset_staging
> ALTER TABLE row_level_security_filters ROW_FORMAT=DYNAMIC;

On an-tool1005:

razzi@an-tool1005:~$ sudo su superset
superset@an-tool1005:/home/razzi$ source /srv/deployment/analytics/superset/venv/bin/activate
(venv) superset@an-tool1005:/home/razzi$ export PYTHONPATH=/etc/superset
(venv) superset@an-tool1005:/home/razzi$ superset db upgrade
(venv) superset@an-tool1005:/home/razzi$ superset init

Found a client-side error: when creating a new chart from pageviews_hourly, when attempting to add a metric in the chart creator, the frontend app crashes.

I'm guessing this has to do with the frontend needing to be recompiled. There's a step in the deploy frozen-requirements-custom-build.txt which seems to relate to this:

# Inside docker:
# cd /superset_upstream/superset-frontend/
# npm ci && npm run build

I'll see if I can run that in docker and if it produces artifacts that fix the error.

Hm, I see now that the built javascript files are contained within the superset 1.0.1 wheel, so they should be in sync.

Tested with another data source and another browser (originally used Firefox, Safari also shows error) and each setup repro'd the error.

@razzi great finding, if you don't manage to solve the problem I suggest to check https://github.com/apache/superset/issues to see if anything is outstanding for Superset 1.1 and if not, to open an issue :)

I installed superset from source to run the frontend in development mode, and reported the error I found upstream: https://github.com/apache/superset/issues/13396