Superset 1.4.2 has been released: https://github.com/apache/superset/releases/tag/1.4.2
In particular this closed upstream bug was reproduced by @nshahquinn-wmf : https://github.com/apache/superset/pull/16945
Let's deploy this to the staging instance and test it out before rolling it out to production.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Upgrade superset to 1.4.2 | analytics/superset/deploy | master | +6 -1 | |
| Upgrade superset to 1.5.0 | analytics/superset/deploy | master | +11 -1 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | BTullis | T304972 Upgrade Superset to 1.4.2 | |||
| Resolved | BUG REPORT | JArguello-WMF | T308441 Error when updating dashboard |
Change 774924 had a related patch set uploaded (by Razzi; author: Razzi):
[analytics/superset/deploy@master] Upgrade superset to 1.4.2
Superset 1.4.2 is running on superset-staging: an-tool1005. When making the change, I had to pin markupsafe to 2.0.1 since the default markupsafe it downloaded is not compatible with the version of flask that superset is using.
The upgrade itself was uneventful; migrations ran fine and a first glance at the UI looks good.
I thought I'd update the staging database to be the same as production before sharing superset staging widely, and I'm glad I did, because it looks like there's some sort of database issue with the update.
I dumped the superset production database and restored it to superset staging as per https://wikitech.wikimedia.org/wiki/Analytics/Systems/Superset#Administration, but when I ran superset init, I got this error:
$ superset init ... 2022-04-05 21:57:14,046:INFO:superset.security.manager:Creating missing datasource permissions. Traceback (most recent call last): File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt decrypted = decrypted.decode('utf-8') UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa6 in position 1: invalid start byte During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/deployment/analytics/superset/venv/bin/superset", line 8, in <module> sys.exit(superset()) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask/cli.py", line 586, in main return super(FlaskGroup, self).main(*args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask/cli.py", line 426, in decorator return __ctx.invoke(f, *args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask/cli.py", line 426, in decorator return __ctx.invoke(f, *args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/cli.py", line 95, in init security_manager.sync_role_definitions() File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/security/manager.py", line 704, in sync_role_definitions self.create_missing_perms() File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/security/manager.py", line 646, in create_missing_perms merge_pv("datasource_access", datasource.get_perm()) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/connectors/sqla/models.py", line 619, in get_perm return f"[{self.database}].[{self.table_name}](id:{self.id})" File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 294, in __get__ return self.impl.get(instance_state(instance), dict_) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 730, in get value = self.callable_(state, passive) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state session, state, primary_key_identity, passive File "<string>", line 1, in <lambda> File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 850, in _emit_lazyload session.query(self.mapper), primary_key_identity File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 615, in _load_on_pk_identity result = list(bq.for_session(self.session).params(**params)) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances cursor.close() File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ with_traceback=exc_tb, File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ raise exception File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances rows = [proc(row) for row in fetch] File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp> rows = [proc(row) for row in fetch] File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 588, in _instance populators, File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full dict_[key] = getter(row) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py", line 1278, in process return process_value(impl_processor(value), dialect) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 477, in process_result_value value = super().process_result_value(value=value, dialect=dialect) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 422, in process_result_value decrypted_value = self.engine.decrypt(value) File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 130, in decrypt raise ValueError('Invalid decryption key') ValueError: Invalid decryption key
It looks like other people have run into this in https://github.com/apache/superset/issues/8538. It looks like this will be resolvable by removing the database configuration secrets and recreating it. I'll try this on staging now.
Ok, looks like the following will resolve it:
razzi@an-coord1001:~$ sudo mysql superset_staging MariaDB [superset_staging]> update dbs set encrypted_extra = null;
Then go to production (superset.wikimedia.org), go to the database config, click "edit" on presto and copy the advanced > secure extra json value. For staging change an-tool1010 to an-tool1005 before saving.
Then superset init runs fine.
Here's the edited config for staging; no secrets in it so fine to post here.
{"connect_args":
{"KerberosConfigPath":"/etc/krb5.conf",
"KerberosKeytabPath":"/etc/security/keytabs/superset/superset.keytab",
"KerberosPrincipal":"superset/an-tool1005.eqiad.wmnet@WIKIMEDIA",
"KerberosRemoteServiceName":"presto",
"requests_kwargs":
{"verify":"/etc/ssl/certs/Puppet_Internal_CA.pem"}}}Hi Product Analytics, superset 1.4.2 is ready to be tested on staging. Once we confirm there are no showstopping bugs we'll release it to production.
Access it using an ssh tunnel as follows:
ssh -NL 8080:an-tool1005.eqiad.wmnet:80 an-tool1005.eqiad.wmnet
then go to http://localhost:8080 and you'll be prompted for your wikitech username and password.
Comment here if you find any new issues, or create subtasks of this task.
A few of us (@mpopov, @MNeisler , @Iflorez , @cchen , @Mayakp.wiki, and I) will each test a chart or dashboard in staging this week and follow up if we encounter any issues.
We're currently blocked on this by not being able to SSH to an-tool1005 and are being asked for a password.
Change 789683 had a related patch set uploaded (by Razzi; author: Razzi):
[analytics/superset/deploy@master] Upgrade superset to 1.5.0
I tried out Superset 1.5 briefly but found it requires python 3.8, and an-tool1005 is currently running python 3.7. The error:
ImportError: cannot import name 'TypedDict' from 'typing' (/usr/lib/python3.7/typing.py)
So we'll have to upgrade the host to a later Debian version or install a new Python before upgrading past 1.4.
Change 789683 abandoned by Razzi:
[analytics/superset/deploy@master] Upgrade superset to 1.5.0
Reason:
This patch is fine, but we'd need to upgrade to python 3.8 first, so closing for now.
Change 774924 merged by Razzi:
[analytics/superset/deploy@master] Upgrade superset to 1.4.2