The new version is out for testing!
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | odimitrijevic | T211706 Superset Updates | |||
| Resolved | elukey | T249495 Upgrade to Superset 0.36.0 |
Event Timeline
Comment Actions
Change 586328 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] superset: handle upgrade to 0.36.0
Comment Actions
Opened https://github.com/apache/incubator-superset/issues/9468 to upstream to fix a bug found in 0.36.0rc3
Comment Actions
Change 586328 merged by Elukey:
[operations/puppet@production] superset: handle upgrade to 0.36.0
Comment Actions
Change 591440 had a related patch set uploaded (by Elukey; owner: Elukey):
[analytics/superset/deploy@master] Release upstream version 0.36.0
Comment Actions
Differences that I noticed when checking the staging Superset:
- http://localhost:9080/superset/dashboard/externalsearch/ contains a lot of `Since and Until time bounds should be specified when using the Time Shift feature.`, but I think it is due to the fact that the time range is not specified and something might have changed in Superset 0.36.0 about it. Setting a time range does solve the problem.
- http://localhost:9080/superset/dashboard/16 doesn't display a lot, again I think it is due to some new defaults added to time ranges. If I adjust the time range (for example, granularity month and last 7 months) it works.
- http://localhost:9080/superset/dashboard/72/ seems to show errors like "None of [Index(['country_code'], dtype='object')] are in the [columns]", meanwhile the correspondent prod dashboard shows "No data". I think this is a bug like https://github.com/apache/incubator-superset/issues/9468, due to the fact that we are not using SQLAlchemy-based druid datasources.
Comment Actions
Created https://github.com/apache/incubator-superset/pull/9671 to upstream, it should fix some weird errors returned. Will wait until upstream reviews/merges before applying it to our Superset version.
Comment Actions
The above patch was reverted for a little bug, and https://github.com/apache/incubator-superset/pull/9789 was sent and merged afterwards.
I have rebuilt superset 0.36 to include the last two patched filed to upstream (and only in master, not released with 0.36), currently under testing in staging.
Filed also https://github.com/apache/incubator-superset/issues/9855 this morning, it seems a little visualization issue only happening with Druid datasources (and not with Druid Tables, namely the ones defined using sqlalchemy). I think it is a minor bug that shouldn't prevent us deploying 0.36.
Comment Actions
Change 597991 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] superset: upgrade gunicorn app name for all envs
Comment Actions
Change 597991 merged by Elukey:
[operations/puppet@production] superset: upgrade gunicorn app name for all envs
Comment Actions
Change 591440 merged by Elukey:
[analytics/superset/deploy@master] Release upstream version 0.36.0
Comment Actions
For some strange reason, when I attempted the upgrade I have got the following error during the db upgrade step (not got when doing the same in staging):
Loaded your LOCAL configuration at [/etc/superset/superset_config.py]
logging was configured successfully
INFO:superset.utils.logging_configurator:logging was configured successfully
/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask_caching/__init__.py:189: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
"Flask-Caching: CACHE_TYPE is set to null, "
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade cca2f5d568c8 -> c2acd2cf3df2, alter type of dbs encrypted_extra
Traceback (most recent call last):
File "bin/superset", line 21, in <module>
superset()
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask/cli.py", line 426, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask_migrate/cli.py", line 134, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask_migrate/__init__.py", line 96, in wrapped
f(*args, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/flask_migrate/__init__.py", line 271, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
script.run_env()
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/util/compat.py", line 173, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/migrations/env.py", line 115, in <module>
run_migrations_online()
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/migrations/env.py", line 107, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/runtime/migration.py", line 518, in run_migrations
step.migration_fn(**kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/migrations/versions/c2acd2cf3df2_alter_type_of_dbs_encrypted_extra.py", line 53, in upgrade
existing_nullable=True,
File "/usr/lib/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/operations/base.py", line 325, in batch_alter_table
impl.flush()
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/operations/batch.py", line 79, in flush
fn(*arg, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/ddl/mysql.py", line 98, in alter_column
else existing_comment,
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/ddl/impl.py", line 134, in _exec
return conn.execute(construct, *multiparams, **params)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1020, in execute
return meth(self, multiparams, params)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1075, in _execute_ddl
else None,
File "<string>", line 1, in <lambda>
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 476, in compile
return self._compiler(dialect, bind=bind, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 29, in _compiler
return dialect.ddl_compiler(dialect, self, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 319, in __init__
self.string = self.process(self.statement, **compile_kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 350, in process
return obj._compiler_dispatch(self, **kwargs)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/ext/compiler.py", line 441, in <lambda>
lambda *arg, **kw: existing(*arg, **kw),
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/ext/compiler.py", line 486, in __call__
return fn(element, compiler, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/ddl/mysql.py", line 367, in _mysql_modify_column
comment=element.comment,
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/alembic/ddl/mysql.py", line 400, in _mysql_colspec
compiler.dialect.type_compiler.process(type_),
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 400, in process
return type_._compiler_dispatch(self, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 95, in _compiler_dispatch
return meth(self, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 3440, in visit_type_decorator
return self.process(type_.type_engine(self.dialect), **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 400, in process
return type_._compiler_dispatch(self, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 95, in _compiler_dispatch
return meth(self, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 3418, in visit_string
return self.visit_VARCHAR(type_, **kw)
File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 2061, in visit_VARCHAR
"VARCHAR requires a length on dialect %s" % self.dialect.name
sqlalchemy.exc.CompileError: VARCHAR requires a length on dialect mysqlI found https://github.com/apache/incubator-superset/issues/9878, that seems to point out to something in sqlalchemy, but it is an old bug and doesn't match our config. Will try to repro in staging, very weird.
Comment Actions
The #9878 is not an old bug right? The version of mysql used by the fellow reporting is, but this is an issue with alchemy running the migration for some tables:
INFO [alembic.runtime.migration] Running upgrade cca2f5d568c8 -> c2acd2cf3df2, alter type of dbs encrypted_extra
We can possibly modify the migration file?
Comment Actions
I think this is the migration bit that is throwing an error cause the text filed might require a specific length that is not specified.
try:
- Postgres migration batch_op.alter_column( "encrypted_extra", existing_type=sa.Text(), type_=EncryptedType(sa.Text()), postgresql_using="encrypted_extra::bytea", existing_nullable=True, )
Comment Actions
@Nuria we don't have postgres, the problem comes from the except part of the migration script that fails IIUC from the stacktrace. I had already tried to add length to various type fields but didn't work :(
Upstream answered https://github.com/apache/incubator-superset/pull/8493#issuecomment-632738195 and the solution seems to work, will re-build again pinning sqlalchemy-utils to that version (note: we have only one minor version more than what fixes the problem, sigh..)
Comment Actions
Ok I think I know what went wrong, and... drum roll... it is of course my fault :D
What I did to test this version has been:
- pick 0.36 and attempt the db migration
- repackage two times 0.36 with custom patches (pull requests to fix visualization issues), but every time avoiding to drop the staging db, re-create it, upgrade the db etc.. The assumption was that nothing should have changed in deps to cause troubles.
Enter https://github.com/kvesteri/sqlalchemy-utils/pull/426, in which sqlalchemy-utils introduced a change causing problems, releasing it with version 0.36.5. In 1) I used 0.36.4, that did not carry the bug, and the database migration that I tested of course succeeded.
During 2), the last version of sqlalchemy-utils was pulled in, and it landed in production breaking the db upgrade step.
Lesson learned: always drop the database, re-create it as exact copy of the production one and test the db upgrade step.
Really sorry, will create another build on Monday for staging.
The "fix" is to pin sqlalchemy-utils==0.36.4 in the requirements (just tested it manually via pip in staging, it works).
Comment Actions
Change 598423 had a related patch set uploaded (by Elukey; owner: Elukey):
[analytics/superset/deploy@master] Release upstream version 0.36.0
Comment Actions
New version deployed in staging, waiting for another round of tests before releasing to production.
Comment Actions
Change 598423 merged by Elukey:
[analytics/superset/deploy@master] Release upstream version 0.36.0
Comment Actions
Change 598914 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] superset: set default gunicorn app name
Comment Actions
Change 598914 merged by Elukey:
[operations/puppet@production] superset: set default gunicorn app name