Page MenuHomePhabricator

Review error in table visualization with Superset 0.36.0
Closed, DuplicatePublic

Description

While testing the upgrade to Superset 0.36.0, I noticed that http://localhost:9080/superset/dashboard/4/ shows:

"None of [Index(['project', 'count'], dtype='object')] are in the [columns]"

in the table visualization chart, as opposed to No data (Superset 0.35.2). project is listed in Group By and count in Metrics, both correctly set in the related datasource.

The stacktrace is:

Apr  4 08:42:09 an-tool1005 superset[16630]: "None of [Index(['project', 'count'], dtype='object')] are in the [columns]"
Apr  4 08:42:09 an-tool1005 superset[16630]: Traceback (most recent call last):
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/views/base.py", line 124, in wraps
Apr  4 08:42:09 an-tool1005 superset[16630]:     return f(self, *args, **kwargs)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/utils/decorators.py", line 70, in wrapper
Apr  4 08:42:09 an-tool1005 superset[16630]:     return f(*args, **kwargs)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/views/core.py", line 717, in explore_json
Apr  4 08:42:09 an-tool1005 superset[16630]:     viz_obj, csv=csv, query=query, results=results, samples=samples
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/views/core.py", line 639, in generate_json
Apr  4 08:42:09 an-tool1005 superset[16630]:     payload = viz_obj.get_payload()
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/viz.py", line 406, in get_payload
Apr  4 08:42:09 an-tool1005 superset[16630]:     payload["data"] = self.get_data(df)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/viz.py", line 628, in get_data
Apr  4 08:42:09 an-tool1005 superset[16630]:     df[non_percent_metric_columns],
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/frame.py", line 3001, in __getitem__
Apr  4 08:42:09 an-tool1005 superset[16630]:     indexer = self.loc._convert_to_indexer(key, axis=1, raise_missing=True)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/indexing.py", line 1285, in _convert_to_indexer
Apr  4 08:42:09 an-tool1005 superset[16630]:     return self._get_listlike_indexer(obj, axis, **kwargs)[1]
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/indexing.py", line 1092, in _get_listlike_indexer
Apr  4 08:42:09 an-tool1005 superset[16630]:     keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/indexing.py", line 1177, in _validate_read_indexer
Apr  4 08:42:09 an-tool1005 superset[16630]:     key=key, axis=self.obj._get_axis_name(axis)
Apr  4 08:42:09 an-tool1005 superset[16630]: KeyError: "None of [Index(['project', 'count'], dtype='object')] are in the [columns]"
Apr  4 08:42:09 an-tool1005 superset[16630]: ERROR:superset.views.base:"None of [Index(['project', 'count'], dtype='object')] are in the [columns]"
Apr  4 08:42:09 an-tool1005 superset[16630]: Traceback (most recent call last):
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/views/base.py", line 124, in wraps
Apr  4 08:42:09 an-tool1005 superset[16630]:     return f(self, *args, **kwargs)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/utils/decorators.py", line 70, in wrapper
Apr  4 08:42:09 an-tool1005 superset[16630]:     return f(*args, **kwargs)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/views/core.py", line 717, in explore_json
Apr  4 08:42:09 an-tool1005 superset[16630]:     viz_obj, csv=csv, query=query, results=results, samples=samples
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/views/core.py", line 639, in generate_json
Apr  4 08:42:09 an-tool1005 superset[16630]:     payload = viz_obj.get_payload()
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/viz.py", line 406, in get_payload
Apr  4 08:42:09 an-tool1005 superset[16630]:     payload["data"] = self.get_data(df)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/superset/viz.py", line 628, in get_data
Apr  4 08:42:09 an-tool1005 superset[16630]:     df[non_percent_metric_columns],
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/frame.py", line 3001, in __getitem__
Apr  4 08:42:09 an-tool1005 superset[16630]:     indexer = self.loc._convert_to_indexer(key, axis=1, raise_missing=True)
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/indexing.py", line 1285, in _convert_to_indexer
Apr  4 08:42:09 an-tool1005 superset[16630]:     return self._get_listlike_indexer(obj, axis, **kwargs)[1]
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/indexing.py", line 1092, in _get_listlike_indexer
Apr  4 08:42:09 an-tool1005 superset[16630]:     keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
Apr  4 08:42:09 an-tool1005 superset[16630]:   File "/srv/deployment/analytics/superset/venv/lib/python3.7/site-packages/pandas/core/indexing.py", line 1177, in _validate_read_indexer
Apr  4 08:42:09 an-tool1005 superset[16630]:     key=key, axis=self.obj._get_axis_name(axis)
Apr  4 08:42:09 an-tool1005 superset[16630]: KeyError: "None of [Index(['project', 'count'], dtype='object')] are in the [columns]"

This might be due to a change in pandas (https://github.com/apache/incubator-superset/pull/8985) but it needs to be investigated before reporting to upstream, it might be a misconfig of the chart.

Related Objects

Event Timeline

Forgot to mention that the above is a Druid datasource. After some tests the error seems appearing only when no data is returned for a query, I was able to successfully visualize some results. So the regression seems a minor one, only in the message displayed when no data is returned.

In the 0.36.0 changelog I see https://github.com/apache/incubator-superset/pull/9006/commits that could be related, it would fit with what I am seeing.