| • Tbayer | |
| May 29 2018, 9:35 AM |
| F18666680: Turnilo data cubes 2018-06-02 marked.png | |
| Jun 2 2018, 10:15 AM |
Selecting the pageviews-daily dataset from https://turnilo.wikimedia.org results in the following error message:
Query error
could not resolve $view_count
(Besides View Count, there is also a second measure available named Count, but that's likely the wrong one when one is interested in pageview numbers.)
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| turnilo: disable instrospection autofill-all for pageviews-* | operations/puppet | production | +2 -2 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Ottomata | T194427 Deploy Turnilo (possible pivot replacement) | |||
| Resolved | elukey | T195819 Pageviews-daily broken after move from Pivot to Turnilo |
Thanks a lot for the notification, I think that this is due to the reimage of druid1002 that happened this morning. pageviews-daily is not replicated, so the segments cache is present on one Druid historical only, that I am almost sure was on druid1002. The segments are stored in HDFS, but needs to be reloaded and this is happening as we speak:
https://grafana.wikimedia.org/dashboard/db/druid?refresh=1m&panelId=50&fullscreen&orgId=1
pageviews-hourly is for example replicated two times, and it is available via Turnilo (same thing as webrequest).
The follow up for this task is surely figure out what datasets needs to be replicated at least two times to ease the maintenance and/or a random failure of one Druid node.
This is clearly not right, segments have been loaded and nothing changed. Moreover, from the title (that I didn't pay attention to before) says that it was used to work with pivot but not turnilo, so definitely something that has been happening for a while (not started today).
I'm actually not 100% certain when I last saw it working (I do recall checking some pageview date in recent days since the switchover, but probably only used the hourly version).
Thanks for continuing to look into it!
Change 435997 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] turnilo: disable instrospection autofill-all for pageviews-*
Change 435997 merged by Elukey:
[operations/puppet@production] turnilo: disable instrospection autofill-all for pageviews-*
Sorry, but the pageviews-daily dataset as it was linked in the task description is still broken: While the quoted error message is gone, it still only offers the "Count" measure (which is rather meaningless and likely to mislead users into believing we get only between 3 and 4 million pageviews per day overall) and not the "View Count" measure that we need.
It seems that the separately listed pageviews_daily cube (underscore instead of dash) is the right one where one can access "View Count" (I have been using it recently, and perhaps also @JKatzWMF when he made the comment above).
In the light of this, it may be best to simply delete pageviews-daily from the list:
Thanks a lot for the ping, we indeed have the plan to delete the old dataset but we have not announced the new naming convention for the Druid's datasources yet (all underscores basically). We'd have done it on Monday sending an email to the analytics@ mailing list, apologies for the confusion.