Work out why mobile web dashboard entries abruptly terminate
Closed, ResolvedPublic

Description

The Search dashboards list mobile web data that for some reason doesn't go further than early May. Why? Is the schema still working? If so, why is nothing showing up?

Ironholds updated the task description. (Show Details)
Ironholds raised the priority of this task from to Needs Triage.
Ironholds claimed this task.
Ironholds added a project: Discovery.
Ironholds added a subscriber: Ironholds.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 23 2015, 5:43 AM
Ironholds moved this task from Needs triage to Analysis on the Discovery board.May 23 2015, 5:43 AM

AFAIK the schema was generating extreme load and they had no easy and convenient way to sample it to an appropriate rate, so they simply turned it off. @JKatzWMF or @phuedx can confirm this.

Well, the last timestamp in the schema is 20150527135303 so either the patch didn't take or something else is going on.

HAH. Resolved.

The answer is the data /is/ coming through but the weird spike at the beginning is distorting things. Will throw in a manky patch to fix.

Ironholds set Security to None.

Patched. We might want to use an even narrower time window, though.

@Ironholds: If data's coming through, then the patch didn't take. Which wikis are you seeing events from?

Also, the Web team committed to turning the logging back on at a sane rate this sprint. The work is tracked in T99788.

"yes":

mysql:research@analytics-store.eqiad.wmnet [log]> SELECT DISTINCT(wiki) FROM MobileWebSearch_12054448 WHERE LEFT(timestamp,8) >= '20150525';
+--------------+

wiki

+--------------+

dawiki
dewiki
enwiki
enwiktionary
eswiki
frwiki
hiwiki
itwiki
jawiki
kowiki
plwiki
plwiktionary
ptwiki
ruwiki
zhwiki

+--------------+
15 rows in set (18.78 sec)

Did you stop logging or just stop issuing new tokens? On turning the logging back on: is there also a commitment to have a shared understanding between teams about what we're logging and how, report bugs up to search so we can take note of them, and maintain the whole shebang? These schemas are becoming pretty important at our end and I'm trying to work out if we need to be taking them over or not.

Wat!? Well, I removed the line that registers the handler that logs the events. So…

On turning the logging back on: is there also a commitment to have a shared understanding between teams about what we're logging and how, report bugs up to search so we can take note of them, and maintain the whole shebang? These schemas are becoming pretty important at our end and I'm trying to work out if we need to be taking them over or not.

Yes but I figure it'll work both ways. Any bugs we find that affect logging, for example, I'll make sure to add you to and to let you know if and how MobileWebSearch event logging was affected. I'd expect the same from y'all.

Deskana closed this task as Resolved.Jun 3 2015, 11:21 AM