Large number of popup events not validating
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Nuria
	Mar 2 2015, 5:57 PM

Project Tags

Referenced Files

None

Subscribers

Aklapper

• Jaredzimmerman-WMF

• Nuria

Description

Large number of popup events not validating due to duration being null:

2015-03-01 06:42:21,795 (MainThread) Unable to decode: ?%7B%22event%22%3A%7B%22duration%22%3Anull%7D%2C%22revision%22%3A7536956%2C%22schema%22%3A%22Popups%22%2C%22webHost
%22%3A%22zh.wikipedia.org%22%2C%22wiki%22%3A%22zhwiki%22%7D; cp1069.eqiad.wmnet 41274030 2015-03-01T06:42:21 10.64.0.106 Mozilla/5.0 (Macintosh; In
tel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
(None is not of type u'integer'

Failed validating u'type' in schema[u'properties'][u'event'][u'properties'][u'duration']:

{u'description': u'Duration in milliseconds for how long the popup was displayed.',
 u'required': True,
 u'type': u'integer'}

Related Objects

Mentioned Here: P438 popups event logs
T86378: Popups EventLogging on TOC click fails validation

Event Timeline

• Nuria created this task.Mar 2 2015, 5:57 PM

• Nuria assigned this task to • Prtksxna.

• Nuria raised the priority of this task from to Needs Triage.

• Nuria updated the task description. (Show Details)

• Nuria added a project: MediaWiki-extensions-EventLogging.

• Nuria added subscribers: • Nuria, • Jaredzimmerman-WMF.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 2 2015, 5:57 PM

• Prtksxna added a project: Page-Previews.Mar 3 2015, 7:47 AM

• Prtksxna set Security to None.

Also, wiki is coming as null in many wikis, which again produces ahuge number of events that do not validate:

2015-03-02 06:44:01,254 (MainThread) Unable to decode: ?%7B%22event%22%3A%7B%22duration%22%3A1740%2C%22action%22%3A%22dismissed%22%7D%2C%22revision%22%3A7536956%2C%22sche
ma%22%3A%22Popups%22%2C%22webHost%22%3A%22ur.wikipedia.org%22%2C%22wiki%22%3Anull%7D; cp1069.eqiad.wmnet 42359025 2015-03-02T06:44:01 10.64.0.106 Mozi
lla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
(None is not of type u'string'

Failed validating u'type' in schema[u'properties'][u'wiki']:

{u'description': u"$wgDBName (for example: 'enwiki')",
 u'required': True,
 u'type': u'string'}

• Nuria triaged this task as High priority.Mar 3 2015, 4:46 PM

Any updates or ETAs on this issue?

I have Ori's script in my common.js but I can't seem to reproduce these errors. T86378: Popups EventLogging on TOC click fails validation tracked validation errors in popups but it got closed in late January, which means the fix would have been on the Wikipedias by mid-February.

Are these validation errors from after that? Should I be using some tool other than Ori's script to find these issues? Right now I have no way of replicating, and thus debugging this issue.

In T91272#1082224, @Nuria wrote:

Also, wiki is coming as null in many wikis, which again produces a huge number of events that do not validate
Failed validating u'type' in schema[u'properties'][u'wiki']

This isn't something happening because of Popup's code, right? wiki isn't directly part of the Popup's schema.

@Prtksxna
You can test event validation in vagrant with the test server: http://www.mediawiki.org/wiki/Extension:EventLogging/Guide#Installing_the_EventLogging_devserver

Are these validation errors from after that?

Yes, they are happening as of today, duration is not being sent

For example:

2015-03-05 15:35:23,026 (MainThread) Unable to decode: ?%7B%22event%22%3A%7B%22duration%22%3Anull%2C%22sessionId%22%3A%22pxOqEdQMmDqNcurogYJvIkEWTSsWBefn%22%7D%2C%22revision%22%3A7536956%2C%22schema%22%3A%22Popups%22%2C%22webHost%22%3A%22en.wikipedia.org%22%2C%22wiki%22%3A%22enwiki%22%7D; cp1057.eqiad.wmnet 47360563 2015-03-05T15:35:22 10.64.32.134 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36

{u'description': u'Duration in milliseconds for how long the popup was displayed.'

And ... I do not see the wiki error anymore, so if we could fix the duration it will be great. In case it helps, failures and user agents for about a day:

35 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 YaBrowser/14.12.2125.10034 Safari/537.36

 35 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36 OPR/27.0.1689.76
 35 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0
 38 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
 42 Mozilla/5.0 (Windows NT 5.1; rv:35.0) Gecko/20100101 Firefox/35.0
 42 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
 43 Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
 45 Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36 OPR/27.0.1689.76
 50 Mozilla/5.0 (Windows NT 5.1; rv:36.0) Gecko/20100101 Firefox/36.0
 50 Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.74 Safari/537.36
 54 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
 56 Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
 62 Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:36.0) Gecko/20100101 Firefox/36.0
 70 Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
 70 Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
 91 Mozilla/5.0 (Windows NT 6.3; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0
 98 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/600.3.18 (KHTML, like Gecko) Version/8.0.3 Safari/600.3.18
100 Mozilla/5.0 (Windows NT 6.1; rv:35.0) Gecko/20100101 Firefox/35.0
108 Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
109 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36
114 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
150 Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:36.0) Gecko/20100101 Firefox/36.0
206 Mozilla/5.0 (Windows NT 6.1; rv:36.0) Gecko/20100101 Firefox/36.0
239 Mozilla/5.0 (Windows NT 6.3; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0
288 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0
329 Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
340 Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
389 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0
737 Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
964 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36

Ping .....

We have changed the schema in significant ways so these validation issues might no longer appear.

New schema events have to be tested using teh dev server in vagrant:https://www.mediawiki.org/wiki/Extension:EventLogging/Guide#Installing_the_EventLogging_devserver

In T91272#1123916, @Nuria wrote:

New schema events have to be tested using teh dev server in vagrant:https://www.mediawiki.org/wiki/Extension:EventLogging/Guide#Installing_the_EventLogging_devserver

Thanks @Nuria! I set this up and with the new patches and there doesn't seem to be any problem —

P438 popups event logs

1	# Listening to events.
2	{"wiki": "wiki", "uuid": "d75d992a19455af882471b0332e2ea34", "webHost": "localhost", "timestamp": 1427351613, "clientValidated": true, "recvFrom": "localhost", "seqId": 0, "clientIp": "36e213e00866c427c15e9e42d96aad67c6e69176", "schema": "Popups", "event": {"action": "dismissed", "popupEnabled": true, "duration": 877, "pageTitleSource": "Main Page", "pageTitleHover": "Star Wars"}, "revision": 11625443}
3	-- request ---------------------------------------------------------------
4	?%7B%22event%22%3A%7B%22pageTitleHover%22%3A%22Star%20Wars%22%2C%22pageTitleSource%22%3A%22Main%20Page%22%2C%22popupEnabled%22%3Atrue%2C%22action%22%3A%22dismissed%22%2C%22duration%22%3A877%7D%2C%22clientValidated%22%3Atrue%2C%22revision%22%3A11625443%2C%22schema%22%3A%22Popups%22%2C%22webHost%22%3A%22localhost%22%2C%22wiki%22%3A%22wiki%22%7D; localhost 0 2015-03-26T06:33:33 127.0.0.1
5	-- event -----------------------------------------------------------------
6	{
7	"clientIp": "36e213e00866c427c15e9e42d96aad67c6e69176",
8	"clientValidated": true,
9	"event": {
10	"action": "dismissed",
11	"duration": 877,
12	"pageTitleHover": "Star Wars",
13	"pageTitleSource": "Main Page",
14	"popupEnabled": true
15	},
16	"recvFrom": "localhost",
17	"revision": 11625443,
18	"schema": "Popups",
19	"seqId": 0,
20	"timestamp": 1427351613,
21	"uuid": "d75d992a19455af882471b0332e2ea34",
22	"webHost": "localhost",
23	"wiki": "wiki"
24	}
25	-- validation ------------------------------------------------------------
26	Valid.
27	--------------------------------------------------------------------------
28	{"wiki": "wiki", "uuid": "1343a93ed6af5240961781475a4c2e3e", "webHost": "localhost", "timestamp": 1427351616, "clientValidated": true, "recvFrom": "localhost", "seqId": 1, "clientIp": "36e213e00866c427c15e9e42d96aad67c6e69176", "schema": "Popups", "event": {"action": "dismissed", "popupEnabled": true, "duration": 1731, "pageTitleSource": "Main Page", "pageTitleHover": "Star Wars"}, "revision": 11625443}
29	-- request ---------------------------------------------------------------
30	?%7B%22event%22%3A%7B%22pageTitleHover%22%3A%22Star%20Wars%22%2C%22pageTitleSource%22%3A%22Main%20Page%22%2C%22popupEnabled%22%3Atrue%2C%22action%22%3A%22dismissed%22%2C%22duration%22%3A1731%7D%2C%22clientValidated%22%3Atrue%2C%22revision%22%3A11625443%2C%22schema%22%3A%22Popups%22%2C%22webHost%22%3A%22localhost%22%2C%22wiki%22%3A%22wiki%22%7D; localhost 1 2015-03-26T06:33:36 127.0.0.1
31	-- event -----------------------------------------------------------------
32	{
33	"clientIp": "36e213e00866c427c15e9e42d96aad67c6e69176",
34	"clientValidated": true,
35	"event": {
36	"action": "dismissed",
37	"duration": 1731,
38	"pageTitleHover": "Star Wars",
39	"pageTitleSource": "Main Page",
40	"popupEnabled": true
41	},
42	"recvFrom": "localhost",
43	"revision": 11625443,
44	"schema": "Popups",
45	"seqId": 1,
46	"timestamp": 1427351616,
47	"uuid": "1343a93ed6af5240961781475a4c2e3e",
48	"webHost": "localhost",
49	"wiki": "wiki"
50	}
51	-- validation ------------------------------------------------------------
52	Valid.
53	--------------------------------------------------------------------------

Bump!

Once you deploy the new code we can close this ticket.

The new code has been deployed.
Let's reopen it if needed.

Quiddity moved this task from Backlog to Done on the Page-Previews board.Jun 21 2015, 10:49 PM

Large number of popup events not validating Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Large number of popup events not validating
Closed, ResolvedPublic
Actions