Page MenuHomePhabricator

Analyze UploadWizard funnel logs
Closed, DuplicatePublic

Description

Now that Javascript errors have been added to the funnel logs (T94428), make another pass of analyzing the results, try to identify whether Javascript errors play a large role in abandonment (and if so, which ones), and modify the tasks in T91652 as needed.

Event Timeline

Tgr claimed this task.
Tgr raised the priority of this task from to Needs Triage.
Tgr updated the task description. (Show Details)
Tgr added a project: UploadWizard.
Tgr subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

The first full day's data:

mysql:research@analytics-store.eqiad.wmnet [log]> select event_message, count(*) count from UploadWizardExceptionFlowEvent_11717009 where timestamp like '20150408%' group by event_message order by count desc;
+------------------------------------------------------------------------------------------------------------------+-------+
| event_message                                                                                                    | count |
+------------------------------------------------------------------------------------------------------------------+-------+
| Unable to get property 'call' of undefined or null reference                                                     |   527 |
| Die Eigenschaft "call" eines undefinierten oder Nullverweises kann nicht abgerufen werden.                       |   362 |
| Script error.                                                                                                    |   153 |
| 未定義または NULL 参照のプロパティ 'call' は取得できません                                                              |   140 |
| 정의되지 않음 또는 null 참조인 'call' 속성을 가져올 수 없습니다.                                                          |   126 |
| Error: Unable to parse title                                                                                     |   113 |
| Unable to get value of the property 'call': object is null or undefined                                          |   107 |
| Det går inte att hämta värdet för egenskapen call: objektet är null eller odefinierat                            |    81 |
| No se puede obtener la propiedad 'call' de referencia nula o sin definir                                         |    71 |
| Uncaught Error: Unable to parse title                                                                            |    66 |
| Impossible d’obtenir la propriété  « call » d’une référence null ou non définie                                  |    51 |
| Egenskaben 'call' kan ikke hentes for reference, der er udefineret eller null                                    |    50 |
| Nem lehet lekérni a(z) call tulajdonságot, amelyhez nem definiált vagy null értékű hivatkozás tartozik           |    42 |
| Não é possível obter a propriedade 'call' de referência indefinida ou nula                                       |    42 |
| Uncaught TypeError: undefined is not a function                                                                  |    39 |
| Für die Eigenschaft "call" kann kein Wert abgerufen werden: Das Objekt ist Null oder undefiniert                 |    28 |
| Impossibile recuperare la proprietà 'call' di un riferimento nullo o non definito                                |    19 |
| No se puede obtener valor de la propiedad 'call': el objeto es nulo o está sin definir                           |    17 |
| Uncaught TypeError: Cannot read property 'code' of undefined                                                     |    17 |
| Impossible d’obtenir la valeur de la propriété  « call » : objet null ou non défini                              |    17 |
| プロパティ 'call' の値を取得できません: オブジェクトは Null または未定義です。                                             |    16 |
| Script error                                                                                                     |    13 |
| NS_ERROR_NOT_AVAILABLE:                                                                                          |    11 |
| 'call' 속성 값을 가져올 수 없습니다. 개체가 null이거나 정의되지 않았습니다.                                                 |     4 |
| Uncaught TypeError: Cannot read property 'getPrefixedText' of null                                               |     4 |
| Error: Unknown dependency: ext.gadget.shortcuts                                                                  |     2 |
| Uncaught TypeError: Cannot read property 'valid' of undefined                                                    |     2 |
| オブジェクトは 'resolve' プロパティまたはメソッドをサポートしていません。                                                   |     2 |
| Error: tried to reject a fulfilled promise                                                                       |     2 |
| Object doesn't support property or method 'resolve'                                                              |     2 |
| TypeError: undefined is not a function (evaluating '$.Deferred.resolve()')                                       |     2 |
| 無法取得未定義或 Null 參考的屬性 'call'                                                                              |     1 |
| Kan de eigenschap call van een niet-gedefinieerde verwijzing of een verwijzing naar een lege waarde niet ophalen |     1 |
| Impossibile recuperare la proprietà 'getPrefixedText' di un riferimento nullo o non definito                     |     1 |
+------------------------------------------------------------------------------------------------------------------+-------+
  • ~200 "Script error" (these are errors where the details are suppressed by same-domain security limitations)
  • ~ 1700 messages about 'call' not being defined
  • ~200 "Unable to parse title" errors
  • ~50 other

None of the frequent errors (count > 10) have a useful URL/line/column location. URL is either null or Special:UploadWizard or [some jquery location] > eval - these are probably the various ways in which browsers report eval-ed code. This is not entirely surprising as ResourceLoader caches its code in localStorage and then evals it which causes the connection with the original file to be lost.

After filtering out error logs which are useless for identifying the source,, there is actually nothing left:

mysql:research@analytics-store.eqiad.wmnet [log]> select event_message, event_url, event_line, event_column, count(*) count from UploadWizardExceptionFlowEvent_11717009 where timestamp like '20150408%' and event_url not in ('', 'undefined') and event_url not like '%Special:UploadWizard%' and event_url not like '%> eval' and event_message not in ('Script error', 'Script error.', 'NS_ERROR_NOT_AVAILABLE: ') group by event_url, event_line, event_column order by count desc \G
*************************** 1. row ***************************
event_message: Error: Unknown dependency: ext.gadget.shortcuts
    event_url: https://bits.wikimedia.org/commons.wikimedia.org/load.php?debug=false&lang=pl&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=20150407T185357Z
   event_line: 156
 event_column: 828
        count: 2
*************************** 2. row ***************************
event_message: Error: tried to reject a fulfilled promise
    event_url: https://pstatic.eshopcomp.com/nwp/v0_0_502/release/Shared/SharedApp.js
   event_line: 2
 event_column: 2657
        count: 2
2 rows in set (0.00 sec)

We will need either T90524 or T91347 (to log full stack traces) to proceed.

On the plus side, Timo was correct that CORS limitations are not a big deal due to RL localStorage caching (for editor--facing interfaces like UW, at least).

Flow event logging was apparently broken by the refactor:

mysql:research@analytics-store.eqiad.wmnet [log]> select count(*) from UploadWizardStep_8851805 where timestamp like '20150408%' and event_step = 'thanks' \G
count(*): 1999

mysql:research@analytics-store.eqiad.wmnet [log]> select count(*) from UploadWizardFlowEvent_11562780 where timestamp like '20150408%' \G
count(*): 166

mysql:research@analytics-store.eqiad.wmnet [log]> select distinct event_event from UploadWizardFlowEvent_11562780 where timestamp like '20150408%';
+------------------------------+
| event_event                  |
+------------------------------+
| retry-uploads-button-clicked |
| flickr-upload-button-clicked |
+------------------------------+

While this should be redone with more data (we only have a few dozen upload sessions for the first day), some stats about aborted UW sessions:

mysql:research@analytics-store.eqiad.wmnet [log]> create table staging.UploadWizard_flow_events ( timestamp varbinary(14) not null, flowId float not null, type varchar(32) not null, data varchar(256) not null);
Query OK, 0 rows affected (0.02 sec)

mysql:research@analytics-store.eqiad.wmnet [log]> insert into staging.UploadWizard_flow_events (timestamp, flowId, type, data) select timestamp, event_flowId as flowId, 'step' as type, event_step as data from UploadWizardStep_8851805 where timestamp like '20150408%' and event_skipped is null UNION ALL select timestamp, event_flowId as flowId, 'event' as type, event_event as data from UploadWizardFlowEvent_11562780 where timestamp like '20150408%' UNION ALL select timestamp, event_flowId as flowId, 'olderror' as type, event_code as data from UploadWizardErrorFlowEvent_9924376 where timestamp like '20150408%' UNION ALL select timestamp, event_flowId as flowId, 'jserror' as type, event_message as data from UploadWizardExceptionFlowEvent_11717009 where timestamp like '20150408%';
Query OK, 15323 rows affected (0.36 sec)
Records: 15323  Duplicates: 0  Warnings: 0

mysql:research@analytics-store.eqiad.wmnet [log]> select type, count(*) from staging.UploadWizard_flow_events join (select max(timestamp) max_timestamp, flowId id from staging.UploadWizard_flow_events group by flowId) last_event_time on flowId = id and timestamp = max_timestamp group by type;
+----------+----------+
| type     | count(*) |
+----------+----------+
| event    |        1 |
| jserror  |        1 |
| olderror |       13 |
| step     |       36 |
+----------+----------+
4 rows in set (0.02 sec)

mysql:research@analytics-store.eqiad.wmnet [log]> select count(*) from staging.UploadWizard_flow_events e1 left join staging.UploadWizard_flow_events e2 on e1.flowId = e2.flowId and e1.type = 'jserror' and e2.type = 'step' and e2.timestamp > e1.timestamp where e1.type = 'jserror' and e2.flowId is null\G
*************************** 1. row ***************************
count(*): 1

drop table staging.UploadWizard_flow_events;

i. e. almost all sessions end in a step event (which presumably means either success or UX-related abortion) or an API or form validation error. There is a single session (within 50 or so) where a Javascript error happens and is not followed by a step event.

On closer inspection, the data is probably flawed due to frequent flowId collisions - flowId is a float and the numbers used as ids are large enough to lose precision.

Change 203337 had a related patch set uploaded (by Gergő Tisza):
Fix several issues with funnel logging

https://gerrit.wikimedia.org/r/203337

Change 203337 merged by jenkins-bot:
Fix several issues with funnel logging

https://gerrit.wikimedia.org/r/203337