Page MenuHomePhabricator

Many errors on "MobileWikiAppiOSSearch" and "MobileWikiAppiOSUserHistory"
Closed, ResolvedPublic

Description

These two schemas have a pretty high volume of errors, at times many errors per second.

You can see errors on event database on eventerror table, Here is an example. There are two wrong field names: type_of_search and app_install_ID neither of which match schema column names: https://meta.wikimedia.org/wiki/Schema:MobileWikiAppiOSSearch

{"event": {"code": "validation", "message": "Additional properties are not allowed (u'app_install_ID', u'ts', u'type_of_search' were unexpected)", "rawEvent": "?%7B%22revision%22%3A18289062%2C%22event%22%3A%7B%22action%22%3A%22results%22%2C%22number_of_results%22%3A6%2C%22ts%22%3A%222018-10-17T21%3A19%3A07%2B01%3A00%22%2C%22primary_language%22%3A%22en%22%2C%22session_id%22%3A%2289DB9A73-2D4B-40BA-A346-9DD4AE79DB3F%22%2C%22session_token%22%3A%22825529FA-7BCF-4AD8-ADE8-9B14A9FFED65%22%2C%22app_install_ID%22%3A%22FF8AA8ED-0CEE-43CE-AB32-6FED18B7B437%22%2C%22time_to_display_results%22%3A404%2C%22type_of_search%22%3A%22prefix%22%2C%22is_anon%22%3Atrue%7D%2C%22schema%22%3A%22MobileWikiAppiOSSearch%22%2C%22wiki%22%3A%22enwiki%22%7D\tcp3042.esams.wmnet\t275283408\t2018-10-18T20:59:59\t82.132.223.121\t\"WikipediaApp/6.0.2.1493 (iOS 11.4.1; Phone)\"", "revision": 18289062, "schema": "MobileWikiAppiOSSearch"}, "recvFrom": "eventlog1002.eqiad.wmnet", "revision": 14035058, "schema": "EventError", "timestamp": 1539896400, "uuid": "c6d49544d31811e8b7e61418775b0d42", "wiki": ""}

Please have in mind that these types of validation errors can be tested on beta before launching to prod:
https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/TestingOnBetaCluster#How_to_log_a_client-side_event_to_Beta_Cluster_directly

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
chelsyx added a subscriber: NHarateh_WMF.

Thanks @Nuria !

@NHarateh_WMF Can you fix the name of these fields?
For https://meta.wikimedia.org/wiki/Schema:MobileWikiAppiOSSearch
"app_install_ID" -> "app_install_id"
"ts" -> "event_dt"
"type_of_search" -> "search_type"

elukey triaged this task as High priority.Oct 19 2018, 5:49 AM
Nuria raised the priority of this task from High to Unbreak Now!.Nov 1 2018, 4:09 PM

Errors from this schemas have fired up again as of today.

I confirmed MobileWikiAppiOSSearch errors but I don't see any errors associated with MobileWikiAppiOSUserHistory. @Nuria, could you paste one of the MobileWikiAppiOSUserHistory error logs?

@NHarateh_WMF
Errors are available in hive (but also in subtask hanging from this ticket). See: https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging#Errors_for_schemas

hive> select * from eventerror where event.schema like 'MobileWikiApp%' and year=2018 and month=11 and day=1 limit 10;

An example:

{"code":"validation","message":"u'predeterminado' is not one of ['default', 'sepia', 'dark', 'black']","rawEvent":"?%7B%22revision%22%3A18222579%2C%22event%22%3A%7B%22readinglist_showdefault%22%3Afalse%2C%22theme%22%3A%22predeterminado%22%2C%22app_install_id%22%3A%2217AB37C6-A8A5-4795-BAC8-CA18FE945D34%22%2C%22measure_readinglist_listcount%22%3A1%2C%22event_dt%22%3A%222018-11-01T00%3A27%3A15%2B01%3A00%22%2C%22feed_enabled_list%22%3A%7B%22pd%22%3Atrue%2C%22ns%22%3A%7B%22on%22%3A%5B%22es%22%2C%22en%22%5D%7D%2C%22od%22%3A%7B%22on%22%3A%5B%22es%22%2C%22en%22%5D%7D%2C%22pl%22%3A%7B%22on%22%3A%5B%22es%22%2C%22en%22%5D%7D%2C%22rp%22%3Atrue%2C%22fa%22%3A%7B%22on%22%3A%5B%22es%22%2C%22en%22%5D%7D%2C%22cr%22%3Atrue%2C%22rd%22%3A%7B%22on%22%3A%5B%22es%22%2C%22en%22%5D%7D%2C%22tr%22%3A%7B%22on%22%3A%5B%22es%22%2C%22en%22%5D%7D%7D%2C%22is_anon%22%3Atrue%2C%22primary_language%22%3A%22en%22%2C%22search_tab%22%3Afalse%2C%22measure_readinglist_itemcount%22%3A0%2C%22measure_font_size%22%3A80%2C%22readinglist_sync%22%3Afalse%2C%22feed_disabled%22%3Afalse%2C%22session_id%22%3A%225C68078A-C0CF-4650-BF1F-06BFF7363C8B%22%2C%22trend_notify%22%3Afalse%7D%2C%22schema%22%3A%22MobileWikiAppiOSUserHistory%22%2C%22wiki%22%3A%22enwiki%22%7D\tcp3030.esams.wmnet\t361031661\t2018-10-31T23:27:15\t79.156.187.53\t\"WikipediaApp/6.0.2.1493 (iOS 12.0.1; Phone)\"","revision":18222579,"schema":"MobileWikiAppiOSUserHistory"}

@Nuria @NHarateh_WMF
The bug in MobileWikiAppiOSSearch and the bug in MobileWikiAppiOSUserHistory (T207427) should be fixed in v6.1 (see https://github.com/wikimedia/wikipedia-ios/pull/2700 and https://github.com/wikimedia/wikipedia-ios/pull/2703), which was released yesterday. We should start to see less of these errors as users starting to upgrade their apps to v6.1

Its now been 5 days of updates to 6.1, can someone verify this remains an "Unbreak Now!" issue?

@JMinor I just checked the errors for v6.1 in the eventerror table:

  • For MobileWikiAppiOSSearch, we still need to add a source field to identify the source from which the search interface was invoked: the top of feed, the search tab, or from the article view. This was stated in T195937, I reopened that ticket.
  • No more validation error in MobileWikiAppiOSUserHistory

This UBN! has not had activity for about a week, what is the status?

v6.1.1 was released on Nov 16, and I verified there was no validation error in this new version. Therefore, this ticket can be closed.