Page MenuHomePhabricator

Streams with empty configs should be rendered as {} in the JSON returned by StreamConfig API
Closed, ResolvedPublicBUG REPORT

Description

Came across this issue when deserializing the JSON retrieved from streamconfigs API in the Event Platform Client Library for iOS. Streams that are in the config but don't have any configs are rendered as [] (bad) instead of {} (good). Please fix this, thank you.

Steps to Reproduce:

https://meta.wikimedia.org/w/api.php?action=streamconfigs&format=json

Actual Results:

{
  "streams": {
    "eventlogging_SearchSatisfaction": {
      "destination_event_service": "eventgate-analytics-external"
    },
    "mediawiki.client.error": {
      "destination_event_service": "eventgate-logging-external"
    },
    "kaios_app.error": {
      "destination_event_service": "eventgate-logging-external"
    },
    "test.event": [],
    ...
  }
}

Expected Results:

{
  "streams": {
    "eventlogging_SearchSatisfaction": {
      "destination_event_service": "eventgate-analytics-external"
    },
    "mediawiki.client.error": {
      "destination_event_service": "eventgate-logging-external"
    },
    "kaios_app.error": {
      "destination_event_service": "eventgate-logging-external"
    },
    "test.event": {},
    ...
  }
}

Testing with https://pai-test.wmflabs.org/streams for now

Related Objects

StatusSubtypeAssignedTask
Openjlinehan
OpenNone
ResolvedOttomata
OpenOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
DuplicateOttomata
ResolvedOttomata
Resolvedmforns
ResolvedGehel
ResolvedMilimetric
ResolvedOttomata
OpenOttomata
OpenOttomata
DeclinedOttomata
ResolvedOttomata
OpenOttomata
ResolvedGilles
DeclinedOttomata
ResolvedOttomata
Resolvedmforns
ResolvedMholloway
ResolvedOttomata
DuplicateNone
DuplicateNone
DuplicateNone
DuplicateNone
ResolvedMholloway
DuplicateNone
ResolvedOttomata
OpenSBisson
OpenSBisson
OpenSBisson
Resolvedmforns
ResolvedOttomata
Resolvedmforns
ResolvedOttomata
ResolvedOttomata
DeclinedNone
DeclinedNone
Openbmansurov
ResolvedJAllemandou
Resolvedmforns
ResolvedOttomata
ResolvedOttomata
OpenOttomata
DuplicateNone
ResolvedOttomata
OpenMNeisler
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
ResolvedOttomata
Openjlinehan
Openjlinehan
Resolvedkzimmerman
Resolvedmpopov
DeclinedOttomata
ResolvedOttomata
Openjlinehan
Openmforns
Openmforns
OpenNone
Resolvedmpopov
ResolvedBUG REPORTMholloway
OpenSBisson

Event Timeline

mpopov triaged this task as High priority.Aug 7 2020, 7:25 PM
mpopov moved this task from Inbox to Watching on the Product-Data-Infrastructure board.

Thanks, sounds like a classic PHP can't tell the difference between and empty object and an empty list bug.

mforns moved this task from Incoming to Event Platform on the Analytics board.
mforns added a project: Analytics-Kanban.

I have a strong suspicion that this bug also affects mw.eventLog.submit() because:

  • mw.eventLog.streamInSample('test.event') returns false
  • mw.eventLog.streamConfig('test.event') returns null
  • mw.eventLog.streamConfig('eventlogging_SearchSatisfaction') returns { "destination_event_service": "eventgate-analytics-external" }
  • mw.eventLog.streamInSample('eventlogging_SearchSatisfaction') returns true

streamInSample() doesn't use streamConfig() (although there's an argument to be made that it should) and checks config directly via config.streamConfigs[ streamName ] === undefined, so "test.event" stream should be in-sample because it is in the config and so we would expect config.streamConfigs['test.event'] to be {}, but it must be evaluating to undefined which causes that stream to be marked as not in sample

I have a strong suspicion that this bug also affects mw.eventLog.submit() because:

  • mw.eventLog.streamConfig('test.event') returns null

This is not possible if the value of config.streamConfigs[ 'test.event' ] is {} or []. These are truthy in JS. See code. The only possibility is if $.extend( true, {}, {} ) or $.extend( true, {}, [] ) evaluate to null, which I don't think that they do.

  • mw.eventLog.streamInSample('test.event') returns false

Again, this does not seem possible unless there is, in fact, no key 'test.event' in config.streamConfigs.

Are you sure that 'test.event' is really defined?

Nevermind, test.event is a stream in wgEventStreams but is not registered with EventLogging in wgEventLoggingStreamNames. Okay, so we're good on MediaWiki! Sorry for the false alarm.

This is still a blocker for apps clients, though.

I don't think it is; it is not listed in wgEventLoggingStreamNames. BTW, I am removing 'test.event' as a stream; we are moving towards eventgate service specific test event streams, e.g. 'eventgate-analytics-external.test.event'.

BTW, I am removing 'test.event' as a stream; we are moving towards eventgate service specific test event streams, e.g. 'eventgate-analytics-external.test.event'.

Question about data lifecycle… After you remove that stream, if we configure a "test.event" stream (as part of T259714) using an analytics test schema (with client_dt, session_id, device_id, pageview_id fields), what happens to the existing test_event table in the database? If a stream name is re-used with a totally different schema, what happens?

There's nothing special about the test event streams, other than eventgate services know how to produce them. You can certainly make an analytics specific test schema and stream.

But what happens to the existing table in the database that was created from the old stream under a different schema? Does it need to be manually dropped via Hive CLI before data produced to the replacement stream gets stored? Can anyone drop it from the events database (and events_sanitized, for that matter) or just your team?

I guess we can drop it, but we don't really care about the data in test.event. It's just test data. The table will stick around, but no more data will be written to it.

under a different schema?

BTW, the schema is the same one (test/event), they are just new stream names.

I was just using test.event as an immediate example, but the question I'm asking is about what happens in general when a stream name is re-used with a different, potentially incompatible schema. A team expecting their instrument's events to show up in the database would be surprised to learn that none of that data is making it in because they just happened to name their stream the same name as a previously configured-and-then-deleted stream and the old table/data was never dropped when the stream was removed from the config.

what happens in general when a stream name is re-used with a different, potentially incompatible schema.

Bad stuff, it won't work, unless we do the work to totally purge any previous artifacts of that stream, like its Hive table.

But I see your point, a collision due to removing the stream config could be a problem. I guess this is true of legacy EventLogging metawiki schemas too; someone could delete a schema page, and then later someone could create a new one with the same name.

what happens in general when a stream name is re-used with a different, potentially incompatible schema.

Bad stuff, it won't work, unless we do the work to totally purge any previous artifacts of that stream, like its Hive table.

Thoughts https://wikitech.wikimedia.org/wiki/Event_Platform/Instrumentation_How_To#Decommissioning including a bit on filing a Phab request for Analytics to delete the associated tables from both databases when removing a stream from the config?

Should streams specify owners in their configs, especially for cases when the person/team who created the stream is not the same as the person who removes it? Or, say Analytics removes a stream because the system is getting flooded. Would y'all use git blame to figure out who added that stream and then check with them how they want to handle retention/dropping of any collected data? Seems we should coordinate on a policy for that stuff.

Also this conversation has gone way off-topic from the actual phab task and we should continue it elsewhere.

Should streams specify owners in their configs, especially for cases when the person/team who created the stream is not the same as the person who removes it? Or, say Analytics removes a stream because the system is getting flooded. Would y'all use git blame to figure out who added that stream and then check with them how they want to handle retention/dropping of any collected data? Seems we should coordinate on a policy for that stuff.

This is a larger issue and an important one but we should create a new ticket about tracking/identifying data responsibility that can capture this as well as other issues around this.

This is one of the user stories listed in T205319: Modern Event Platform: Stream Configuration, so indeed stream config is probably the place to do this. However, we should probably coordinate with some bigger data governance projects that both PA and AE have mentioned wanting to work on. E.g. where does this type of information fit in if we were to use something like Apache Atlas? Will we build some kind of EventStreamConfig + Atlas (or whatever) integration?

I'm closing this ticket because it's no longer necessary to fix this issue. With all stream consolidated into a single config per T251935, it is necessary for EPC-iOS & EPC-Android to include &constraints=destination_event_service=eventgate-analytics-external in the stream config API request to only retrieve relevant stream configs. The primary purpose this will serve is to lighten the load (in a major way!), while also ensuring that only destination_event_service: "eventgate-analytics-external" streams are produced to, since the "intake-analytics.wikimedia.org/v1/events" URI is baked into the library.

As such, the response will never include streams with empty configs in practice.

Ottomata reopened this task as Open.EditedAug 19 2020, 4:38 PM

Ah, I'd like to keep this open! It is a real bug and shouldn't happen. What if the constraints is something else that really does return no streams? We can keep this as low priority, but hopefully we'll fix it soonish.

Example:

$ curl -s 'https://meta.wikimedia.org/w/api.php?action=streamconfigs&format=json&constraints=woo=yeah' | jq .
{
  "streams": []
}

Expected:

$ curl -s 'https://meta.wikimedia.org/w/api.php?action=streamconfigs&format=json&constraints=woo=yeah' | jq .
{
  "streams": {}
}
mpopov lowered the priority of this task from High to Low.Aug 19 2020, 4:43 PM

Good call/catch!

mpopov raised the priority of this task from Low to Unbreak Now!.Aug 27 2020, 4:14 PM

Okay, it seems destination_event_service is now omitted in the stream configs (fair enough) but that means all returned streams are now [], which is blocking the MEP iOS pilot:

$ curl -s 'https://meta.wikimedia.org/w/api.php?action=streamconfigs&format=json&constraints=destination_event_service=eventgate-analytics-external' | jq .
{
  "streams": {
    "eventlogging_SearchSatisfaction": [],
    "eventlogging_TemplateWizard": [],
    "eventlogging_Test": [],
    "eventgate-analytics-external.test.event": [],
    "eventgate-analytics-external.error.validation": []
  }
}

Change 622881 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/extensions/EventStreamConfig@master] Recursively set array type 'assoc' on ApiStreamConfig results

https://gerrit.wikimedia.org/r/622881

Mholloway lowered the priority of this task from Unbreak Now! to High.Aug 28 2020, 2:07 AM

Chatted with @mpopov and this doesn't need to be deployed on an emergency basis, so I'll be bold and drop this to High.

Change 622881 merged by Ottomata:
[mediawiki/extensions/EventStreamConfig@master] Recursively set array type 'assoc' on ApiStreamConfig results

https://gerrit.wikimedia.org/r/622881

mpopov reassigned this task from fdans to Mholloway.
mpopov added a subscriber: fdans.

Thanks, Michael! I pulled latest EventStreamConfig for our dev wiki and tested it with the stream configs that are there:

$ curl -s 'https://epc-test.wmcloud.org/w/api.php?action=streamconfigs&format=json' | jq .
{
  "streams": {
    "test.instrumentation": {},
    "test.instrumentation.sampled": {
      "sampling": {
        "rate": 0.5
      }
    },
    "ios.edit_history_compare": {}
  }
}

Looking forward to being able to use the official endpoint at metawiki.