We use the same whitelist in beta that we do in prod to keep events from going to MySQL. But there's nothing BUT MySQL for querying events in beta. We should allow events through! :)
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Default to not using eventlogging-valid-mixed filter | operations/puppet | production | +34 -16 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | MMiller_WMF | T205754 [EPIC] Growth: Understanding first day | |||
| Resolved | Etonkovidova | T207307 Understanding first day: testing and QA | |||
| Resolved | Ottomata | T208359 Allow (almost?) all EventLogging events to go into MySQL in beta |
Event Timeline
Comment Actions
Thanks for filing this, @Ottomata. I nested it under the Growth team's "Understanding first day" epic because it is blocking our ability to test our new EditorJourney schema.
Comment Actions
because it is blocking our ability to test our new EditorJourney schema.
It shouldn't, EditorJourney events on beta are available in kafka, and while having those in MYSQL will help, it is a nice-to-have, you can also consume directly from kafka in beta and see events and errors.
See:
https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/TestingOnBetaCluster#See_all_EventLogging_schema_Kafka_topics
https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/TestingOnBetaCluster#Validated_events
cc @Catrope
Comment Actions
Thanks for the links, @Nuria. I should have been more clear -- it is not strictly blocking our ability to test (since our engineers can look at Kafka), but it is blocking our ability to test thoroughly with the code we intend to do the analysis with. We'd like to add this capability, which I think aligns with what the Better Use of Data working group has come up with around test-driven development for EventLogging.
Comment Actions
Change 470862 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Default to not using eventlogging-valid-mixed filter, this is set to true in prod
Comment Actions
Change 470862 merged by Ottomata:
[operations/puppet@production] Default to not using eventlogging-valid-mixed filter
Comment Actions
Ok! I just disabled the whitelist filter in beta. Produce some events and they *should* show up in the MySQL database on deployment-eventlog05.
Comment Actions
hey hey look at that! :)
MariaDB [log]> show tables LIKE 'EditorJourney_%'; +---------------------------------+ | Tables_in_log (EditorJourney_%) | +---------------------------------+ | EditorJourney_18504997 | +---------------------------------+ 1 row in set (0.00 sec)
Comment Actions
Nice, nice, let's please update docs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/TestingOnBetaCluster