Fri, Apr 2
Thu, Apr 1
The fixed patch finally rolled out on Tuesday 3/30 with 1.36.0-wmf.36. It looks like it's almost completely eliminated duplicate entries from the PrefUpdate table, down from several hundred duplicate events per day prior to the change. Yesterday there were no duplicate events, and today so far there is only one duplicate popupsreferencepreviews change.
Wed, Mar 31
Mon, Mar 22
No worries, @nray! Thanks for the update.
Mon, Mar 15
Since it's a near-term follow-up, my vote would be to keep this one open.
Mar 12 2021
Mar 11 2021
Ah, sorry, not happening anymore. Looks like this was only occurring for a brief window at 22:30.
I'm seeing a lot of this in the logs, too, and it doesn't make any sense to me, because the patch updates both signatures:
From my preliminary spot checking, the patch appears to have eliminated duplicates on the group 0 and 1 wikis to which it's been deployed so far. I look forward to seeing the results after it's promoted to group 2 this afternoon.
Mar 10 2021
Looking good! Thank you, @hashar!
To add a data point, my new environment is still working well starting from a cold boot this morning.
Mar 9 2021
OK, I've started from a fresh slate with DB replication and things are looking better. When I take the environment down and spin it back up, there are a few of the same errors from the job runner container, but it eventually connects after the replica DB finishes starting up. With the single-MySQL installation, the job runner just kept spewing the same error indefinitely. Maybe for some reason it's expecting a replica DB that doesn't exist?
Caveat: I'm not positive that my wiki was configured correctly when I tried setting up replication earlier. I'll try setting up a fresh installation using DB replication from the start and see if I have better luck. In any case this is a bug for installations using a single MySQL instance.
@Mayakp.wiki, I've added some general info on ad blocker effects on event logging on-wiki at https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging#Ad_blockers. In a nutshell, if your ad blocker uses the EasyPrivacy block list, then requests to our analytics event intake services will be blocked. There may be others that we don't yet know about.
Mar 8 2021
@Mayakp.wiki You should now be a member of the data-qa group on Test Wikipedia. You can verify this by viewing your current groups here: https://test.wikipedia.org/w/api.php?action=query&meta=userinfo&uiprop=groups
Mar 5 2021
deployment-mcs01 is obsolete, superseded by deployment-docker-mobileapps01. I'll shut it down momentarily.
This ship has sailed. Declining as non-actionable.
Thanks for the explanation and pointers, @nray. That clears things up.
OK, the code for this is teed up. The MW config change shouldn't be merged/deployed until next week because of the no-Friday-deploys rule, but the WikimediaEvents change can be merged anytime. Once the config change lands, I'll add @Mayakp.wiki to the data-qa group on testwiki (and also myself, to ensure that it works first) using createAndPromote.php.
Mar 4 2021
Oops, assigned to @mforns. I teed up a patch for you. ;)
Mar 3 2021
I just took a quick look, and it seems to be working fine. For example, just now, I set the source language to English and the target language to Esperanto, entered "Town" in the search box, and hit enter, and got several results, just as I'd expect.
@kzimmerman Updated the wiki; please let me know if that looks good!
Mar 2 2021
Verified that timestamps are now coming in with expected format and time zone.
hive (event)> select dt, client_dt, meta.dt as meta_dt from serversideaccountcreation where year = 2021 and month = 3 and day = 2 order by meta_dt desc limit 10; <snip> OK dt client_dt meta_dt 2021-03-02T18:59:52Z 2021-03-02T18:59:48Z 2021-03-02T18:59:52.534Z 2021-03-02T18:59:44Z 2021-03-02T18:59:40Z 2021-03-02T18:59:44.410Z 2021-03-02T18:59:43Z 2021-03-02T18:59:41Z 2021-03-02T18:59:43.909Z 2021-03-02T18:59:25Z 2021-03-02T18:59:19Z 2021-03-02T18:59:25.190Z 2021-03-02T18:59:23Z 2021-03-02T18:59:20Z 2021-03-02T18:59:23.166Z 2021-03-02T18:59:21Z 2021-03-02T18:59:17Z 2021-03-02T18:59:21.948Z 2021-03-02T18:59:12Z 2021-03-02T18:59:08Z 2021-03-02T18:59:12.697Z 2021-03-02T18:59:10Z 2021-03-02T18:59:07Z 2021-03-02T18:59:10.122Z 2021-03-02T18:59:05Z 2021-03-02T18:59:03Z 2021-03-02T18:59:05.412Z 2021-03-02T18:58:54Z 2021-03-02T18:58:49Z 2021-03-02T18:58:54.788Z Time taken: 44.371 seconds, Fetched: 10 row(s)
Feb 25 2021
Feb 24 2021
No, it shouldn't be a problem. I was being extremely conservative when I wrote that script, more because I was worried about blowing past our Google Cloud Vision budget than for any other reason. Either updating the script to be more efficient or running multiple instances in parallel should be fine.
Feb 23 2021
Whoops, moving back to signoff for now.
Faulty bug tag, sorry!
The behavior of getRandomValues is specified here: https://www.w3.org/TR/WebCryptoAPI/#Crypto-method-getRandomValues,
and the Gecko implementation is here: https://github.com/mozilla/gecko-dev/blob/0db73daa4b03ce7513a7dd5f31109143dc3b149e/dom/base/Crypto.cpp#L38.
Feb 22 2021
I added a new section https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging#Ad_blockers to the EventLogging docs.
Feb 19 2021
Feb 17 2021
OK, after running a couple of queries for the past couple of days, it looks like the worst offender is in fact mf_amc_optin, followed by popupsreferencepreviews, then discussiontools-betaenable, then a handful of others with a duplicate or two per day.
I've been poking at this as time permits but haven't yet managed to track down a cause. Interestingly, eyeballing some data from yesterday, it seems to happen particularly often for popupsreferencepreviews events.
Feb 16 2021
I believe this task has become invalid since mobileapps was migrated to Kubernetes.
Feb 12 2021
I picked up @Tgr's old patch and updated it to consume data from event.mediawiki_api_request, along with some other cosmetic changes. The HQL scripts seem to work well as updated. I also kicked off an Oozie job (using mholloway as the target database) but got cold feet and killed it.
Feb 10 2021
Further demo work to be continued in T274175: [Client libraries] Demonstrate new capabilities.
Feb 9 2021
I could probably pick this up as a 10%-ish exercise, if someone (@Milimetric?) can spare some code review cycles.
Looking at the data collected yesterday, 2/8/21, on the group 0 and 1 wikis:
We discussed this briefly in a board grooming meeting yesterday and would appreciate a review by the iOS team about which of the remaining items are still needed or wanted, and how they should be prioritized (individually or collectively). As far as I can tell, the remaining "possibly moot if persistence is solved" items still remain open and valid, but I could be wrong.
Feb 1 2021
Removed the [spike] label as this is really more of a bikeshed task. The implementation will be straightforward.
I would support using a map-typed labels field for this. As described in the ECS spec it fits our intended purpose rather well, I think, and it gets us around the question of what does and doesn't constitute metadata. What do you think, @jlinehan?