Page MenuHomePhabricator
Create Task
Maniphest T205759

Understanding first day: prototype instrumentation approach
Closed, ResolvedPublic
Actions

Description

Experiment with an approach to instrumentation that allows us to understand the complete flow of new editor activities right after account creation, while safeguarding user privacy.

Details

SubjectRepoBranchLines +/-
Fix parameter order for hash_hmacmediawiki/extensions/WikimediaEventsmaster+5 -3
Fix configuration variable name for wgWMEUnderstandingFirstDayoperations/mediawiki-configmaster+1 -1
Provide capability to log page views for users younger than 24 hoursmediawiki/extensions/WikimediaEventsmaster+665 -2
WIP: Proof-of-concept external URL redirection for event loggingmediawiki/coremaster+29 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

MMiller_WMF created this task.Sep 28 2018, 10:57 PM
JTannerWMF moved this task from Inbox to Q2 2019-20 on the Growth-Team board.Oct 1 2018, 3:18 PM
JTannerWMF moved this task from Q2 2019-20 to FY 2019-20 on the Growth-Team board.
MMiller_WMF assigned this task to kostajh.Oct 2 2018, 6:27 PM
MMiller_WMF edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.
MMiller_WMF moved this task from Incoming to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.
kostajh moved this task from Ready for Development to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 3 2018, 5:44 PM
MMiller_WMF edited parent tasks, added: T205763: Understanding first day: "Schema:EditorJourney" instrumentation specification; removed: T205758: Understanding first day: define questions, metrics, and data.Oct 3 2018, 10:35 PM
gerritbot subscribed.Oct 5 2018, 3:03 PM

Change 464432 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/WikimediaEvents@master] WIP: Log page data on each view

https://gerrit.wikimedia.org/r/464432

gerritbot added a project: Patch-For-Review.Oct 5 2018, 3:03 PM
kostajh moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 9 2018, 2:11 PM

@Catrope @SBisson I've moved this to code review so you all can provide some feedback. I'm not sure about adding API requests at this point, it seems like we would get an overwhelming number of events, but I could add that in if you want. (Or we could create a separate task for exploring that.)

kostajh moved this task from Code Review to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 10 2018, 4:04 PM
kostajh moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 12 2018, 9:03 PM
gerritbot added a comment.Oct 16 2018, 2:07 AM

Change 467553 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] WIP: Proof-of-concept external URL redirection for event logging

https://gerrit.wikimedia.org/r/467553

Catrope mentioned this in T206543: Make it possible to conditionally oversample Edit events.Oct 17 2018, 6:09 AM
gerritbot added a comment.Oct 17 2018, 5:41 PM

Change 467553 abandoned by Kosta Harlan:
WIP: Proof-of-concept external URL redirection for event logging

Reason:
Abandon in favor of 468047

https://gerrit.wikimedia.org/r/467553

kostajh mentioned this in T207115: Determine solution for logging clicks to external links.Oct 19 2018, 2:19 PM
kostajh added a project: Privacy.Oct 19 2018, 2:28 PM
MMiller_WMF closed subtask T207115: Determine solution for logging clicks to external links as Declined.Oct 22 2018, 6:01 PM
gerritbot added a comment.Oct 23 2018, 10:53 PM

Change 464432 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Provide capability to log page views for users younger than 24 hours

https://gerrit.wikimedia.org/r/464432

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.2; 2018-10-30).Oct 23 2018, 11:00 PM
gerritbot added a comment.Oct 24 2018, 5:36 PM

Change 469471 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[operations/mediawiki-config@master] Fix configuration variable name for wgWMEUnderstandingFirstDay

https://gerrit.wikimedia.org/r/469471

kostajh moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 24 2018, 5:44 PM
gerritbot added a comment.Oct 24 2018, 6:22 PM

Change 469471 merged by jenkins-bot:
[operations/mediawiki-config@master] Fix configuration variable name for wgWMEUnderstandingFirstDay

https://gerrit.wikimedia.org/r/469471

kostajh moved this task from QA to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 24 2018, 9:20 PM
gerritbot added a comment.Oct 24 2018, 9:21 PM

Change 469520 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/WikimediaEvents@master] Fix parameter order for hash_hmac

https://gerrit.wikimedia.org/r/469520

gerritbot added a comment.Oct 24 2018, 10:30 PM

Change 469520 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Fix parameter order for hash_hmac

https://gerrit.wikimedia.org/r/469520

kostajh moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.Oct 25 2018, 2:56 PM
Etonkovidova added a comment.Oct 29 2018, 11:50 PM

Looking into eventlogging on betalabs:

(1) the events are recorded only client-side-events.log

@deployment-eventlog05:/srv/log/eventlogging$ grep 'schema.*EditorJourney.*' all-events.log | wc 
      0       0       0
@deployment-eventlog05:/srv/log/eventlogging$ grep 'schema.*EditorJourney.*' client-side-events.log | wc 
    162     162  168496

(2) all-events.log does not record any events:

-rw-r--r-- 1 eventlogging eventlogging         0 Oct 26 06:25 all-events.log
drwxrwxr-x 2 eventlogging eventlogging      4096 Oct 29 06:25 archive
-rw-r--r-- 1 eventlogging eventlogging 240502691 Oct 29 23:38 client-side-events.log
drwxr-xr-x 2 eventlogging eventlogging      4096 Oct 29 06:25 systemd

(3) I did not see any failures of EditorJourney in systemd :

@deployment-eventlog05:/srv/log/eventlogging$ grep  'EditorJourney.*'  systemd/eventlogging-processor@client-side-01.log |wc
  0       0       0
nettrom_WMF mentioned this in T206182: Understanding first day: write analytics queries in advance.Oct 30 2018, 12:04 AM
Etonkovidova closed this task as Resolved.Nov 1 2018, 8:33 PM

Verified "EditAttemptStep" and "EdiorJourney" (betalabs) - the events are recorded and stored in log db on @deployment-eventlog05.

sbassett triaged this task as Medium priority.Oct 16 2019, 4:38 PM
sbassett moved this task from Intake to Done on the Privacy board.
sbassett removed a project: Patch-For-Review.
Aklapper removed a subscriber: Growth-Team.May 16 2023, 10:22 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits