Looks good to me, the only missing part is save-attempt events vs save-complete events for the different edit funnels (CTA1, Section Edit, Tab Edit).

I've updated the specs on this page, based on Stage 3 events:
http://meta.wikimedia.org/wiki/Research:Article_feedback/Clicktracking#CTA1_update_.28October_2012.29

Let me know if you have any question

Dario: pushed to Gerrit (https://gerrit.wikimedia.org/r/#/c/26782/) & prototype. Any change you can test in on prototype already.
There's a few changes to the events you proposed at http://meta.wikimedia.org/wiki/Research:Article_feedback/Clicktracking#CTA1_update_.28October_2012.29

• the "-bottom" suffix is removed from all events, since - with the disappearance of the overlay - bottom is all there is left
• cta_learn_more will not be shown if it's at 0%, not even as fallback - maybe we should set the percentages to 99% (CTA1) - 1% (CTA2)?

In more detail:

option6X-init (1%) -> OK
option6X-noedit-init (1%) -> OK - note: users tracked with this one do not also get option6X-init

option6X-impression-bottom -> changed to option6X-impression (there is no more overlay form, so -bottom suffix makes no longer sense) - is also at 1%
option6X-submit-bottom -> changed to option6X-submit_attempt (followed by option6X-submit_success or option6X-submit_error_<error>)

option6X-cta_edit-impression-bottom -> changed to option6X-cta_edit-impression
option6X-cta_learn_more-impression-bottom -> This one will not trigger: if a CTA is bucketed at 0%, it can not even be shown as fall-back CTA. Maybe do 99%-1%?

option6X-cta_edit-button_click-bottom -> changed to option6X-cta_edit-button_click
option6X-cta_learn_more-button_click-bottom -> This one will not trigger: if a CTA is bucketed at 0%, it can not even be shown as fall-back CTA. Maybe do 99%-1%?

option6X-cta_edit-edit_attempt-bottom -> changed to option6X-cta_edit-edit_attempt
option6X-cta_edit-edit_success-bottom -> changed to option6X-cta_edit-edit_success

option6X-edit_tab_link-click -> OK
option6X-section_edit_link-click -> OK
option6X-view_source_tab_link-click -> OK

option6X-edit_tab_link-edit_attempt -> OK
option6X-section_edit_link-edit_attempt -> OK

option6X-edit_tab_link-edit_success -> OK
option6X-section_edit_link-edit_success -> OK

This looks good, except that I'm not sure what you mean when you say:

option6X-cta_learn_more-button_click-bottom -> This one will not trigger: if a
CTA is bucketed at 0%, it can not even be shown as fall-back CTA. Maybe do
99%-1%?

and

option6X-cta_learn_more-impression-bottom -> This one will not trigger: if a
CTA is bucketed at 0%, it can not even be shown as fall-back CTA. Maybe do
99%-1%?

I'm under the impression that we'd like the "learn more" CTA to be a fallback only when the reader does not have the rights to edit the page. Does your note state that we'd have to show the "learn more" CTA to some readers who *do* have rights to edit in order to allow it to be a fallback in the cases where readers do not? If so, that seems ... odd.

Well if CTA1 (edit) is at 100%, CTA2 (learn more) will be at 0%.
CTA's bucketed at 0% are never - not even as fallback - displayed & changing that would be quite a lot of work.
I'd suggest enabling CTA1 at 99% (effectively receiving 99% - <protected pages>) & CTA2 at 1% (effectively receiving 1% + <protected pages>)

I recommend that we run CTA1 at 100% and drop CTA2 entirely. On semiprotected pages there will simply be no fallback CTA, which is fine as we want to focus on anonymous edit activity for editable pages only. Let me know if you have any concerns with this.

I also requested that we switch from the CT session to the persistent anon token set by mw.user.id(), which will allow us to get clickthrough/conversion data about recurrent edits from the same client across sessions. Matthias noted that it's trivial to make this change before we push this to production on Thursday.

"I recommend that we run CTA1 at 100% and drop CTA2 entirely."
I would recommend against dropping CTA2. While we may not focus on CTA2 for this metrics study, we'll miss a chance to reach out to those editors that fill out feedback on a protected page in this timeframe (they'll only see "
Thanks! Your post can be viewed on this feedback page.") - there probably will only be very few though. Would this interfere with gathering accurate data?

Likewise, CTA4 will need to be enabled as well (or anon users won't get any CTA) - this won't mess with the percentage of users that are getting CTA1, but you'll be seeing option6X-cta_signup_login-XYZ as well, though I'm assuming this won't be a problem?

"I also requested that we switch from the CT session to the persistent anon
token set by mw.user.id()"
Done & pushed to prototype - could you verify this works as expected?

Final setup, as agreed on IRC:

• CTA1 (Edit) at 100% for anons
• CTA6 (Visit Teahouse) at 100% for registered (as this is not affecting anons)
• CTA2 (Learn More) and CTA4 (Signup) entirely disabled

On semiprotected pages no CTA will be served to anons.

This is live on testwiki, I tested and looked up the log and saw a number of issues, see sample below

testwiki ext.articleFeedbackv5@10-option6X_-cta_edit-button_click 20121011200909 0 x38YiXpxrvF17aqUxE6Btv8mF4gFjcSG 0 0 0 0 0 Golden-crowned_Sparrow|147432
testwiki ext.articleFeedbackv5@10-option6X_-cta_edit-edit_attempt-bottom 20121011200921 0 x38YiXpxrvF17aqUxE6Btv8mF4gFjcSG 0 0 0 0 0 Golden-crowned Sparrow|147432
testwiki ext.articleFeedbackv5@10-option6X_-cta_edit-edit_attempt-bottom 20121011200933 0 x38YiXpxrvF17aqUxE6Btv8mF4gFjcSG 0 0 0 0 0 Golden-crowned Sparrow|147432
testwiki ext.articleFeedbackv5@10-option6X_-cta_edit-edit_success-bottom 20121011200936 0 x38YiXpxrvF17aqUxE6Btv8mF4gFjcSG 0 0 0 0 0 Golden-crowned Sparrow|147436

High priority:

• All impression events (both option6X-impression and option6X-cta_edit-impression-bottom) are missing and they are critical for the metrics we need to capture

Lower priority:

• there is an underscore in the bucket name that should be removed (option6X_)
• the "-bottom" suffix is still there, while according to the comments above it should have gone

Anon token is now logged as requested instead of the clicktracking session and the cookie reset if missing, thanks.

section edit and tab edit events logged as expected, thanks.

Fixed all issues, in production as we speak

Re-opening this ticket, because of issues associated with the data we collected, according to Dario:

On Oct 23, 2012, at 3:06 PM, Dario Taraborelli wrote:

Fabrice,

after reviewing the problem with Aaron, Matthias and S, we decided to unstage the AFT changes and hold off with the enwiki deployment until we've fixed it.

The problem is caused by the use of inconsistent tokens in the CT log (the clicktracking session, i.e. CT's default, for some events, the anonymous token generated by mw.user.id() for other events).

Where we are now:
• Aaron confirmed that the CTA1 data we collected last week is affected by this bug (this applies both to CTA edits and regular edits). He'll report further issues with the data on this thread.
• S and Matthias went ahead and deployed the campaign support in ACUX, which has already been fully tested and QA'd. This means that whenever we're ready to run the CTA4 again, we'll be able to log events straightaway
• I recommend that we review both the code and the data and target Thursday for a deployment of CTA1 at .5 and CTA4 at .5. This should give us enough data for the basic metrics we were aiming to get and allow us to decommission CT by 10/30 as originally planned.

Let me know if you have any concerns with this plan. Thanks everybody (and Matthias in particular) for helping nailing down this problem.

Dario