We would like to study the ways in which the collapsible sidebar is affecting user behavior. This task is created to determine the instrumentation required for us to do so.
- Answer the following questions
- What is the frequency that users collapse and uncollapse the sidebar
- What is the difference in interaction with sidebar links between people with the sidebar collapsed (the majority of the time) versus uncollapsed (the majority of the time). i.e. Once the sidebar is collapsed, do users open it when they need to, or do they just ignore it altogether?
- What is the overall difference in interaction with sidebar links before and after the change
- In particular, what is the overall difference in interaction with the donate link?
- state of sidebar
- session id
- user status (logged-in, logged-out)
- edit count bucket
- clicks to donation link
- skin version (From task T250282 : Instrumentation should be run on both the new version and old version of the skin on a given wiki)
Additional Context around the analysis
What is the goal of this analysis? why do yo want to measure the effects of the collapsible sidebar? This would be helpful to understand this in order to determine if we need to consider more or less in the effects do we need to measure'.
- Usage for sidebar is low and each link is not instrumented. Web team is not looking for a Success metric, the simply want to understand the effects and want to see how ppl are using the sidebar.
except Donate link: for fundraising
What is the frequency per session that users collapse and uncollapse the sidebar (logged-in, logged-out, edit count bucket, wiki)
What is the default state of the sidebar?
- default uncollapsed until language switcher deployment. after - default collapsed for anon, uncollapsed for logged-in. Logged-in - preserve your choice across sessions
Do you mean, how many people collapse an uncollapsed sidebar? how many people discover that the sidebar is collapsible?
Will need instrumentation to track the 'collapsing' action;
- Interaction with the button per session.
Will need new Schema that can be used for this and other features that will be rolled out as a part of DIP
Fyi, Privacy issue to instrument each link in sidebar. will need legal consultation if we ever think about it. Not required now.
What is the difference in interaction with sidebar links between people with the sidebar collapsed (the majority of the time) versus uncollapsed (the majority of the time). i.e. Once the sidebar is collapsed, do users open it when they need to, or do they just ignore it altogether? (add session id, timestamp to new schema)
Define 'majority of time' i.e. how much time?
- given an average session length, do ppl have it open or closed for 50% of the time of their avg session length.
Do you mean 'Of all the people that collapse the sidebar, how many people reopen it i.e. uncollapse it again?
- Open or closed for time per session; recording the state and having it per user
- which of these , open or close is more popular ?
- how does that behavior present itself
**What is the overall difference in interaction with sidebar links before and after the change
- (rerun Megan's report : sidebar and lang links)
Define interaction: is it collapse-uncollapse; or click on a sidebar menu
- traffic to each sidebar link using webrequest data
Will need to instrument the lang links when it is deployed, later
In particular, what is the overall difference in interaction with the donate link?
Will need to instrument 'Donate' link.
Fundraising knows current interaction details. we will check the numbers 'after', once instrumented, in our analysis
Timeline for Analysis:
Start Analysis around Aug 17th and complete in early September