New questions:
- what are the specific 'triggers' that cause someone to visit the Teahouse? Can we infer from someone's editing patterns that they need help, and provide them an invite in a more timely fashion? Because previous research strongly suggests that Wikipedia loses a lot of potentially valuable new contributors within their first few edits/hours because they get confused and/or have a negative experience.
- who does the Teahouse NOT help? Can we identify new editors who experiencing challenges that the Teahouse, for whatever reason, does not seem to address for them? If we can identify valuable newcomers whose needs are probably not currently being met by the Teahouse (mobile-first users, to pick one example of many), then we can design and test interventions targeted at supporting those users.
- Does the Teahouse mitigate the effects of negative feedback on newcomers? The Teahouse was designed partially in response to work that Dr. Morgan and I did in 2011 while working as fellows for the Wikimedia Foundation. We showed that the negative feedback that newcomers receive is strongly correlated with decreased retention rates -- independent of their initial investment (# of edits in first session) and the quality of their work. It would be very useful to know if the Teahouse is especially effective in helping these users keep contributing.
- Are our technological interventions more effective on editors that are invited to the teahouse? I think that the reason we don't see significant gains in retention and productivity for VE is that the benefits of VE are too short-lived. Even if a newcomer has a better initial experience with editing, they're still likely to get negative feedback and leave. If the Teahouse reduces the social/motivation barriers to participation, then more immediate, technical interventions should have a larger apparent effect. See my sketch of entry barriers, VE and the Teahouse: https://www.mediawiki.org/w/index.php?title=File%3ATeahouse_research_showcase_October_2015.pdf&page=48