User Details
- User Since
- Feb 19 2021, 3:23 PM (294 w, 12 h)
- Availability
- Available
- LDAP User
- Unknown
- MediaWiki User
- MRaish (WMF) [ Global Accounts ]
Yesterday
- This week we completed our tenth of 10 interviews with a mix of undergraduate and graduate student participants from around the world.
- Sandra has started a report/findings deck, and I've been helping revise and shape it.
- We hope to share an initial version with the team next week. It currently intends to cover 4 areas:
- a review of 7 relevant studies looking at AI use by students, published from 2023-present
- participant use of AI chatbots in their academic work
- participant use of Wikipedia in their academic work
- findings specifically about the usability and potential for the MCP tool (which generally received a quite positive reception from participants)
- relatedly, I continue to provide informal/mentorship support to the NYU student group that reached out to FA earlier, and who offered to complete a "consulting" project for the team that I have nudged in the direction of peer interviews around the areas of peer use of AI and Wikipedia. This has taken the form of weekly 1-hour meetings with their members in which I deliver feedback and instruction on materials and methods
- This week I learned that part of our V1 research is being used by project stakeholders to shape their planning around edge cases
- I provided lightweight feedback to an approach to attribution signals that emerged from a recent community Discord call
- We've received the final versions of the designs for testing from Sarah
- Jahanvi and I are planning/coordinating a research plan on this basis, the current version of which can be seen here
- We've started the recruitment conversations with Bethany, and after additional consultation with the team we have pivoted toward recruiting app users from our internal participant database
- We are drafting a screening survey, drawing inspiration from our previous Collections project completed last year (i.e., with a focus on streaks and goals rather than collections)
- once the screening survey has been finalized, we will reconvene with the stakeholder PM (Haley) to maintain alignment with team goals
- recruitment may begin as early as next week
Fri, Oct 2
We met with the requesting team (PM and designer) to further gather information about their needs. The goal of this research at this point is to use the opportunity of CR sessions to systematically show app users examples of features that may be added to the current activity tab in order to support the team's roadmap for the upcoming quarters. We are pursuing the possibility of specifically recruiting app users who already engage with the activity tab in some way. This may or may not be possible on our timeline (designs are expected to be ready next week). If it ends up not being feasible, these interviews should "work" with a broader selection of app users. In the future, and especially if the CR sessions for the iOS team become a permanent structure, we will need to recruit via avenues that don't require engineering support. Database app users are plentiful, however it's not guaranteed that they will be familiar with the activity tab, which would be a major bonus for this round of research.
Interviews are underway, with 3 completed so far. We plan to have all interviews completed by next week, with analysis and reporting to begin shortly afterward.
This week Jahnavi and I finalized the internal literature review, documenting the theme of "donors as a user group," on the basis of documentation and research emerging from both the Product and Advancement departments in recent years. I've shared this document with Jazmin, and will share here after receiving an indication of whether changes are required. Unless we hear otherwise, I believe that our work in support of donor interventions is currently wrapped up. Future interventions may result in additional requests for user research support, though. I'll update here with a link to the document after it's been reviewed, and in the meantime I'll mark this task as completed.
Thanks Debra for the notes above. After meeting with Olga this week and attending the team sync, it sounds like the requesting stakeholders are going to reach out soon with a specific interest area for qualitative research. The timeline is too tight to do anything that will directly support the October A/B test. If we haven't started work by the time the A/B test results are in, then we may find ourselves using qualitative research to further contextualize the results (as we've done on a few occasions in the past).
The only update here this week is that the newcomer homepage A/B test is launching in 2 weeks. We don't have time to inform the designs used in this test via user research, however the results of the experiment may affect the direction that project stakeholders feel is most urgent for this research.
Fri, Sep 25
This project was ultimately separated from the donor consent work that I've been reporting in a separate task. The Reader Experiences team remains in need of support from the UX Research team, and as the donor work has started to slow down, these conversations have picked back up. Over the last two weeks, I've had a few meetings and Slack communications with the Reader Experiences designer, who has also been coordinating with their PM to organize interest areas. Their research interests are currently captured in this document.
This week I met with a colleague from the Applied Science side of the Research team to discuss this request. Before suggesting a wording change, I plan to ask the Apps team about the following:
Semantic search is being rolled out on Web much sooner than anticipated. This strategy document provides more details, but the tl;dr is that Semantic Search will be released on Web much sooner than anticipated. Our Product colleagues coordinating this initiative have requested additional user or qualitative research from us in support of this major push.
This week we continued work on the internal literature review, delivery of which was pushed back to next week. This deliverable is intended to support Product+Advancement collaboration in the long term, and isn't needed urgently. When this is delivered, we'll consider this donor-focused user research effort to be concluded, at least until our Product colleagues tap us for additional support in upcoming phases of the Product+Advancement collaboration.
This week we were on a bit of a pause. We reviewed our first pilot interview session, which uncovered a few unexpected things. Specifically, the first student participant (sourced from Userlytics) reported leaning more on Wikipedia than AI chatbots for their schoolwork, which was unexpected. They reacted well to the MCP tool and appreciated being linked to so many Wikipedia articles at once in the tool UI. However, it raises some recruitment questions—we have seen evidence from a few different sources that heavy Wikipedia use and heavy AI chatbot use have an inversely proportional relationship to each other (at least, trends like this have emerged in a few separate studies on different topics). Since we're interested in knowing both (a) how students use Wikipedia to support schoolwork, and (b) how students use AI chatbots (in their own lived experience) to support schoolwork, we may want to think about forming separate groups in recruitment. UX Research will continue discussing this and make necessary adjustments. We expect to begin recruiting in earnest next week, wrapping up in time for our new mid-October deadline.
This week I continued discussions with the team. They were originally interested in using these interviews to show Android users designs in support of upcoming roadmap decisions. The designs are in development and expected to be ready before the team designer goes OOO for an extended period on October 8.
Thu, Sep 17
This week we continued communications with the requesting team and agreed to move forward with a version of the Continuous Research model. They would like a commitment from us of ~5 research sessions per quarter. I've asked them for feedback on a draft plan for these sessions that I put together. We can likely start scheduling sessions next week and conducting them the week after depending on how things go. Assuming we continue to move forward, we'll be refining their request next week asynchronously and aiming to recruit their users for initial sessions the week after that.
This week we:
- continued communications with the external student group and are awaiting their response. I'm considering this to be an "extra" for this study, and am considering our internal WMF study to meet our obligations to the Future Audience team and the MCP.
- FA shared that their timeline for this work has been pushed back to mid-October. We will adjust our timeline to match, aiming to deliver the first or second week of October.
- We have finalized the screening surveys and have begun to distribute them.
- Our first pilot interview is scheduled for tomorrow, with next week being used to triangulate interests with the FA team. These interviews are slightly unusual, as we are using them both to test a mostly-built tool and to explore an unfamiliar product context. I anticipate that their feedback on our first interviews will lead to significant changes.
This week we:
- continued work on the internal literature review, expected to wrap up next week;
- managed communications between various audiences in Product and Advancement, specifically around the wording of two bullet points to be included in the donor consent pop-up. The Reader Experiences designer revisited the materials and updated the pop-up copy.
- We are currently running an unmoderated study in which self-reported donors respond to this updated copy, at the request of the Product statkeholders who are in need of more insight about how different versions of the wording of this bullet point may be understood by and appeal to potential future donor users. This study is using the same format as before, and will result in a final participant pool size for this work of around 15 people, with ~5 seeing each version of the pop-up.
- I'm currently leaning away from running a quantitative comprehension study, although will remain attentive to emerging Product desires and needs.
Fri, Sep 11
This week we:
- formalized work on the 'literature review' portion of this request. We're working in the direction of something like an "annotated bibliography of donor research from the Product perspective". This should take a week or two.
- further discussed the results of the usability exploration we reported above. There is some interest in continuing this work, specifically in the direction of the donor-facing wording that the donor consent pop-up will contain. We are interested in testing the current "stop seeing generic fundraising appeals" bullet point against a positively-framed alternative, such as "see fundraising appeals that matter to you." This comparison may take the form of a larger-scale survey measuring message clarity, a smaller scale qualitative exploration with donors recruited on userlytics, or both. We're awaiting more input on timeline (i.e., if results are needed quickly, we'll stick to Userlytics, but if there's enough time we'd prefer to support the qualitative findings with empirical/quantitative data)
- determined that the Reader Experiences team, specifically, isn't looking for more donation support in the current cycle given the findings reported above.
- determined that, after the internal literature review is completed, the main remaining question area this work was intended to address will be found in the wording comparison described here.
We are about to start actively recruiting for interviews (Wikipedia-and-chatbot-using students) and expect to schedule our first pilot rounds next week.
Sep 4 2026
Thank you @DKumar-WMF for sharing the studies above. Sandra has started processing these and a few others in this tab of the project document.
This week we completed a user testing exploration in which we (a) recruited self-reported donors on Userlytics, (b) shown them the designs for the upcoming donor consent experiment in a controlled, unmoderated setting, and (c) interpreted and reported their responses in a report deck. Work on an internal literature review is ongoing, and next week I intend to hold conversations with relevant Product stakeholders to finalize the remaining phases of this research initiative.
Sep 3 2026
Aug 28 2026
This week we continued gathering background research about the donor space, in an effort both to comprehensively understand the state of internal research and to better target upcoming efforts to recruit donors for participation in user research. We continue to run limited explorations on Userlytics, where the participant panel indeed does appear to include recent donors to the WMF, and we expect this work to acquire more focus in the coming period as more internal research becomes available to us. In the short term, we anticipate launching a small-scale unmoderated test with donors who "live" in the Userlytics panel to explore their perceptions around issues like Wikipedia accounts, giving consent (to the WMF or in general) for accepting cookies, and the potential value of accounts for these Userlytics-sourced self-identified donors.
This week we met with the Future Audiences stakeholders to gather information and needs around this project. We used these to compose a Research Brief, in which an interview-based study is proposed to be completed in the month of September.
Aug 26 2026
Aug 24 2026
The hypothesis under which this work falls has been marked as Completed. This update restates the update summary as shared on Asana:
Aug 14 2026
Hypothesis update as shared on Asana by the hypothesis owner:
Jul 31 2026
Hypothesis update as shared on Asana:
Jul 20 2026
Update (summary of UX research activities for this hypothesis)
This work has been concluded and is reported in the report deck available on Wikimedia Commons. It is also referenced in the associated [[ https://meta.wikimedia.org/wiki/Research:Wikimedia_Attribution_Framework:_Ecosystem_Growth_research_support | Meta page ]]for this project.
Apr 30 2026
We have reported our qualitative findings in the form of a WMF-internal slide deck. Our qualitative research findings with synthesized in conjunction with contributions from @TLessa-WMF of the WMF Design team in the in-progress internal written report, to be shared more widely in the coming days or weeks.
Apr 24 2026
This project wrapped up at the end of January, 2026, and is reported in this slide deck.
Apr 3 2026
Project update as shared on Asana:
Mar 20 2026
status update as shared on Asana:
Mar 12 2026
Weekly update as shared on Asana:
Mar 6 2026
Weekly update as shared on Asana:
Feb 27 2026
Weekly update as shared on Asana:
We have produced a first draft of the research report. The method used to arrive here entailed:
Feb 4 2026
Jan 26 2026
Noting the following recommendations for copy (to consider implementing after review of the MVP's performance)
Jan 20 2026
Interviews have begun, with the first pilot being concluded last week. Interviews are ongoing this week, with analysis and reporting anticipated to conclude by January 30.
Jan 15 2026
This initial research was concluded with the high-level takeaway that the tested signals benefit from reader engagement with the text (especially the sources and information contained in the text), and that this engagement is more natural or more likely to occur in the LLM chatbot context than in the SERP context. An initial report has been published on Meta, with the possibility of further amendments depending on future feedback.
Jan 12 2026
We supported the initial round of Semantic Search development by asking ~50 external Wikipedia readers to compare sets of search results from Semantic vs Legacy search. In most cases, they agreed with the researchers' subjective evaluation of the results sets; that is, semantic search appears to return more relevant results in general for the tested real-world Wikipedia queries. (Link to the project page on Meta)
Dec 12 2025
Over the last few weeks we were able to design and field an experimental survey intended to address the relationship between attribution signals and "trust" (i.e., perceptions of accuracy of overall information presented and Wikipedia information, specifically). The research report remains under construction and subject to internal review. The following summary has been provided to the Design team:
Sep 5 2025
Presented the findings deck and final written report to Reader Experiences yesterday, concluding the project. The hypothesis motivating this project is being tracked separately on Asana.
Aug 29 2025
Final written report has been drafted; findings slide deck has been mostly drafted; findings presentation to the Reader Experiences team scheduled for next week. Looks like we'll be wrapping up right around when we were aiming for (end of August), leaving some additional time in Q1 for design/product explorations based on the findings
Aug 22 2025
We are quite close to the completion of a first draft of the written research report which summarizes findings from the interviews, take-home task, and close-out survey. Next week we will begin work on the presentation slide deck, and an initial readout for the requesting product team has been scheduled for September 4.
Aug 15 2025
Data collection has fully concluded and we are firmly in the report-writing phase. The composition of the research team may shift slightly as one member is moved to a different project, but we remain on track to have a report by the end of the month.
We continue work on Phase 1/Stage 1 in which we are attempting to elaborate a use-case framework from data collected via open-ended responses provided by readers. We are also in the planning stage of Phase 2, anticipated to take the form of a diary study of Wikipedia readers. We remain on track.
Jul 25 2025
At this point we have more-or-less concluded interviews, although 1-2 may yet occur next week. We have also collected take-home-task responses from interviewed participants, and are preparing to recruit ~20 additional participants to a 'take-home only' activity. We appear to still be on track to wrap up analysis by mid-August.
The most recent actions on this project are detailed at T399736—we have fielded an updated version of the Stage 1 QuickSurvey and will continue analyzing results for the next two weeks, in collaboration with stakeholders and the research team. We have also begun drafting the survey instruments to be used in Phase 2, the anticipated Diary Study. This will be hosted on Qualtrics and delivered to (a) Wikipedia engaged readers (sourced internally), and (b) Wikipedia infrequent or non-readers (sourced externally).
Jul 10 2025
Work continues. We will deploy our first QuickSurvey on Monday. Work on this project is now being tracked as WE3.6.4 on Asana.
We've conducted two pilot interviews and have begun recruiting for the next round of interviews and take-home tasks. We aim to conduct interviews through the end of July/beginning of August. Participants will also be invited to participate in an optional take-home activity following each interview. The target for number of participants has changed—we have shifted emphasis to prioritize our interviews, and invite a proportion (possibly all) of interview participants to the takehome task. We will recruit on a rolling basis until the end of July. Although we were originally aiming for 20, the eventual number will likely be lower than that.
Jul 2 2025
Work on this project continues, with initial steps being taken in the direction of initial data collection. Our early work has consisted of gathering diverse perspectives and planning for research activities that can effectively accommodate those perspectives. Details of our steps toward data collection can be seen in T398370, where our initial Quicksurvey effort is described. As detailed in the research brief for this project, we plan to progress through a series of survey-based research stages before shifting our efforts to support a diary study and interviews with participants. In our next update, we will provide a draft timeline of anticipated steps.
We continue to work with the requesting team to explore this topic. This week, we completed one (and one more has been scheduled) provisional, pilot interview in which we explored the theme of collection and curation of information with an engaged Wikipedia reader. When we reconvene, we will reflect on this experience and formalize our plan for continued data collection in support of this hypothesis.
Jun 25 2025
update: We have had several meetings with the requesting team and are quickly moving into the preliminary research stage. Interested parties are encouraged to take a look at the research brief in which the plan and research questions are stated.
May 2 2025
A more-or-less complete draft of the final report has been delivered to the Web team, and a larger readout has been scheduled for Monday: WMF-internal link to Simple Summaries report
Two interviews completed this week (including two no-shows on Friday); three currently scheduled for Monday/Tuesday of next week. If the 3 scheduled are actively completed, I hope to provide the Editing team with a lightweight report by EOD on Wednesday of next week. One very early observation is that both of the first two participants, when talking about neutrality, associated "emotion" with non-neutrality. That is, statements were described as non-neutral if they clearly expressed what the participants perceived as "emotion."
Apr 28 2025
after consulting with the Editing team, we've decided to move forward with a short-term and lightweight series of user interviews around this feature, anticipated to be completed within the next 2 weeks. Recruitment and early interviews are expected to begin this week.
Mar 24 2025
After ongoing consultation with Research team colleagues, this project has moved into the pilot testing phase. @JScherer-WMF has helpfully provided static mockups to be used as visual artifacts, and they are being shown to participants in a survey hosted on Qualtrics. Currently, participants entering the survey have an equal chance of being routed into the "good summary," "bad summary," and "control (no summary" conditions, after which they have an equal chance of seeing article pairs easyA+hardA, hardA+easyA, easyB+hardB, or hardB+easyB. There is a pre-test measuring trust in and knowledge of Wikipedia, AI usage, and selected demographic variables, and a post-test measuring opinions about AI+Wikipedia, followed by a repetition of the Wikipedia trust questions shown in the pre-test.
Feb 14 2025
We have finalized the documentation of V2 Archetypes, which will be updated on Commons pending a minor adjustment.
In consultation with the Web team and Research team colleagues, we are approaching alignment on a structure for this research, as outlined in the research brief. Briefly, we are planning to use panel participants (sourced externally) to investigate what happens when people interact with good or bad summaries in more difficult or easier articles. Next steps will include aligning with the team on experimental materials, further consultation with Research colleagues, and setting up the instruments on Qualtrics as we move toward the pilot stage.
Jan 31 2025
@HCoplin-WMF and @JTweed-WMF completed their interviews with notifications informants, and the Platform team working on implementation, using the insights gathered during the listening tour as guidance. Design Research participated in this project by helping to shape the structure of the structured interviews and by synthesizing the output of the interviews in the synthesis document. Thank you to all participants and supporters of this project for your collaboration.
Jan 17 2025
The interviews were completed at the end of December, and analysis wrapped up this week, resulting in this findings deck: https://docs.google.com/presentation/d/1N9yLxLEzmrdUAa0CFRJeba5-9CfNh2ejAOEodkQfJAs/edit#slide=id.g25ba630002a_0_0 (WMF login required). It is anticipated that this work will be presented to the Growth team at some point in the context of a team meeting, but for now we can consider the research work more-or-less concluded.
Jan 6 2025
FYI, the podcast episode was recently published:
Dec 20 2024
Dec 6 2024
updated task due date to allow time for interviews and analysis. We are still aiming for ~5 interviews with non-editing readers (English speaking). We are exploreing the possibility of using this opportunity to explore the moderated testing capabilities of the Userlytics platform (e.g., for 1-2 of the interviews), which we haven't yet tried. This effort is also expected to help us continue to gather information about how to improve our mobile testing capacity.
After connecting with @HCoplin-WMF , I'm plan to spend some of the next 1.5 weeks synthesizing findings from the ~7 already-completed interviews into a draft findings document that identifies categories and examples of notification-relevant pain points experienced by the consulted teams and individuals. We'll reconvene when everyone is back from the upcoming offsite in Barcelona
Dec 2 2024
Nov 26 2024
Nov 22 2024
We received input from the Design team during the RDS+Design offsite and are in the process of applying it to the current draft cards. We're also making minor adjustments in a few places. We anticipate using December to invite asynchronous comment from stakeholders on WMF opportunity areas for these Archetypes, followed by a mid-to-late December working session for synchronous input. All-in-all, we appear to remain on schedule.
Nov 8 2024
Draft Archetypes have been prepared, and we are anticipating further workshopping them at the upcoming RDS+Design offsite.
Nov 1 2024
We have identified 5 Archetypes among our V2 respondents, as well s 4 WMF-internal Archetypes representing Wikimedia contributors. We are currently drafting the first set of deliverables related to these groups (the Archetype cards) and will be presenting them in draft form to colleagues at the upcoming Research+Design offsite for feedback on their structure and contents. Overall, we appear to still be on track to finish by the end of Q2, and we will likely have significant parts of the expected deliverables ready before the end of Q2.
Oct 21 2024
@leila that's a good question! I wasn't given an indication of that, but I can share it here when it's shared with me. I expect there will be at least some video editing involved.
The podcast was successfully recorded on Wednesday, October 16, and the list of questions as communicated by the affiliate organizers was adhered to exactly as communicated. Special thanks to everyone who helped me prepare for this opportunity!
Oct 18 2024
Data analysis is ongoing. We've been making groups of people from our external respondents and comparing them to the responses of internal folks who responded to a recent survey of contributors. We are currently on track for our internal project timeline.
Oct 11 2024
Data analysis is ongoing. Short and potentially interesting updates include:
- French-speaking respondents scored higher on a series of T/F questions intended to measure knowledge of Wikipedia and its functions than ES or EN respondents.
- The generalist vs specialist distinction (on the basis of respondents' "most important to you" contribution form) remains meaningful in V2
- The Arabic survey is on-track
With @HCoplin-WMF we have decided to split this work into two streams, with the first being represented by the sub-task T377041 which describes the addition of an API module to the annual Developer Satisfaction Survey administered by the WMF Research team.
