User Details
- User Since
- Jul 11 2023, 2:50 PM (92 w, 5 d)
- Availability
- Available
- LDAP User
- Unknown
- MediaWiki User
- JScherer-WMF [ Global Accounts ]
Wed, Apr 16
Challenges
If we directly adapt our current component, several issues arise:
Fri, Apr 4
Spoke to @jwang. We will start analysis towards the end of this sprint.
Wed, Apr 2
ticket for analysis created.
cc @ovasileva
Tue, Apr 1
I've taken this to a natural stopping point. Here's a video of the demo.
Given the search query: "${query}", provide a strategy for navigating Wikimedia projects to satisfy the goals implied in the query. Include categories, list pages, and other Wikimedia project links that will help the user reach their goals.
Current prompting:
I have early access to GitHub Spark for this, so I'm going to use that as a prototyping tool.
Fri, Mar 28
@jwang is the above enough for the data platform folks to work with?
For the designs here, I think we should keep it simple to A/B test these things.
Wed, Mar 26
I drafted a high-level interim report. More interviews, detailed analysis, and reports to come.
I have a few options for folks to consider.
I'm finished the affinity mapping. I just need to write up the report.
Mar 20 2025
Mar 19 2025
Mar 18 2025
@jwang @ovasileva Do we have an analysis ticket for this yet?
Mar 13 2025
It seems like we've decided on a approach here, so I'm going to move this for sign off and create child tickets.
Mar 12 2025
Looks good!
I'm thinking about MinT and how they were able to create a machine-in-the-loop process for translation. A human editor manually commits and has direct control over every word that ends up on wiki. We could pre-generate summaries, admins could choose which articles become summary candidates, and then we could rely on editors to manually commit the summaries. After a certain number of changes in the source text, we could remove a summary and add it back to the queue.
Mar 11 2025
All good questions! Thanks, Jon.
Mar 10 2025
Here is a WIP list of potential affordances we could consider building:
- Admins choose which articles get summaries and which don't via a community configuration
- A quick way to manually remove summaries from specific articles
- Editors can manually edit summary content like normal article content
- Automatically removing the summary from an article after 5 readers have marked it "unhelpful"
- Automatically removing the summary from an article after more than 20% of readers who have seen it mark it "unhelpful"
- Readers generate a new summary "on demand" every time they open a summary. Editors have no ability to view, moderate, or edit individual summaries
- A community-maintained simple summaries template that editors can add to or remove from any article
- Admins can adjust the prompt engineering for summaries on an entire wiki
- WMF generates the summaries for an entire wiki up-front and automatically publishes them in an "unverified" state
- A reader preference for hiding and showing summaries
- An LLM-powered dashboard that shows the quality of summaries on a wiki, as well as other pertinent metrics
- Automatically re-generate the summary of an article when the article has had XX bytes changed and/or XX number of diffs/edits
- A sandbox to test out the impact of prompts on simple summaries
- Tools to discover the articles on any wiki that would benefit the most from simple summaries
- A list of all summaries currently live on any given wiki
Jan and I discussed this this afternoon. To my mind, the main thing we're looking to learn from this exercise is an answer to the following questions:
@JScherer-WMF @Jdrewniak to pair on this today/tomorrow.
Mar 6 2025
Thanks for the discussion here @matmarex and @MikhailRyazanov.