@AikoChou or @BWojtowicz-WMF could you please provide a quick status update here sometime on Tuesday? it can be just a few simple bullet points that I can copy over into Asana

@gkyziridis Could you please write an update here about the overall status of this work, that I can use in Asana reporting? I have to submit these every Friday. I can write the sections about how many reviews we have per language right now, but would really appreciate your updates on the investigation and the overall ticket

@Samwalton9-WMF could you please keep me in the loop about timing for switching over to the other model? I don't think you need anything from us, but I just want to make sure we're available in case of issues

Besides Teahouse, another useful resource could be the enwiki Reference Desk: https://en.wikipedia.org/wiki/Wikipedia:Reference_desk

@AikoChou Great question, and great thinking! I'll start a thread for this in the WE1.1 channel so we can propose this and get feedback from Kirsten, Lauren, and Peter.

@KStoller-WMF @DMburugu just chiming in to say that the prioritization of T392012 is blocked until we confirm whether this task is a priority under WE1.1. We started to talk about this prioritization decision here and I'm happy to continue the conversation whenever it makes sense!

I've created Annotool instances for most of the additional languages in which we want to evaluate the model. Project numbers, translations, and labels for each of these languages can be found in this doc.

@Trizek-WMF good question! We still ideally want to have 5+ evaluators per language, like we did last time, but we don't want this to be a blocker to moving forward if we have a strong and consistent signal about the model's performance in that language. I believe that the number of samples per evaluator is the same as last time. cc @ppelberg

This is kind of a unique case for us, since most of our hosting requests come from the Research team, so I'm sorry that we don't have a process built around this yet!

Hey @derenrich, sorry for the delayed follow-up. This is on me personally; I'm balancing a few time-sensitive items right now and a bit behind on the things that are non-blocking (as I understand this request to be - please correct me if I've misunderstood).

@Miriam yes totally, sorry for not doing that sooner! I'll do a quick pass of our board to see if there are other places where y'all are unnecessarily tagged

@Miriam hi!! yes this is! I don't think anticipate that we'll need support from Research, other than pinging @diego with occasional questions in the #structured-task-improve-tone channel. Is that okay?

We met on Friday to discuss Ilias' questions above and talk through some potential solutions. See meeting notes and meeting recording for more.

Hi y'all! Just wanted to send a BIG thank you to everyone who participated in evaluating the Tone Check model! We really appreciate your help.

Hey @kostajh ! Thanks so much for submitting this! Totally agree with you about how the work described here would magnify the impact of the RR Filters Rollout, but I'm curious how this connects to FY2025-2026 OKRs. Can you please share more about whether this rollout impacts any KRs you're working towards - whether as a hard blocker / dependency or as a soft blocker / nice to have? I'm asking because in order to prioritize this ticket we'd like to figure out: (a) Cost of delay or non-prioritization, and (b) How we'd report on this work

@DMburugu thank you so much for this helpful response!! It totally makes sense that you're figuring out the details of what this experience will look like, and how you'll be using the multilingual RR data, and we'd love to reconvene in Q1 as you get more clarity. For now, I'll list this request in our Intake Tracker with a status of "Info needed" and I'll list it in our roadmap as a potential item in late Q1/early Q2.

Thanks so much for getting back to me, and no worries at all about timing @Kgraessle! I have a few more questions - please bear with me! These are questions that I'm discussing with all teams who are submitting requests to ML for Q1+, in order to help us prioritize and plan our roadmap. These are also questions that we haven't always asked in the past; as shared in our Engagement Model doc, we are moving towards an operating model in which we partner closely with teams toward a shared definition of success, rather than fulfilling requests without actually taking on the mission/purpose of the work.

Hi @Kgraessle ! Can you please clarify the following?

In T393474#10834359, @Michael wrote:

Could we improve the existing pipeline first, before we add new features? While it is surely good to keep that use-case in mind as we design the new system, I'm concerned that we might be doing too much at once if also want to introduce new functionality while migrating to a new pipeline at the same time.

Hi @DMburugu! Responsibility for this request has been transferred from the Research team to the ML team, and we will indeed have a hypothesis for Add-a-Link improvements (including model quality improvements and retraining pipelines) in Q1. This quarter we are planning the work by: auditing tickets like this one, conducting diagramming sessions like the one we had today, and working with you all to establish a set of product requirements for the next iteration of Add-a-Link. By the end of this quarter, we aim to have an implementation plan that has been reviewed and agreed upon by Growth, Research, and DPE - so that we can begin implementation in early Q1. I'm not sure whether this ticket will be directly incorporated into that plan, or if my engineering partners will propose a different solution to the problems and requirements we've discussed, but we'll definitely keep you looped in since your feedback is crucial in this process.

@Kgraessle I've logged the second part of your request (re: the multilingual RR model) as a request to the ML team on our intake request tracker. Based on our recent conversations, my understanding is that the Moderator Tools team is moving forward with just the language-agnostic RR model for now. As such, the ML team is deprioritizing this request for support with the multilingual model. Please feel free to re-submit this request using this template if you need the datasets for the multilingual model in the future.

Excited for this! Can we add a requirement about the demo implementing a probability score threshold (eg. only surfacing the peacock check when the probability score is 0.8 or higher)? This will allow us to demo something that is closer to what we expect the production experience to look like. We could also test different thresholds during these interviews if it makes sense to do so.