User Details
- User Since
- Apr 28 2017, 8:33 PM (270 w, 1 d)
- Availability
- Available
- LDAP User
- Unknown
- MediaWiki User
- MEpps (WMF) [ Global Accounts ]
Tue, Jun 28
Most recent final timestamp: 20220426131656.
Fri, Jun 10
@ERayfield I really like the logic flow! Your notes above are very helpful. But I don't quite see the pros and cons of sending files. What does
May 31 2022
@ERayfield What were your findings here?
As noted in the task above, I have that it is still a huge failure rate with only .05% of images being sent to photoDNA successfully.
May 24 2022
I grepped the logs today and got 574 successes out of 971476 files sent.
Most recent timestamp: 20210430135351
May 17 2022
I was thinking the first approach. Creating the statistic on an ongoing basis is out of scope for this task, as we need to get these numbers sooner to know how to proceed.
May 16 2022
Or this message: Creating moderation request for file
We could maybe use Ellen's output
@jsn.sherman suggests using number of files rather than existing error messages.
May 12 2022
Oh! I got exactly one "No hash match found" message now. Also, David Rochford says that there have been a total of 25 emails sent with matches.
May 9 2022
It doesn't look like they have, but I was searching for "Hash match found for file" and "No hash match found for file". I wish we had Ellen's messages in now, because it would have been helpful.
I can confirm that I saw no success messages in the logs today.
There appears to be a file type check in MediaModerationService::processUploadedMedia but only if checkOnUpload (MediaModerationCheckOnUpload) is true and it seems to not be true for us currently.
May 2 2022
Apr 29 2022
Apr 28 2022
From Cindy on sending files versus urls: "I recall that change being made. I will check my email archive to see if I can reconstruct the reason. I have a feeling it was because they have a relatively small size limit if you send the file itself."
Apr 27 2022
Apr 26 2022
We can also increase the logging level per https://phabricator.wikimedia.org/T303312.
Question from engineering talk: should we stop sending urls and send the photo instead? Why wasn't this done in the past? https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MediaModeration/+/576338 seems to have done this before. The next step might be asking Cindy Cicalese.
Apr 19 2022
@ERayfield will be updating this today with how she had approached the code.
I approve this request.
Mar 28 2022
@ERayfield aside from color ;), it looks like just what was asked for to me! I'd go ahead and push it up for @aminalhazwani and the team to review. We could also set up a patch demo for it. @Scardenasmolinar knows how to do this.
With some local testing I did find that if there's an error in the creation of the thumbnail it's not always caught. I'll need more info to know if that's what's happening here.
Mar 23 2022
I got a response that needs more investigation:
Mar 21 2022
Mar 16 2022
I got some replies:
Mar 14 2022
I followed up today because I haven't heard any kind of response.
Mar 10 2022
I have sent an email to PhotoDNA support.
Mar 9 2022
I've reached out to Nicholas to confirm the best way to reach PhotoDNA.
Mar 8 2022
For my own notes: I'll be asking about the error we're seeing, as well as developer credentials for a sandbox account, if available.
I've reached out to @drochford in slack to make sure I contact the correct person.
@jsn.sherman Add null option to be able to run local tests.
@jsn.sherman summarized:
@ERayfield suggests that we have a starting message then dots for each batch run then an ending message.
Mar 7 2022
A few more pieces of info I gathered today.
The error code is 3206.
Mar 3 2022
@drochford Yup, It looks like a few hundred thousands have already run.
Hmm, I ran the script starting at 20220104105243 with the batch-count halved and now got a message saying "Script processed all files. Nothing left!".
Feb 15 2022
Hmm, even more curious @Urbanecm...and that you were able to see it on the debug server.
The current hypothesis is that the "+fawiki" is not currently merging with the empty default array. @jsn.sherman and @eigyan are looking into why that would cause an issue. We may need to switch it for "fawiki" but it's also worth understanding why this is happening.
I'm actually on a call with @eigyan and other members of the team as we try to reproduce this right now.
Thank you so much for continuing to look into this @Urbanecm! I just went to a random page on fawiki and added a parameter for an example survey, but we've concluded the example surveys are not deployed (wisely). The survey was reverted so I'm assuming you mean on beta?
Thank you so much for this update @Urbanecm, and for the revert. We'll look into this on our end.
Feb 13 2022
@drochford Yes, how many do we want?
Feb 11 2022
The privacy policy can be configured in LocalSettings.php. Currently in QuickSurvey.vue, the footer is outside the "completed" conditional. If we wanted it to disappear after survey completion, we could just put inside the conditional.
This is tracked better in https://phabricator.wikimedia.org/T258603.
@Madalina This is ready for your review. Let me know if you have questions!
Feb 9 2022
Thanks for the links @jsn.sherman!
@jsn.sherman
Small feedback below:
What were the underlying pitfalls that led to this failure?
The documentation for new extension developers did not link me to the information I needed to know.
Could you update these docs?
Feb 8 2022
from @jsn.sherman and @essex
- Library in question is wvui
- Proposal to make a new component and try including it.
Feb 7 2022
@jsn.sherman Do my changes above address your feedback?
Feb 3 2022
Feb 1 2022
Jan 28 2022
Jan 27 2022
It would required a deeper code re-write but we could also make the survey show up on mediasearch as in the screenshot that's part of the ticket above. I reached out to Luca to ask if we know what pages editors visit most often. Locally, this is how that appears now but we could probably improve it.
So I've reviewed the ticket above. They originally wanted to put a survey on the page where people used media search but decided against it, and eventually used a static main namespace wiki page with info about media search.
I paused the script just after 20220104105243 because there are millions of jobs on grafana and I wanted to let it work through those jobs first. When I restart it, I will half the batch count.
Luca also pointed me to https://phabricator.wikimedia.org/T258419, where they used QuickSurveys before on commons!
Luca from Structured Data suggested placing the survey under the options on the right side. We would also need to think about mobile view.
Jan 26 2022
I added the Structured Data tag because we might want some review from them. I also pinged @Seddon on slack to see the best way to reach out to the team because I couldn't find a specific channel.
I'm looking at the file pages on Commons and I'm not sure where the survey could fit. Same with the main page.
Survey on my local mainpage:
{F34932007}Well I got a survey to appear on a file page, but there would definitely be styling needed.
Jan 25 2022
I'm currently looking at https://commons.wikimedia.org/w/index.php?title=Special:Version to see which extensions Commons uses.
Jan 24 2022
Apparently. you can also target pages outside the main namespace through the audience pageIds array.
I'd guess to do this we'd want to add a config option to display on file pages because on other wikis we probably wouldn't want to display them outside the main namespace.
I found where we filter for main namespace in includes/SurveyContextFilter.php:L35.
I'm re-running the script and still seeing these errors. I don't have a lot more information about why, or how many images are getting correctly scanned. I'd like the script to give more feedback in the long run.
I just started the job at 20210430135017.
Jan 18 2022
We are discussing this task in the engineering meeting. Some outstanding questions:
Hmm so I was able to confirm that the api key is current.
Jan 12 2022
@TAndic @eigyan I can actually confirm that I made an editor account on https://ca.wikipedia.beta.wmflabs.org/, made 5 edits, and then saw the survey on https://ca.wikipedia.beta.wmflabs.org/wiki/Applebutter.{F34915694}