Reviewed dashboard with the structured data team, add Median number of structured data elements per file to dashboard.

@MPhamWMF there was a copy and paste error, I updated the go bar searches count for zh_yuewiki.

Updated the notes and presented in the team sharing meeting.

Please find the analysis for required metrics related to go-bar searches below. The numbers are from my pull of 1 week of data from 04/03/2022 to 04/10/2022.

In T252443#7873533, @matthiasmullie wrote:

Question: the "SDC edits" tab mentions "only look[ing] at non-bot edits to files".
MediaInfo submits all edits with a bot: 1 flag via the API (code for statements, code for captions) (the reason is simply "because that's how Wikidata does it")
Does that mean that those edits (via the regular SDC UI) are not included in this data, or does that param have no impact on how a "bot edit" is determined?

Following metrics were added to the draft dashboard.

provided visualizations via email. and data for product health timeline slide deck

Provided data and responded to comments for pageview health timelines deck.

schema and notes updated in the document.

In T300706#7741524, @Isaac wrote:

Thanks for this analysis @cchen and linking to the notebook! Just wanted to point out a small piece of context for others that those median statistics are for just users who added at least one piece of media to an article that month. If you include all the folks who didn't add any media, I assume the median values are 0 (because most users never edit media in their editing work)

Will run one more time before releasing.

A first pass on calculating these baselines has now been done. The numbers and the calculations can be found in this Jupyter notebook. It uses monthly edits for December 2021 as source of the data for ptwiki and ruwiki.

Thank you @Seddon! does the user_properties includes only the image notifications or does it includes opt-outs for other notifications as well?

In T275888#7704624, @Isaac wrote:

@cchen this is an awesome analysis! I wanted to highlight something for any future ones (as I don't think this affect either of these two analyses): the compare API that you used doesn't just return what wikitext was changed, it also returns a few lines of context. So if there was an image file near the edit, that would potentially be counted. I say it probably doesn't affect either of these two analyses because these bots were generally adding e.g., new galleries and in my checking of some of the edits, I didn't see any instances where the context had an image in it. To improve this in future analyses, you might be able to extract just the changed content from the diff or just use the regexes that I compiled and remove any images that show up more than once in the diff HTML (as you'll know then that those were unchanged by the edit) -- e.g.,:

IMAGE_EXTENSIONS = ['.jpg', '.png', '.svg', '.gif', '.jpeg', '.tif', '.bmp', '.webp', '.xcf']
VIDEO_EXTENSIONS = ['.ogv', '.webm', '.mpg', '.mpeg']
AUDIO_EXTENSIONS = ['.ogg', '.mp3', '.mid', '.webm', '.flac', '.wav', '.oga']
MEDIA_EXTENSIONS = list(set(IMAGE_EXTENSIONS + VIDEO_EXTENSIONS + AUDIO_EXTENSIONS))

# build regex that checks for all media extensions
EXTEN_REGEX = ('(' + '|'.join([e + r'\b' for e in MEDIA_EXTENSIONS]) + ')').replace('.', r'\.')
# join in the extension regex with one that requiries at least one alphanumeric and/or a few special characters before it
EXTEN_PATTERN = re.compile(fr'([\w ,().-]+){EXTEN_REGEX}', flags=re.UNICODE)

...

images = EXTEN_PATTERN.findall(diff_html)
image_count = len([i for i in images if images.count(i) == 1])

In T301001#7698487, @Iflorez wrote:

backfill for cchen.repo_active_editors:

editor data for all projects for January 2020 I used the following query to check what data is missing. but looks like editor data for all projects for Jan 2020 is there.

@cchen thank you for these backfills.
I'm still seeing missing data in cchen.repo_active_editors for January 2020 and January 2016. You can see what I'm seeing here in this Superset chart and also in this schreenshot:

The numbers and analysis for media matching bots can be found in this Jupyter notebook. For this analysis, we are collecting metrics for Lsjbot running on Cebuano Wikipedia from 01 Nov 2021 to 31 Jan 2022. We use the comment text Images from API to identify the image edits.

In T299712#7693209, @Isaac wrote:

I ran the code on Russian and Portuguese Wikipedia and was able to get the baselines stats for media additions. I also able to use this on Cebuano Wikipedia to count the images added by Lsj-bot, which the images were all added to the image gallery.

That's awesome! Did you stick to a single month's worth of data or scale up? If so, how long did it take (not a big deal if you don't remember)?

backfill for cchen.new_editors is done

@Isaac I ran the code on Russian and Portuguese Wikipedia and was able to get the baselines stats for media additions. I also able to use this on Cebuano Wikipedia to count the images added by Lsj-bot, which the images were all added to the image gallery.

backfill for cchen.repo_active_editors:

In T300094#7676655, @CBogen wrote:

In T300094#7676495, @egardner wrote:

We are working on making some small changes to ResourceLoader that should restore functionality to users of desktop & mobile Safari from version 11+. Versions 9 and 10 will still be excluded, but if there is significant Safari 10 traffic we can investigate further changes. Do we have traffic stats for Special:MediaSearch or SuggestedTags?

MediaSearch traffic stats are here, but they're not broken down by browser: https://superset.wikimedia.org/superset/dashboard/236/ @cchen might be able to help if it's needed.

CAT usage stats are here, they're also not broken down by browser:
https://analytics.wikimedia.org/published/notebooks/computer-aided-tagging/CAT-usage-report.html

Thank you @Isaac! I will test it out on target wikis.

@cchen Great, thanks! What about

• How many images on average are added by a user per month? (To understand what the current behavior is today and how people engage with images; would help us know if engagement increases based on our work).

@CBogen

• Number of notifications sent (To get a sense of the scope of our work and whether we should increase or decrease)
• Number of opt-outs (To see whether users are annoyed by our notifications or not interested in them, etc)

@CBogen @matthiasmullie yes, we can look for all the revisions of relevant pages and check the revert status.

The draft dashboard have the following monthly metrics available from Jun 2021 - Dec 2021:

Pageview topics dataset is up to date with the latest topic data.

The baselines for countries in emerging market are in this sheet: https://docs.google.com/spreadsheets/d/1lS_ZbLqfgX1KQisA8DwiHAbnmDfyO_JtrBN7vyBNEBk/edit#gid=0