Page MenuHomePhabricator

🧹 [Random Sampling] Compile results of the random sampling investigation (😅)
Open, Needs TriagePublic

Description

This task is about how we tie together the various bits of information we learn about each page. The overall results should be in a convenient format, and the choice of columns is important. HTML diffs could be stored separately as .html files, since we will probably want to go over these manually.

Draft columns:

ColumnDescription
wikiSource wiki dbname (eg. enwiki)
page_titleName of the page
page_idDatabase ID, can be used on-wiki as ?curid=
page_urlCourtesy link to the page on its home wiki
revision_idRevision ID seen in the 1 June snapshot, and re-rendered.
rc_countHow many Wikidata-injected RC logs were seen for this page over the time frame. Must be at least 1 to be in our sample. More than 1 injected change means that multiple Wikidata changes affected the page, we may end up excluding these.
has_text_diffWas there a difference in visible text content? (yes / no)
has_img_alt_text_diffWas there a difference in img alt text? (yes / no)
is_language_linkIs this a change to sitelinks? (yes / no)
snapshot_html_urlPublic URL to a copy of the page HTML, taken from the Enterprise snapshot.
rerendered_html_urlPublic URL to the page HTML as it looked when we re-rendered.
diff_html_urlPublic URL to the diff between the snapshotted and rerendered page.
rc_commentComment from the first RC log.
wbc_entity_usage_aspectEntity usage aspect from the first RC log.
change_classificationHow would we classify the change according to the taxonomy in T249654: Categorize different types of Wikidata re-use within Wikimedia projects?
is_reviewed_okayDiff was manually reviewed and looks like a Wikidata thing. (yes / no / blank)
notesManually-entered comments about the page.

HTML diffs (as well as original / new page HTML) should be published somewhere that can be publicly hyperlinked, so we can all easily navigate between results CSV and the diffs. If we follow https://wikitech.wikimedia.org/wiki/Data_Platform/Web_publication our data is from public sources and therefore the lowest risk category "tier 3", so it should be okay to push the results directly to a subdirectory of https://analytics.wikimedia.org/published/datasets/one-off/ from the analytics cluster.

A potential manual review workflow is that we import the results CSV into a shared tool eg. a Google Drive spreadsheet. (Design this import in a way that we can re-import revised results without overwriting manual work.) Then the public HTML files are linked, and we can record our notes in the shared spreadsheet.

Details

Other Assignee
Nicholusmuwonge_wmde

Event Timeline

Lucyfediachambers renamed this task from 🧹 Compile results of the random sampling investigation to 🧹 Compile results of the random sampling investigation (😅).Jul 3 2026, 8:13 AM

This is blocked for now because its awaiting the text diffing functionality

Lucyfediachambers renamed this task from 🧹 Compile results of the random sampling investigation (😅) to 🧹 [Random Sampling] Compile results of the random sampling investigation (😅).Mon, Jul 13, 8:59 AM

Code to review: https://gitlab.wikimedia.org/repos/wmde/wikidata-integration/external-rc-sampler/-/merge_requests/19

Smushed the outputs together with these commands:

head -1 outputs/arwiki/results.csv > outputs/all-wikis-results.csv
for o in outputs/*/results.csv; do tail +1 $o >> outputs/all-wikis-results.csv; done

Now publishing all results under https://analytics.wikimedia.org/published/datasets/one-off/external-rc-sampler/2026-07/

Published using these commands:

mkdir -p /srv/published/datasets/one-off/external-rc-sampler/2026-07
rsync -av diffs outputs /srv/published/datasets/one-off/external-rc-sampler/2026-07