This task is about how we tie together the various bits of information we learn about each page. The overall results should be in a convenient format, and the choice of columns is important. HTML diffs could be stored separately as .html files, since we will probably want to go over these manually.
Draft columns:
| Column | Description |
|---|---|
| wiki | Source wiki dbname (eg. enwiki) |
| page_title | Name of the page |
| page_id | Database ID, can be used on-wiki as ?curid= |
| page_url | Courtesy link to the page on its home wiki |
| revision_id | Revision ID seen in the 1 June snapshot, and re-rendered. |
| rc_count | How many Wikidata-injected RC logs were seen for this page over the time frame. Must be at least 1 to be in our sample. More than 1 injected change means that multiple Wikidata changes affected the page, we may end up excluding these. |
| has_text_diff | Was there a difference in visible text content? (yes / no) |
| has_img_alt_text_diff | Was there a difference in img alt text? (yes / no) |
| is_language_link | Is this a change to sitelinks? (yes / no) |
| snapshot_html_url | Public URL to a copy of the page HTML, taken from the Enterprise snapshot. |
| rerendered_html_url | Public URL to the page HTML as it looked when we re-rendered. |
| diff_html_url | Public URL to the diff between the snapshotted and rerendered page. |
| rc_comment | Comment from the first RC log. |
| wbc_entity_usage_aspect | Entity usage aspect from the first RC log. |
| change_classification | How would we classify the change according to the taxonomy in T249654: Categorize different types of Wikidata re-use within Wikimedia projects? |
| is_reviewed_okay | Diff was manually reviewed and looks like a Wikidata thing. (yes / no / blank) |
| notes | Manually-entered comments about the page. |
HTML diffs (as well as original / new page HTML) should be published somewhere that can be publicly hyperlinked, so we can all easily navigate between results CSV and the diffs. If we follow https://wikitech.wikimedia.org/wiki/Data_Platform/Web_publication our data is from public sources and therefore the lowest risk category "tier 3", so it should be okay to push the results directly to a subdirectory of https://analytics.wikimedia.org/published/datasets/one-off/ from the analytics cluster.
A potential manual review workflow is that we import the results CSV into a shared tool eg. a Google Drive spreadsheet. (Design this import in a way that we can re-import revised results without overwriting manual work.) Then the public HTML files are linked, and we can record our notes in the shared spreadsheet.