Change Details

== Wikidata Analytics Request == > This task was generated using the [Wikidata Analytics](https://phabricator.wikimedia.org/project/profile/5408) request form. Please use the task template linked on our project page to create tasks for the team. Thank you! === Purpose === > Please provide as much context as possible as well as what the produced insights or services will be used for. As part of the Wikidata Integrations Team's annual goals, we would like to track the awareness of Wikidata Integration by monitoring the pages of particular help pages hosted on Wikidata. These types of help pages provide useful information for editors who want to use WIkidata's data in their projects. An example of such a page includes the recently revamped [[ https://www.wikidata.org/wiki/Help:Sitelinks | Help Sitelinks ]] page. We want to check if improving the information on the pages attracts more users. === Specific Results === > Please detail the specific results that the task should deliver. A pipeline for this directly such that the data is saved on a regular basis e.g. on a monthly or quarterly basis. The pages we are concerned about are: - [[ https://www.wikidata.org/wiki/Help:Items | Help:Items ]] - [[ https://www.wikidata.org/wiki/Help:Sitelinks | Help:Sitelinks ]] - [[ https://www.wikidata.org/wiki/Help:Edit_summary | Help:Edit Summary ]] - [[ https://www.wikidata.org/wiki/Wikidata:Wikidata_in_Wikimedia_projects | Wikidata in Wikimedia Projects ]] - [[ https://www.wikidata.org/wiki/Wikidata:How_to_use_data_on_Wikimedia_projects | How to use Wikidata in Wikimedia Projects ]] === Desired Outputs === > Please list the desired outputs of this task. [x] A notebook that explores the process to get these numebrs - [T390891_wd_integrations_doc_page_views.ipynb](https://gitlab.wikimedia.org/repos/wmde/analytics/-/blob/main/tasks/wikidata_integrations/2025/T390891_wd_integrations_doc_page_views/T390891_wd_integrations_doc_page_views.ipynb?ref_type=heads) [x] A query that can be run to generate the number on Superset or any other recommended platform - View query at https://phabricator.wikimedia.org/T390891#10726926 [x] The values for February and March reported **February 2025** - `total_help_items_views`: `47,017` - `total_help_sitelinks_views`: `6,255` - `total_help_edit_summary_view`: `76` - `total_wikidata_wiwp_views`: `469` - `total_wikidata_how_to_use_wd_views`: `3,679` - Total from summing the above: `57,496` **March 2025** - `total_help_items_views`: `12,557` - `total_help_sitelinks_views`: `4,076` - `total_help_edit_summary_view`: `70` - `total_wikidata_wiwp_views`: `530` - `total_wikidata_how_to_use_wd_views`: `1,974` - Total from summing the above: `19,207` NOTE: The process of adding in a DAG will be added to this task - see below: [x] A DAG to collect these metrics on a monthly basis [x] The data from the original task being added to the new table === Deadline === > Please make the time sensitivity of this request clear with a date that it should be completed by. If there is no specific date, then the task will be triaged based on its priority. 09.04.2025 --- **Information below this point is filled out by the task assignee.** == Assignee Planning == === Sub Tasks === > A full breakdown of the steps to complete this task. **February and March Numbers** [x] Explore potentially useful tables ([DataHub](https://datahub.wikimedia.org/)) - [wmf.webrequest](https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,wmf.webrequest,PROD)/Schema?is_lineage_mode=false&schemaFilter=) - `wmf_deprecated.webrequest` [x] Derive method of getting views for specific pages [x] Get totals per page and aggregate for February and March **Monthly DAG** [x] Write Iceberg table create table script [x] Create a Iceberg table in Hive/HDFS within the `wmde` namespace [x] Convert monthly stats query to production Airflow query [x] Generate testing table generation and query scripts [x] Write DAG to run job query [x] Write DAG tests [x] Run tests on process as possible [x] Deploy DAG **Adding Old Data** [x] Add data from notebook and `wmf_deprecated.webrequest` to the new Iceberg table === Estimation === Estimate - Feb/Mar numbers: 1/2 a day Actual - Feb/Mar numbers: 1/2 a day Estimate - DAG: 1/2 a day Actual - DAG: 1/2 a day === Data === > The tables that will be referenced in this task. - `wmf.webrequest` - `wmf_deprecated.webrequest` === Notes === > Things that came up during the completion of this task, questions to be answered and follow-up tasks. NOTE: We needed to use `wmf_deprecated.webrequest` for this task as the data for February and March isn't in `wmf.webrequest` because of the `HAProxy` migration.

== Wikidata Analytics Request == > This task was generated using the [Wikidata Analytics](https://phabricator.wikimedia.org/project/profile/5408) request form. Please use the task template linked on our project page to create tasks for the team. Thank you! === Purpose === > Please provide as much context as possible as well as what the produced insights or services will be used for. As part of the Wikidata Integrations Team's annual goals, we would like to track the awareness of Wikidata Integration by monitoring the pages of particular help pages hosted on Wikidata. These types of help pages provide useful information for editors who want to use WIkidata's data in their projects. An example of such a page includes the recently revamped [[ https://www.wikidata.org/wiki/Help:Sitelinks | Help Sitelinks ]] page. We want to check if improving the information on the pages attracts more users. === Specific Results === > Please detail the specific results that the task should deliver. A pipeline for this directly such that the data is saved on a regular basis e.g. on a monthly or quarterly basis. The pages we are concerned about are: - [[ https://www.wikidata.org/wiki/Help:Items | Help:Items ]] - [[ https://www.wikidata.org/wiki/Help:Sitelinks | Help:Sitelinks ]] - [[ https://www.wikidata.org/wiki/Help:Edit_summary | Help:Edit Summary ]] - [[ https://www.wikidata.org/wiki/Wikidata:Wikidata_in_Wikimedia_projects | Wikidata in Wikimedia Projects ]] - [[ https://www.wikidata.org/wiki/Wikidata:How_to_use_data_on_Wikimedia_projects | How to use Wikidata in Wikimedia Projects ]] === Desired Outputs === > Please list the desired outputs of this task. [x] A notebook that explores the process to get these numebrs - [T390891_wd_integrations_doc_page_views.ipynb](https://gitlab.wikimedia.org/repos/wmde/analytics/-/blob/main/tasks/wikidata_integrations/2025/T390891_wd_integrations_doc_page_views/T390891_wd_integrations_doc_page_views.ipynb?ref_type=heads) [x] A query that can be run to generate the number on Superset or any other recommended platform - View query at https://phabricator.wikimedia.org/T390891#10726926 [x] The values for February and March reported **February 2025** - `total_help_items_views`: `47,017` - `total_help_sitelinks_views`: `6,255` - `total_help_edit_summary_view`: `76` - `total_wikidata_wiwp_views`: `469` - `total_wikidata_how_to_use_wd_views`: `3,679` - Total from summing the above: `57,496` **March 2025** - `total_help_items_views`: `12,557` - `total_help_sitelinks_views`: `4,076` - `total_help_edit_summary_view`: `70` - `total_wikidata_wiwp_views`: `530` - `total_wikidata_how_to_use_wd_views`: `1,974` - Total from summing the above: `19,207` NOTE: The process of adding in a DAG will be added to this task - see below: [x] A DAG to collect these metrics on a monthly basis - Populates the table `wmde.wit_docs_pageview_metrics_monthly` [x] The data from the original task being added to the new table === Deadline === > Please make the time sensitivity of this request clear with a date that it should be completed by. If there is no specific date, then the task will be triaged based on its priority. 09.04.2025 --- **Information below this point is filled out by the task assignee.** == Assignee Planning == === Sub Tasks === > A full breakdown of the steps to complete this task. **February and March Numbers** [x] Explore potentially useful tables ([DataHub](https://datahub.wikimedia.org/)) - [wmf.webrequest](https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,wmf.webrequest,PROD)/Schema?is_lineage_mode=false&schemaFilter=) - `wmf_deprecated.webrequest` [x] Derive method of getting views for specific pages [x] Get totals per page and aggregate for February and March **Monthly DAG** [x] Write Iceberg table create table script [x] Create a Iceberg table in Hive/HDFS within the `wmde` namespace [x] Convert monthly stats query to production Airflow query [x] Generate testing table generation and query scripts [x] Write DAG to run job query [x] Write DAG tests [x] Run tests on process as possible [x] Deploy DAG **Adding Old Data** [x] Add data from notebook and `wmf_deprecated.webrequest` to the new Iceberg table === Estimation === Estimate - Feb/Mar numbers: 1/2 a day Actual - Feb/Mar numbers: 1/2 a day Estimate - DAG: 1/2 a day Actual - DAG: 1/2 a day === Data === > The tables that will be referenced in this task. - `wmf.webrequest` - `wmf_deprecated.webrequest` === Notes === > Things that came up during the completion of this task, questions to be answered and follow-up tasks. NOTE: We needed to use `wmf_deprecated.webrequest` for this task as the data for February and March isn't in `wmf.webrequest` because of the `HAProxy` migration.