Change Details

The [rollout of temporary users](https://www.mediawiki.org/wiki/Trust_and_Safety_Product/Temporary_Accounts) requires some changes from us. The deadline is the introduction of temporary accounts on the first pilot wikis. As of August 2024, this is scheduled to happen in January 2025. First, we must update our official intermediate tables. T333225 should be finished by October 2024, so we should be able to apply the changes to the new, Airflow-updated tables only. [] `editor_month` (should include temporary users with a `user_is_temp` field) [] `new_editor` (should exclude temporary users) [] `active_editors_monthly` (should exclude temporary users) Second, we must update the [movement metrics code](https://gitlab.wikimedia.org/repos/movement-insights/movement-metrics). This should only require updating the active editors query to exclude temporary users (since it pulls from `editor_month`).

The [rollout of temporary users](https://www.mediawiki.org/wiki/Trust_and_Safety_Product/Temporary_Accounts) requires some changes from us. Now that Data Engineering has made the final data changes to adapt to temp accounts (T377293), we can update our pipelines accordingly. **Step 1**: update and backfill the Airflow-based `editor_month` table [] Make changes to [SQL queries](https://gitlab.wikimedia.org/repos/movement-insights/sql/-/tree/main/editor_month) to add a new `user_is_temporary` field (we won't use the modified create query, but Data Engineering says it's standard practice to change it anyway) [] Once the changes are merged, add a Git tag for the new version of the queries [] Update the [Airflow job](https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/analytics_product/dags/editor_month/editor_month_dag.py?ref_type=heads) to use the new version of the update query [] Once the job changes are merged, deploy the changes and run a manual alter table to add the new field (make sure it's in the same position as in the update query since I'm pretty sure SQL inserts only follow the position, not the name) [] Run the [backfill script](https://gitlab.wikimedia.org/repos/movement-insights/sql/-/blob/main/editor_month/backfill_editor_month.py) for the months when temporary editors have been deployed at any wiki (No changes needed for `new_editor` since `NOT event_user_is_created_by_system` already excludes temp users.) **Step 2**: update the [movement metrics code](https://gitlab.wikimedia.org/repos/movement-insights/movement-metrics). [] Update `active_editors.sql` to use `wmf_contributors.editor_month` and filter out temp users with the new field rather than the regex workaround (since the two filters are equivalent, this will not require a backfill) [] Update `active_editors_region_type.sql` as necessary [] Update `regional_active_editors.sql`, re-enable the query, and backfill the missing data [] Update `new_editor_retention.sql` to use `wmf_contributors.new_editor`

The [rollout of temporary users](https://www.mediawiki.org/wiki/Trust_and_Safety_Product/Temporary_Accounts) requires some changes from us. The deadline is the introduction of temporary accounts on the first pilot wikis. As of August 2024Now that Data Engineering has made the final data changes to adapt to temp accounts (T377293), this is scheduled to happen in January 2025. First, we must update our official intermediate tables. T333225 should be finished by October 2024, so we should be able to apply the changes to the newwe can update our pipelines accordingly. **Step 1**: update and backfill the Airflow-based `editor_month` table [] Make changes to [SQL queries](https://gitlab.wikimedia.org/repos/movement-insights/sql/-/tree/main/editor_month) to add a new `user_is_temporary` field (we won't use the modified create query, Airflow-updated tables only.but Data Engineering says it's standard practice to change it anyway) [] `editor_month` (should include temporary users with a `user_is_temp` field)Once the changes are merged, add a Git tag for the new version of the queries [] `new_editor` (should exclude temporary users)Update the [Airflow job](https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/analytics_product/dags/editor_month/editor_month_dag.py?ref_type=heads) to use the new version of the update query [] `active_editors_monthly` (should exclude temporary usersOnce the job changes are merged, deploy the changes and run a manual alter table to add the new field (make sure it's in the same position as in the update query since I'm pretty sure SQL inserts only follow the position, not the name) [] Run the [backfill script](https://gitlab.wikimedia.org/repos/movement-insights/sql/-/blob/main/editor_month/backfill_editor_month.py) for the months when temporary editors have been deployed at any wiki (No changes needed for `new_editor` since `NOT event_user_is_created_by_system` already excludes temp users.) Second**Step 2**: update the [movement metrics code](https://gitlab.wikimedia.org/repos/movement-insights/movement-metrics). [] Update `active_editors.sql` to use `wmf_contributors.editor_month` and filter out temp users with the new field rather than the regex workaround (since the two filters are equivalent, we must update the [movement metrics code](https://gitlab.wikimedia.org/repos/movement-insights/movement-metrics).this will not require a backfill) [] Update `active_editors_region_type.sql` as necessary [] Update `regional_active_editors.sql`, re-enable the query, This should only require updating the active and backfill the missing data [] Update `new_editors query to exclude temporary_retention.sql` to users (since it pulls from ` `wmf_contributors.new_editor_month`).`