Change Details

In this task we should: [] Rename the table from `wmf_dumps.wikitext_raw_rc2` to `wmf_dumps.wikitext_raw`. If we cannot rename, then re-create, re-backfill, and run data quality checks. [] Check all DAG dependencies to make sure they are sending alerts to the team. [] Check all DQ checks to make sure they are sending alerts to the team. [] Add config to have it on DataHub [] Announce its availability on the usual channels. More: >>! In T366542#10472130, @xcollazo wrote: > Looks like production is happily catching up: https://airflow-analytics.wikimedia.org/home?tags=mediawiki_content. > > I think we can close this task, but wrting down a few clean up tasks that perhaps we can do elsewhere: > [] Bump the flink app back down from 20 task managers to 2. > [] Delete old DAGs at https://airflow-analytics.wikimedia.org/home?tags=mediawiki_dumps > [] Delete old wikitext_raw pool at https://airflow-analytics.wikimedia.org/pool/list/ > [] Update docs at https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Content/Mediawiki_content_history_v1

In this task we should: [x] Rename the table from `wmf_dumps.wikitext_raw_rc2` to `wmf_content.mediawiki_content_history_v1`. If we cannot rename, then re-create, re-backfill, and run data quality checks. [] Check all DAG dependencies to make sure they are sending alerts to the team. [] Check all DQ checks to make sure they are sending alerts to the team. [] Add config to have it on DataHub [] Announce its availability on the usual channels. More: >>! In T366542#10472130, @xcollazo wrote: > Looks like production is happily catching up: https://airflow-analytics.wikimedia.org/home?tags=mediawiki_content. > > I think we can close this task, but wrting down a few clean up tasks that perhaps we can do elsewhere: > [] Bump the flink app back down from 20 task managers to 2. > [] Delete old DAGs at https://airflow-analytics.wikimedia.org/home?tags=mediawiki_dumps > [] Delete old wikitext_raw pool at https://airflow-analytics.wikimedia.org/pool/list/ > [] Update docs at https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Content/Mediawiki_content_history_v1

In this task we should: [x] Rename the table from `wmf_dumps.wikitext_raw_rc2` to `wmf_dumps.content.mediawikitext_raw`_content_history_v1`. If we cannot rename, then re-create, re-backfill, and run data quality checks. [] Check all DAG dependencies to make sure they are sending alerts to the team. [] Check all DQ checks to make sure they are sending alerts to the team. [] Add config to have it on DataHub [] Announce its availability on the usual channels. More: >>! In T366542#10472130, @xcollazo wrote: > Looks like production is happily catching up: https://airflow-analytics.wikimedia.org/home?tags=mediawiki_content. > > I think we can close this task, but wrting down a few clean up tasks that perhaps we can do elsewhere: > [] Bump the flink app back down from 20 task managers to 2. > [] Delete old DAGs at https://airflow-analytics.wikimedia.org/home?tags=mediawiki_dumps > [] Delete old wikitext_raw pool at https://airflow-analytics.wikimedia.org/pool/list/ > [] Update docs at https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Content/Mediawiki_content_history_v1