DAG has been deployed and the originally reported data has been inserted into the new table. We'll get the first run on the 1st of May 😊

Queries have been converted over for a monthly DAG, with the output being:

Seen, @Ifrahkhanyaree_WMDE! Thanks :)

Realized I'd made a task for this already in T391463, but this task also had some extra details that I've moved over to the original :)

T391813 was made as a monthly repeating task and T391463 was already made to create the DAG for the metrics, so we're good to close this :)

Hey @Ifeatu_Nnaobi_WMDE 👋 Yes, let's plan on being in touch at the end of April for me to run the notebook again on May 2nd (a Friday). I'll make another task for this and post it here. Once that's done we should be good to resolve :)

Leaving this task as In Progress as I'd suggest we do the DAG for this early next week so the process is finished up :) All work for the February and March values is done though. I really am not sure why we had such a dip in March. Yes we're using wmf_deprecated.webrequest as the March numbers in wmf.webrequest has restricted historical data now, but I did check wmf_deprecated.webrequest and the maximum day in March is 31, so we do have the full month :)

The query for this can be found below. The value for month = 2 in WITH wdi_doc_page_view_flags AS (...) needs to be changed to get the values from other months.

Stalling this task as we need to figure out how to get MariaDB data into the data lake for beforehand, and this will likely be done in T360296 :)

The stall was was meant for T391463 :)

Thanks @colewhite! Good point on the Grafana part of all of this. We've been very focused on the Graphite side of things. I'll look into Grafana early next week when I'm back from a vacation. Hopefully we can get this done on Monday :) Saving this message from you at the top of my notifications.

Moved to In Review as the results will be sent along in Mattermost shortly :)

Note that included in this would be a quick check of the Enhanced view setting as discussed on Mattermost here.

Full overview of the work done to migrate those Airflow based processes can be seen in https://phabricator.wikimedia.org/T377352#10679786 :)

Claiming this to say that the rest of this was finished in T389205 given the comment above is referencing the same dashboard that had its metrics migrated over to new Airflow DAGs.

The patch for T359252 was combined into Wikibase/+/1126096, as my assumption is that we can't do a scoped migration of one sub-directory of WIkibase. I'd done this for Wikibase/view, but instances of StatsdDataFactory were being called from what I assume were other parts of Wikibase.

Wikibase/+/1125464 was getting errors from the BufferingStatsdDataFactory being passed to StatsFactory when all instances of StatsdDataFactory in Wikibase/view had been switched over. My assumption was that we're getting factories from calls in other parts of Wikibase, so the work within Wikibase/+/1125464 was moved over to Wikibase/+/1126096.

Yes that would also make sense, @Ifrahkhanyaree_WMDE! Thanks :) Is definitely something we should look into more, and I can talk to the author about it as that's the person I do my 1:1s with for data engineering work.

@lmata, reassigning this task to you for review/confirmation of the work that was done here. There are three open sub-tasks, each of which has a merged patch with changes due to roll out on April 8th's train:

Happy to help, @Snwachukwu, and thanks for your coordination here! 😊

Helloooo! No need to apologize whatsoever :) T390723 has been created as a follow-up task. Looking forward to looking into this more later!

Back to In Review as merge conflicts are fixed and tests are again passing.

Back to In Review as merge conflicts are fixed and tests are again passing.