Synced with @mforns and finished one set of deployment staging, and some metrics(daily edits, and daily edits by registered users) are deployed to https://edit-analysis.wmflabs.org/editor-engagement/, with same language configurations as the limm-ee dashboards have for these metrics. More metrics coming.

Thanks for the tip! Will add in the next commit

@elukey I tried to access pivot with my Wikitech credential, and now it finally worked. Thanks for pointing me to the right direction! Much appreciated.

May I be added to the WMF group? I tried to access https://pivot.wikimedia.org/ with my Wikitech credential but it did not work...My wikitech user name is Hjiang. Thanks a lot and I will really appreciate your help!

Amongst the four metrics to move, the first step is taken(modifying queries, routing to pull data from stats machines, etc.). The four metrics are: daily edits; registered human users; registered mobile users; anonymous users. First two tasks are completed for the first step(aka replicated what Marcel showed me), the latter two should be replicated fairly in a straightforward manner. Second step: integrate with Dashiki.

@mpopov I tried 3 times to reproduce your results with your code, but each time the drift and fit_google actually match up with results from @chelsyx. Also, from what I have dealt with time series, there are different non-stationary behaviors to check for before differencing: trends, cycles, random-walks, disturbance, etc., and it helps a lot to try different stationarizing techniques, depending on what kind of non-stationary behavior the TS exhibits. De-trending, de-differencing(the key stats are constant over periods or seasons), etc. could be helpful in this context

In addition, ongoing conversations is underway to further understand old queries, their constructions, and the rationales behind the decisions(parts of documentations are in https://meta.wikimedia.org/wiki/Research:Standard_metrics), and decide whether/what to keep in the new dashboard. Also, for potential Dashiki layouts for the migration, see metrics-by-project layout in Dashiki documentation.

Analytics(@Milimetric, @mforns) and Editing(@HJiang-WMF) are working on this. We will start out by picking a single query or report vertical slice(for testing purpose and straight out the process), then implement, adapt, and test deployed once data is in and set up the dashboard.

Majority of documentations are posted at: https://wikitech.wikimedia.org/wiki/Analytics/Dashiki in the "Technical documentation - understanding Dashiki" section.

Keeping with the config page writing - notices added. 2nd MWE with JS is WIP

Updated with a minimal working example and key dependencies that are most likely to cause confusions.

Hi @Aklapper - yes, that is the plan! I'm still making edits and trying to add small examples. so currently it's shared amongst only a few folks for feedback. The feedback has been very positive so far, so the document should be on a public wiki page soon.

Rough first draft finished.
Link:
https://docs.google.com/a/wikimedia.org/document/d/18dK-2W1V5RQaIxlXb8bZHZqDEKKY5CN5n1J70FTkQnU/edit?usp=sharing

In fact, even just from an aggregated point of view, it's fairly obvious that it doesn't matter whether the "Main page" in page_title is in English or local language. For example:

A workaround HQL query is in T142505, thanks to pointers from @Milimetric about the user-agent. Examining the same UA around July 20th, the UA went from being usually fewer than 5% of pageviews on a particular wiki, to suddenly become more than two times the rest of pageviews combined.

The query. This example is for getting data between August 1st and 16th of RU wiki pageview data in the formatted manner as posted above.

Since the outrageous spike happened starting on July 20th with Chrome 41 on Windows being the driver behind the spike, I also looked into pageview data before July 20th to find out the usual percentage of Chrome 41 on Windows. It turns out that it is roughly between 3.5% to 5% of normal pageview data. So if we would want further finetuning adjustment, probably that could be considered, too.

All preliminary adjusted pageview counts for the three wikis:

Below is the minimally formatted output from Hive on correct EN Wiki pageview starting July 20th to July 31st, the 1st day we start to see skewed pageview data.

Issue resolved. Thanks everyone! Turns out that I accidentally overwrote the alias for bast4001 and that is the root cause of the hung-up ssh access. 1 alias is enough. Closing the ticket.

I logged in to stat1002 previously, but the login process at stat1003 has been so slow that I'm not sure if it has ever logged me it at all. So looks like that it is not ever trying to connect me to the machine at all...

ssh -v stat1003 output with changed config add !bast*001 command:
(thel laptop died from producing the output so I took a pictyre and transcribed the output)

ssh -v stat1003 output:
(thel laptop died from producing the output so I took a pictyre and transcribed the output)

The "password" issue and connection to bastion hosts are resolved so closing the ticket. The time out issue of logging in to stat machines is located at: https://phabricator.wikimedia.org/T142126. Thanks everyone for helping!

Clarification: stat1003 log in is so slow that I'm not certain if it would actually be logged in. The "password" issue is resolved, thanks!

No, I meant that even with my ssh config specified that I could log in just at my local machine with ssh stat1003, it did not log me in and dragged my machine to halt and death every time...My ssh config specified details from stat1001 to stat1004, and all but stat1003 work...

Hi, I tried the chmod 600 * for a 2nd time and was able to log in to myusername@bast1001 without sudo, but when I tried to ssh from there in to any specific stat machines (e.g. stat1001), the same "password" prompt appeared and the log in credential I have on WikiTech nor Phabricator authenticated me...I configed the ssh config file and set the identity file to my key. I'm very confused about what is to be done from here...

ssh-add -L lists exactly the key listed in the previous comment

Outage of access to bast1001.

Thanks a lot Dan! Will do the config and try the connection as you suggested.

SSH public key:

As instructed by https://wikitech.wikimedia.org/wiki/Production_shell_access, my preferred shell user name is: hjiang

I already signed the Acknowledgment of Wikimedia Server Access Responsibilities.