@Stefan_Schneider_WMDE Your daily update for 21. October 2018 is ready.

I will be reporting on the latest data very soon, Stefan

@Jan_Dittrich Here we go:

Daily Reporting for the Autumn Banner Campaign 2018 is now available from this spreadsheet.

@Jan_Dittrich Thank you for your comments. I will provide all necessary explanations later in the evening.

@Stefan_Schneider_WMDE You're awesome, thank you very much for providing assistance here!

Could you please describe all user journeys for this campaign as precisely as possible by filling out this spreadsheet?

@Jan_Dittrich Please find your Report attached. Pinging @Lydia_Pintscher who might also be interested to take a look at the results.

Very nice, all three found:

@kai.nissen Kai, could you please let me know what campaign tags do we expect to find in relation to the two test registrations that you have made during our discussion with @Stefan_Schneider_WMDE yesterday? Thank you. If they are 'lp1' or 'lpn' - your test user registrations did not hit the log database.

Warning: two accounts that were registered by Kai during our discussion earlier today are not found in log.ServerSideAccountCreation_17719237 (the place where user registrations live).

@Stefan_Schneider_WMDE Not yet; I guess we will have to wait some time for these two to show up in the table.

I can confirm that the registrations for both tested campaign tags can be obtained from the relevant schema now.

@johl Of course! Thank you very much.

I have re-checked the campaign tags for the Autumn Banner Campaign 2018, beginning from Oct 1.

• Re-checking and getting back to you. I think we would need to get @kai.nissen involved in the discussion then.

• Done. Closed.

@Stefan_Schneider_WMDE User registration test completed:

Pageviews and campaign tags tested:

@Lydia_Pintscher And the solutions is linked from https://grafana.wikimedia.org/dashboard/db/wikidata-entity-usage

@Lydia_Pintscher Here's where the daily update can be found online: http://wdcm.wmflabs.org/WD_percentUsageDashboard/

@johl Hey Jens: do you think you could help @Lea_Lacroix_WMDE and me get thus published on the WMF blog? We think that the product described in this blog post is quite useful to the Wiktionary community. Thanks!

@elukey Done - it's probably the sync period that has caused the confusion. Thanks a lot!

@elukey Still experimenting to compare the mapping from stat1004 and stat1004 onto https://analytics.wikimedia.org/datasets/.

@elukey Interesting. Because from https://analytics.wikimedia.org/datasets/ one does not see this directory (https://analytics.wikimedia.org/datasets/wdUsagePercentArticle/)?

@Lydia_Pintscher Here's a table with the desired statistics.

• Running manually another WDCM full monthly updated because of the changes that were introduced in Aug/Sep 2018;
• cleaning up unnecessary reporting from the main update script during the manual update.

I was surprised to learn that too. Once again: in spite of the fact that the results are somewhat approximate, for the reasons already described, the difference between the time needed to reach the 10th edit between the campaign registered users and those who register spontaneously could not be that large if it was not a consequence of a real, existing effect. I am eyeballing here, I did not conduct any statistical tests, but given the sample sizes and the fact that any reasonable test in this context would be fairly simple, I think I can safely conclude this without any further testing. In my opinion, this is probably the most important effect of our campaigns that we have learned about thus far, or at least since I've started doing analytics for you.

@Stefan_Schneider_WMDE The bug should not have affected anything else, but if something in particular looks suspicious to you: please let me know immediately.

@Stefan_Schneider_WMDE Here are the corrected, much more realistically looking data.

@Stefan_Schneider_WMDE To report back on what I have now: the counts for edits >= 50 (and thus most probably for edits >= 10) for the campaigns are certainly incorrect; working now to find out what has caused the discrepancy.

@Stefan_Schneider_WMDE I have to admit that I was already very tired when I've submitted the results and that I am also surprised to learn. Please let me double check that for you. I will cross-validate the result by fetching the data directly ftom the dewiki.revision table, and not the derivative monthly aggregate.

The only thing to do is change from local user id + wiki to user_text!

Time users need to get to the 10. Edit of the following groups

• newly registered users per campaign
• all other registered users

number of users per campaign of the following groups

• 1 edit
• 2-4 edits
• 5-9 edits
• 10-49 edits
• >50 edits

Monthly edits per month AND accumulated of the following groups

• newly registered users per campaign
• all other registered users Timeframe: from beginning 2017 - end of time

@Verena SORRY: I am on it!

@Lea_Lacroix_WMDE: here's the draft.

• Hierarchical representation for hclust() changed from {networkD3} (bugs in animation) to static radial, Hills plot from {ape}.

• Project Graph completed; also introduced:
• hierarchical representations for hclust() for Items and Projects.

• {DT} related issues fixed; regex was digging out out an item id from Murakami's book "IQ84".

• The annotation problem is solved in the simplest possible way: no tf-idf, no minEnt selection, only pull class annotations for top M items from a topic where M is the same number of items as used to compute topic coherence in model selection;
• some {DT} related issues still persisting, fixing now; most probably some unnoticed vector recycling takes place.

All instances in the wikidataconcepts project needs to upgrade as soon as possible.

@WMDE-leszek Anytime.

@WMDE-leszek I should be able to provide for this data from slight modifications of the already existing WDCM engine scripts.
Let me know if you'd like me to pick this one up.

@GoranSMilovanovic REMINDER: update needed for Sep 24, see: https://docs.google.com/document/d/1ISjgeV1yL5naHjAlWfTG_mfeCn64-Pb8xx_gqbrDdo0/edit?ts=5ba0ebfa#

• {DT} related issues are now fixed.

As of the following:

The following:

@Lydia_Pintscher Let me know if this is for me, and I'll claim it.

@Lydia_Pintscher This looks like... well... should I claim it? Please advise. Thanks.

I will not be able to take a look at this dashboard before Saturday (09/15) or Sunday (09/16).

• With Map-Reduce, this update runs smoothly.
• If I can fix T200609, we will most probably switch to Spark here.

• This is going to be a general WDCM task, not something implemented specifically for the WDCM Wikipedia Semantics (aka. WDCM Sitelinks).
• Removing parent task.

• R datatables on the dashboard not showing all semantic themes; only ten themes per category are present.
• Check; this has to do with the relevant DT:: function parameterization.

• Prima facie validity with M = 15 top topic items seems acceptable.
• This is online and can be tested.
• The 'Distinctive Classes' annotations should be changed to reflect only the classes that typically annotate the top M topic items.
• Implementing changes in annotations will be tracked on T203238.
• Closing.

• Results: as expected, the number of topics encompassed by LDA models selected in respect to topic coherence is larger than when purely statistical (perplexity) measures are used.
• Currently using M = 5 top topic terms (items, in our case);
• Experimenting with M = 10, M = 15 (most frequently mentioned in the literature) values now.

• Topic coherence measure implemented;
• Running model selection in respect to topic coherence.

• Sync complete; WDCM main dashboards updated;
• Changes in the WDCM_SqoopClients.R introduced;
• Soon to re-run the update.