@Hanna_Klein_WMDE You are welcome, Hanna. Let me know once we begin.

Re-assigning.

Re-assigning.

@Hanna_Klein_WMDE The analytics code is in place. I have tested but I still find no pageviews with the campaign tags as defined in the tracking doc. Did we start the campaign already or not yet? Thanks!

@Hanna_Klein_WMDE Thanks, I am on it.

@amy_rc I think I've found the cause of things like T277564#7157895. It definitely has to do with the following observation of yours:

page views of certain landing pages in Wikipedia, Wikidata and Commons and Wikivoyage

• Wikidata Languages Landscape automation from stat1008 Analytcs Client: done.

Could it be the case that mapping relation type is treated a separator - which overrides the single value constraint - and the Curious Facts system then recognizes that only two out of four present values are violations of the single value constraint?

@amy_rc The part unclear to me is the following one:

@amy_rc Could you please clarify T277564#7158984? Thank you.

@MisterSynergy Could you please check the wdcm_topItems.csv dataset now and let me know if it looks alright?

Please disregard all previous findings. The following is based on:

• full manual update of the WDCM Sqoop procedure is now completed;
• 876 partitions (wiki_db) are present in the Data Lake, which means that everything should be fine,
• except for if something changed in the per wiki wbc_entity_usage table.

• Sqoop Shard 4 running now (Commons): in comparison to what was observed from the WDCM Sqoop Clients Log in T284850#7152935, I see no problem in relation to Commons anymore: the Commons database is found, and its wbc_entity_usage table is being sqooped with no problems right now. But then - what went wrong during the previous update?

Monitoring Sqoop procedures from Core MediaWiki databases to goransm.wdcm_clients_wb_entity_usage in the DataLake:

• running a manual update of the WDCM sqoop module now;
• monitoring.

However, I ran into a situation where the data was being retrieved incorrectly. This has happened couple of times. For instance: Qurious Facts: Silver-Russell syndrome (Q2142496) has 2 values for property: OMIM ID (P492). ( This fact was established on: 2021-06-01 22:59:24, and is based on the 2021-05-17 snapshot in hdfs of the Wikidata JSON dump. Edits made after that date are not taken into account.) But, https://www.wikidata.org/wiki/Q2142496 has three OMIM ID and was last edited on 30 November 2020, at 02:52.

@amy_rc Issue descriptions modified to look exactly as suggested in T277551#7137941, please test:

• Full system update completed;
• Issue descriptions are now fixed (local tests completed);
• deploying soon; it will be ready for tests in an hour or so.

@MisterSynergy Thank you. No worries, I will figure this out from the WDCM sqoop logs. Sooner or later.

• On the first sight, there were only 687 projects whose reuse data were sqooped by the WDCM_Sqoop_Clients.R run, and
• that number, as far as I remember, should be higher;
• Could it be that something in the organization of our core Mediawiki databases has changed?
• Inspecting now, here's the first suspect:

Please share the previous version of wdcm_topItems.csv here. I am on it. Highest priority. Thank you for catching this.

As of T255446 which

@amy_rc That is rather strange. I am running a full system update now in relation to T277551; let's wait for the new update and then check out if the problem persists. I will perform the tests and let you know if the problem is systematic or (hopefully) not. Thank you for catching this!

• Part 1 (the so called m1 anomalies): solved; issue descriptions fixed in accordance w. T277551#7137941
• Note: changes will not be visible on the dashboard before the full system update is completed;
• Now running a manual update for the so called m2 class of anomalies and fixing issue descriptions on the fly;
• And then the m3 update remains to be done before the changes take effect.

Current status:

@amy_rc The suggestions given in T277551#7137941 imply a full system update.
I am currently implementing the changes and running the update on the fly to speed up the process.
Necessarily changes in the codebase will happen once the update is completed.
Reporting back here as soon as the new issue descriptions are available from the Curious Facts dashboard.

@amy_rc Thank you. Shall we close this ticket then?

@amy_rc No problem, I am on it.

@Jan_Dittrich Happening now:

Yes. @Lydia_Pintscher and I tested it after reading T277551#7069216. We then made the corresponding notes.

@amy_rc Fixed; please take a look at https://wikidata-analytics.wmcloud.org/app/CuriousFacts

@Lydia_Pintscher Please check it out now: https://wikidata-analytics.wmcloud.org/app/WikidataAnalytics

It should not be too difficult to have an RStudio Shiny app developed for this.

• I need to re-adjust the regular expression for editor reactivations as suggested by Adam in T282563#7110253 now

Q. My first assumption is that all data on the WMDE banner impressions will be found in the wmf.webrequest table where uri_path = "/beacon/impression". Please correct me if I am wrong.

If the zero reactivations category includes anyone for whom the 1+0+1+ regex doesn't match, doesn't this also include active editors who simply have a history like, 000111..., and who are still active?

Bravo... Will take a look at it and correct the analysis if necessary. Thanks again!

Before we proceed with this, please take a look at our WDCM Sitelinks Dashboard: