Page MenuHomePhabricator

GoranSMilovanovic (GoranSM)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Mar 20 2017, 3:58 PM (147 w, 4 d)
Availability
Available
LDAP User
GoranSMilovanovic
MediaWiki User
GoranSMilovanovic [ Global Accounts ]

Recent Activity

Today

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

@Janina_Ottma_WMDE The 2020/01/16 update is complete; two new user registrations.

Fri, Jan 17, 12:30 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

@Janina_Ottma_WMDE Running the 2020/01/16 update now.

Fri, Jan 17, 10:53 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

How are these edits split by edit class (as in final report 3.1.)? Could you add this information?

Fri, Jan 17, 10:48 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

Also if you say the numbers in the spreadsheet are the ones to trust, we would need the timeframe from October 28th onwards because the newsletter was send out on October 28. Could you add them to the table?

Fri, Jan 17, 10:47 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

So just to be sure: these are all page views of Wikipedia_vor_Ort in November from the tag WMDE_neweditors_autumn_2019_nl_lp1?

Fri, Jan 17, 10:42 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)

Yesterday

GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

In reference to: T235839#5809630

Thu, Jan 16, 8:36 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

In reference to: T235839#5809565

Thu, Jan 16, 8:04 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE I will try to respond to T235839#5809565 and T235839#5809630 tonight, but I can do it only later tonight.
I will email you if I make it just to make sure you get the data in the morning at least.

Thu, Jan 16, 6:29 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE And here's the November 2019 pageviews data for the WMDE_neweditors_autumn_2019_nl_lp1 newsletter banner:

Thu, Jan 16, 6:11 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE We have the October 2019 data (or what is left of them because of the wmf.webrequest purge after 90 days):

Thu, Jan 16, 4:16 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE I think I will be able to deliver the pageviews until this evening, and probably even earlier, say around 18:00 CET.

Thu, Jan 16, 2:34 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE The pageviews thing will take a while, sorry - it's just that the code needs to search through a lot of data and was hitting heavy against the cluster resources, so I had to switch it to search day by day through October and November 2019. I will be reporting back as soon as I have something.

Thu, Jan 16, 2:05 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE In order to compare the results, we will pick up everything on:

Thu, Jan 16, 12:18 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

Page Views Wikipedia_vor_Ort and LerneWikipedia: in the page view tool of wikipages we have quite some different numbers (see links), which of course differ a bit but on some days are far higher than the page views you found in the database. Can you check again, if the numbers in the report are correct?

Thu, Jan 16, 12:01 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE As of the following:

Thu, Jan 16, 11:26 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

In an early comment in the phab ticket you mentioned some user registrations happening before the start of the banner campaign (28. - 31.10.19) via the newsletter tags (WMDE_neweditors_autumn_2019_nl_lp2 and WMDE_neweditors_autumn_2019_nl_lp1). In the final report those registrations do not show up again, maybe because the time span of the report starts at the 1st of November? Could you have a look at it again and if you can confirm those registrations update the final report and the spreadsheet if necessary? The campaign time frame starts not only with the banner on Nov 1st but with the Flyers on October 7th (see dates in the graphic in the tracking doc). Could you double check again, that we have all the figures from this whole time span included in the report? This is also relevant for the page views coming from flyers for example. Thank you!

Thu, Jan 16, 11:03 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

@Christine_Domgoergen_WMDE The ticket is re-opened in respect to the following requests:

Thu, Jan 16, 9:50 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic reopened T235839: Create daily tracking reports for campaign as "Open".
Thu, Jan 16, 9:49 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

@Janina_Ottma_WMDE The 2020/01/14 and 2020/01/15 updates are ready, no new user registrations.

Thu, Jan 16, 9:48 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Tue, Jan 14

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

@Janina_Ottma_WMDE The 2020/01/13 update is ready, no new user registrations.

Tue, Jan 14, 12:42 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T242631: Get user/editcount data to determine count at percentiles.

The only thing that I do not understand here is the following planned column:

Tue, Jan 14, 11:26 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata
GoranSMilovanovic moved T240351: Daily Reporting: New Editors Thank You campaign 2019/2020 from Incoming to Prioritized on the User-GoranSMilovanovic board.
Tue, Jan 14, 11:10 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic moved T242631: Get user/editcount data to determine count at percentiles from Technical Wishlist to Incoming on the User-GoranSMilovanovic board.
Tue, Jan 14, 11:10 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata
GoranSMilovanovic added a project to T242631: Get user/editcount data to determine count at percentiles: User-GoranSMilovanovic.
Tue, Jan 14, 11:10 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata

Mon, Jan 13

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

@Janina_Ottma_WMDE @tmletzko Thank you!

Mon, Jan 13, 10:39 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Sun, Jan 12

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

@Janina_Ottma_WMDE 2020/01/11 update is ready.

Sun, Jan 12, 10:36 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Sat, Jan 11

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

Update for 2020/01/10 is in the Spreadsheet, one fresh user registration on January 10.
@Janina_Ottma_WMDE Please let me know until when do we run this campaign. Thank you!

Sat, Jan 11, 5:28 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Fri, Jan 10

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

Updates for 2020/01/07 and 2020/01/08 included, no new user registrations since January 5.
@Janina_Ottma_WMDE When does this campaign end?

Fri, Jan 10, 2:07 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Wed, Jan 8

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

Updates for 2020/01/05, 2020/01/06, and 2020/01/07 are now included.

Wed, Jan 8, 11:43 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Mon, Jan 6

GoranSMilovanovic moved T241416: WikiDaheim 2019 banner data for academic use from Radar to Incoming on the User-GoranSMilovanovic board.
Mon, Jan 6, 1:03 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering

Sun, Jan 5

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

Update for 2020/01/04 is ready.
No new user registrations on 4. January.

Sun, Jan 5, 3:53 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Sat, Jan 4

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

Update for 2020/01/03 is ready.
No new user registrations on 3. January.

Sat, Jan 4, 2:47 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Fri, Jan 3

GoranSMilovanovic added a comment to T217994: WDCM Dashboards Maintenance.

Wikidata Pageviews per Namespace Dashboard is finally operational again.

Fri, Jan 3, 4:24 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

Update for 2020/01/02 is ready.

Fri, Jan 3, 10:19 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Thu, Jan 2

GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

The Daily Reporting Spreadsheet for this campaign is ready.

Thu, Jan 2, 9:08 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Tue, Dec 31

GoranSMilovanovic added a comment to T240361: Create Tracking Report: New Editors Thank You Campaign 2019/2020.

@kai.nissen There were six (6) user registrations from the following banners on 2019/12/30:

Tue, Dec 31, 11:28 AM · WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T217994: WDCM Dashboards Maintenance.

Wikidata Pageviews per Namespace Dashboard not responding following the changes in T239199 (Kerberos Auth for all WMDE Analytics):

Tue, Dec 31, 11:01 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic

Tue, Dec 24

GoranSMilovanovic moved T241416: WikiDaheim 2019 banner data for academic use from Technical Wishlist to Radar on the User-GoranSMilovanovic board.
Tue, Dec 24, 1:40 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added projects to T241416: WikiDaheim 2019 banner data for academic use: WMDE-Analytics-Engineering, User-GoranSMilovanovic.
Tue, Dec 24, 1:39 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added a comment to T241416: WikiDaheim 2019 banner data for academic use.

@WMDE-leszek Please consider this ticket and help me prioritize. Thanks.
@A00604591 Please let me know if there is a specific time frame for this.

Tue, Dec 24, 1:39 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added a comment to T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.

The analytics code for this campaign is in place.

Tue, Dec 24, 10:13 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Fri, Dec 20

GoranSMilovanovic added a comment to T240361: Create Tracking Report: New Editors Thank You Campaign 2019/2020.

@Janina_Ottma_WMDE To put it in a nutshell: we can begin testing on Monday, December 23.

Fri, Dec 20, 12:27 PM · WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Dec 18 2019

GoranSMilovanovic added a comment to T240466: Measure the impact of Tainted References Wikidata feature.

@Addshore @Jan_Dittrich Here is the summary of the approach to collect the baseline data, following our today's meeting:

Dec 18 2019, 12:12 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata Tainted References, Wikidata

Dec 17 2019

GoranSMilovanovic added a comment to T240361: Create Tracking Report: New Editors Thank You Campaign 2019/2020.

@kai.nissen And you're finally back - thank you very much for the clarifications in T240361#5744403 and welcome!
@Janina_Ottma_WMDE More or less I think I get the tracking of the campaign right, let's discuss the details in our forthcoming call.

Dec 17 2019, 9:40 AM · WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T239199: Switch all WDCM/Wikidata Analytics ETL to new Kerberos Auth on stat100**.
  • WD_percentUsage_PRODUCTION.R is back on crontab on stat1004;
  • monitoring; next steps:
  • WD_PageviewsPerType_Engine.R on stat1007;
  • WDCM (T)itles and WDCM (S)itelinks - HiveQL;
  • all WDCM Pyspark ETL.
Dec 17 2019, 9:36 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic

Dec 16 2019

GoranSMilovanovic added a comment to T240361: Create Tracking Report: New Editors Thank You Campaign 2019/2020.

@Addshore No worries you've been pinged only because you were involved in this banner actions thing in the previous campaign.
Ping: @Tim_WMDE @awight Please, gentlemen - we need to see what exactly about banners we can learn and from what schema. Thank you.

Dec 16 2019, 12:23 PM · WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T239199: Switch all WDCM/Wikidata Analytics ETL to new Kerberos Auth on stat100**.

Temporarily not on crontab:

Dec 16 2019, 11:21 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic updated the task description for T239199: Switch all WDCM/Wikidata Analytics ETL to new Kerberos Auth on stat100**.
Dec 16 2019, 11:18 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic renamed T239199: Switch all WDCM/Wikidata Analytics ETL to new Kerberos Auth on stat100** from Switch all WDCM ETL to new Kerberos Auth on stat100** to Switch all WDCM/Wikidata Analytics ETL to new Kerberos Auth on stat100**.
Dec 16 2019, 11:18 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic closed T238672: Create tracking report for follow-up mailing to campaign as Resolved.
Dec 16 2019, 9:03 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic closed T235839: Create daily tracking reports for campaign as Resolved.
Dec 16 2019, 9:03 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)

Dec 15 2019

GoranSMilovanovic added a comment to T240466: Measure the impact of Tainted References Wikidata feature.

@Addshore Well, now it sounds even more complicated than in the ticket description.

Dec 15 2019, 6:33 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata Tainted References, Wikidata
GoranSMilovanovic updated subscribers of T240361: Create Tracking Report: New Editors Thank You Campaign 2019/2020.

@Janina_Ottma_WMDE At this point, your tracking definitions are completely unclear.

Dec 15 2019, 6:21 PM · WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic moved T234161: WD Data Quality: compare quality vs usage on commons vs everything else from Prioritized to Current/Deprioritized on the User-GoranSMilovanovic board.
Dec 15 2019, 1:39 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata
GoranSMilovanovic added a comment to T234161: WD Data Quality: compare quality vs usage on commons vs everything else.

@Lydia_Pintscher Finally, a "non-Commons" data quality report is ready: Wikidata Quality Report - Commons Excluded.
This one encompasses only items that are reused anywhere except in Wikimedia Commons.
Let me know if anything else needs to be done here. N.B. Statistical hypothesis testing comparing various subsets (e.g. Commons vs. Everything else) are possible.

Dec 15 2019, 1:39 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata
GoranSMilovanovic added a comment to T234161: WD Data Quality: compare quality vs usage on commons vs everything else.

@Lydia_Pintscher We now have a separate WD Quality Report for Wikimedia Commons.
Working on its complimentary, "non-Commons", quality assessment now.

Dec 15 2019, 11:31 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata

Dec 13 2019

GoranSMilovanovic moved T239194: WDCM Analytics Portal from WDCM to Prioritized on the User-GoranSMilovanovic board.
Dec 13 2019, 3:17 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic closed T239197: WDCM Usage: Deprecate Similarity Graph, a subtask of T217994: WDCM Dashboards Maintenance, as Resolved.
Dec 13 2019, 3:14 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic closed T239197: WDCM Usage: Deprecate Similarity Graph as Resolved.

Done.

Dec 13 2019, 3:14 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic moved T240466: Measure the impact of Tainted References Wikidata feature from Incoming to Prioritized on the User-GoranSMilovanovic board.
Dec 13 2019, 3:11 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata Tainted References, Wikidata
GoranSMilovanovic added a comment to T240466: Measure the impact of Tainted References Wikidata feature.

My initial observations (continued) - please comment:

Dec 13 2019, 2:46 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata Tainted References, Wikidata
GoranSMilovanovic added a comment to T240466: Measure the impact of Tainted References Wikidata feature.

My initial observations - please comment:

Dec 13 2019, 2:12 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata Tainted References, Wikidata
GoranSMilovanovic moved T239199: Switch all WDCM/Wikidata Analytics ETL to new Kerberos Auth on stat100** from Incoming to Prioritized on the User-GoranSMilovanovic board.
Dec 13 2019, 1:30 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic closed T239196: WDCM Semantic and Geo dashboards do not respond, a subtask of T217994: WDCM Dashboards Maintenance, as Resolved.
Dec 13 2019, 1:25 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic closed T239196: WDCM Semantic and Geo dashboards do not respond as Resolved.
  • Both dashboards repaired;
  • The problem, however, is not so obvious:
    • some data set URLs where not properly url encoded, yet
    • the dashboard was responsive in spite of that fact just until recently.
Dec 13 2019, 1:25 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic added a comment to T238672: Create tracking report for follow-up mailing to campaign.

Do we need any additional work on this or shall we close the ticket? Thanks.

Dec 13 2019, 12:59 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

Once again, please let me know if you need anything else on this campaign or we can close this ticket. Thank you.

Dec 13 2019, 12:58 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)

Dec 12 2019

GoranSMilovanovic moved T239393: Public data set review for T237728 from Technical Wishlist to Current/Deprioritized on the User-GoranSMilovanovic board.
Dec 12 2019, 1:49 PM · Privacy, Analytics, WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic moved T240466: Measure the impact of Tainted References Wikidata feature from Technical Wishlist to Incoming on the User-GoranSMilovanovic board.
Dec 12 2019, 1:49 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata Tainted References, Wikidata
GoranSMilovanovic claimed T240466: Measure the impact of Tainted References Wikidata feature.
Dec 12 2019, 1:48 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata Tainted References, Wikidata
GoranSMilovanovic added a comment to T237728: Featured Page Revision History: Support Phd Researcher in Residence WMDE .

in relation to your question on can we learn what external pages point to some pages in the Wikimedia universe (e.g. the target page of your research project - https://en.wikipedia.org/wiki/Hurricane_Hazel), it seems that the WMF has access to the Google Console service:

Dec 12 2019, 1:48 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic updated the task description for T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.
Dec 12 2019, 1:32 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T240361: Create Tracking Report: New Editors Thank You Campaign 2019/2020.

this is my first time working with Phabricator, so sorry in advance for any future mistakes - I am still learning

No worries, I will help in the process.

Dec 12 2019, 1:28 PM · WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic closed T237013: Refactor the Wikidata Data Quality Report analytics procedures as Resolved.

@Lydia_Pintscher The WD Data Quality Report is now updated.

Dec 12 2019, 1:22 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering

Dec 11 2019

GoranSMilovanovic added a comment to T240361: Create Tracking Report: New Editors Thank You Campaign 2019/2020.

@Janina_Ottma_WMDE If and when you update the tracking info for this campaign, please do not forget to update the description of T240351. Otherwise we might run into confusion. Thank you.

Dec 11 2019, 12:22 PM · WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic updated the task description for T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.
Dec 11 2019, 12:21 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic updated the task description for T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.
Dec 11 2019, 12:21 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)

Dec 10 2019

GoranSMilovanovic moved T240351: Daily Reporting: New Editors Thank You campaign 2019/2020 from Technical Wishlist to Incoming on the User-GoranSMilovanovic board.
Dec 10 2019, 10:47 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic renamed T240351: Daily Reporting: New Editors Thank You campaign 2019/2020 from Campaign report to Daily Reporting: New Editors Thank You campaign 2019/2020.
Dec 10 2019, 10:47 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic claimed T240351: Daily Reporting: New Editors Thank You campaign 2019/2020.
Dec 10 2019, 10:44 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (WMDE New Editor Thanks Campaign 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

Please let me know if anything else is needed here or we can close this ticket.

Dec 10 2019, 10:40 PM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic moved T238672: Create tracking report for follow-up mailing to campaign from Prioritized to New Editors/Campaigns on the User-GoranSMilovanovic board.
Dec 10 2019, 10:40 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T238672: Create tracking report for follow-up mailing to campaign.

Upon another week of observation (2019/12/01 - 2019/12/07) we find no new data for this follow-up.

Dec 10 2019, 10:39 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)

Dec 5 2019

GoranSMilovanovic updated subscribers of T237013: Refactor the Wikidata Data Quality Report analytics procedures.

Since @JAllemandou has kindly provided a fresh copy of the WD JSON dump in hdfs (T209655#5713452), our next WD Quality Report update will be constrained only by the timestamp of the most recent snapshot of the mediawiki_history available to us.

Dec 5 2019, 10:49 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added a comment to T209655: Copy Wikidata dumps to HDFS.

@JAllemandou Thank you - as ever!

Dec 5 2019, 10:42 AM · Research-Backlog, Wikidata, Analytics
GoranSMilovanovic added a comment to T237013: Refactor the Wikidata Data Quality Report analytics procedures.

I care about the storage issues. I dropped unneeded lines and freed 70GB in stat1007.

Dec 5 2019, 10:37 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering

Dec 4 2019

GoranSMilovanovic moved T234161: WD Data Quality: compare quality vs usage on commons vs everything else from Incoming to Prioritized on the User-GoranSMilovanovic board.
Dec 4 2019, 2:43 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, Wikidata
GoranSMilovanovic added a comment to T237013: Refactor the Wikidata Data Quality Report analytics procedures.

@Ladsgroup As of me, you don't need to keep the following ORES updates on stat1007 anymore:

Dec 4 2019, 2:40 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added a comment to T209655: Copy Wikidata dumps to HDFS.

@JAllemandou Do you think it would be possible to produce a new version of this data set?
The latest update seems to be: 2019-10-03 09:29 /user/joal/wmf/data/wmf/mediawiki/wikidata_parquet/20190902 - which you have pointed me at in T209655#5543575.
I would need to update the Wikidata Quality Report soon (Dec 15, say), and the code relies on Spark to process the dump. Thanks.

Dec 4 2019, 2:27 AM · Research-Backlog, Wikidata, Analytics

Dec 3 2019

GoranSMilovanovic added a comment to T237013: Refactor the Wikidata Data Quality Report analytics procedures.

I have started a gradual update procedure (fromt the initial 111Gb run.out -> run_20190901.out -> run_201910.out -> run_201911.out).

Dec 3 2019, 2:00 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added a comment to T239196: WDCM Semantic and Geo dashboards do not respond.
  • Data sets are now complete following a re-run of the wdcmModule_Orchestra.R;
  • Next steps: figure out {curl} (possibly) related problems on WDCM Semantics and WDCM Geo.
Dec 3 2019, 12:30 PM · WMDE-Analytics-Engineering, User-GoranSMilovanovic

Dec 2 2019

GoranSMilovanovic added a comment to T237013: Refactor the Wikidata Data Quality Report analytics procedures.

@Ladsgroup Got it.

Dec 2 2019, 11:43 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added a comment to T237013: Refactor the Wikidata Data Quality Report analytics procedures.

@Ladsgroup Pong. I am on it as soon as I figure out what is wrong with this thing in WDCM: T239196.

Dec 2 2019, 12:24 PM · User-GoranSMilovanovic, WMDE-Analytics-Engineering
GoranSMilovanovic added a comment to T239196: WDCM Semantic and Geo dashboards do not respond.
  • WDCM_Sqoop_Clients.R unexpectedly took ~13h to update;
  • wdcmModule_Orchestra.R was thus run (on 10:00 UTC) out of sync with the wdcm_clients_wb_entity_usage table;
  • and this is the possible cause of the observed missing data.
Dec 2 2019, 11:18 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic added a comment to T239196: WDCM Semantic and Geo dashboards do not respond.
  • Moreover, dewiki and potentially other wikies are not found in the 2019-12-01 11:44 update at all;
  • However, dewiki was sqooped by WDCM_Sqoop_Clients.R from stat1004:
Dec 2 2019, 10:59 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic added a comment to T239196: WDCM Semantic and Geo dashboards do not respond.

WDCM public data set:

  • wdcm_project.csv
  • Update timestamp: 2019-12-01 11:44

has no data on dewiki and many other.
Inspecting now.

Dec 2 2019, 10:50 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic triaged T239196: WDCM Semantic and Geo dashboards do not respond as High priority.
  • This is more serious than what it seemed to be following my initial assessments.
  • Possible cause: change in R packages used on the dashboard, possibly {curl}.
  • Inspecting now.
Dec 2 2019, 10:38 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic
GoranSMilovanovic added a comment to T238672: Create tracking report for follow-up mailing to campaign.

The new Follow up tab in the Email Campaign Spreadsheet has the pageviews for this follow-up.

Dec 2 2019, 9:34 AM · User-GoranSMilovanovic, WMDE-Analytics-Engineering, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T235839: Create daily tracking reports for campaign.

As of T235839#5691491: my bad, the data were sparse and I simply didn't spot the tags, they are in place with https://de.wikipedia.org/wiki/Wikipedia:Oberlausitz/Wikipedia_vor_Ort_2019 as in any other campaign page - check the campaign spreadsheet please.

Dec 2 2019, 8:03 AM · Core Platform Team Workboards (Clinic Duty Team), WMDE-Analytics-Engineering, User-GoranSMilovanovic, WMDE-New-Editors-Banner-Campaigns (Banner Campaign Autumn 2019)
GoranSMilovanovic added a comment to T237728: Featured Page Revision History: Support Phd Researcher in Residence WMDE .

While we wait for the public data review in T239393 to complete, here are the additional data sets.

Dec 2 2019, 7:48 AM · WMDE-Analytics-Engineering, User-GoranSMilovanovic