Page MenuHomePhabricator

Milimetric (Dan Andreescu)
User

Projects (12)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Oct 8 2014, 5:48 PM (270 w, 1 d)
Availability
Available
LDAP User
Milimetric
MediaWiki User
Milimetric (WMF) [ Global Accounts ]

Recent Activity

Yesterday

Milimetric moved T240049: Allow sorting data in the Tabular View by multiple columns from Next Up to In Progress on the Analytics-Kanban board.
Thu, Dec 12, 9:39 PM · Analytics-Kanban, Analytics-Dashiki, Analytics
Milimetric added a comment to T236586: "dashiki" Cloud VPS project jessie deprecation.

This is done, updated docs and deployment code, deleting the instances now.

Thu, Dec 12, 9:37 PM · Patch-For-Review, Analytics-Kanban, Analytics, Cloud-VPS (Debian Jessie Deprecation)
Milimetric added a comment to T236941: Dashiki: Read multiple wikis from single file.

@srishakatux: just a ping that this is done. I still want to update the Dashiki docs which are in a very sad state, but before I get to that. To use the feature, you have to change your script to output a single file, named {submetric}.{format} (in your case maybe something like edit_percent.tsv in the reports/metrics/wmcs/ folder). The format of the file should be like this:

Thu, Dec 12, 9:35 PM · Analytics-Kanban, Analytics
Milimetric moved T236941: Dashiki: Read multiple wikis from single file from Ready to Deploy to Done on the Analytics-Kanban board.
Thu, Dec 12, 9:28 PM · Analytics-Kanban, Analytics
Milimetric moved T236586: "dashiki" Cloud VPS project jessie deprecation from In Progress to Done on the Analytics-Kanban board.
Thu, Dec 12, 9:26 PM · Patch-For-Review, Analytics-Kanban, Analytics, Cloud-VPS (Debian Jessie Deprecation)
Milimetric moved T239625: Improve quality of external referer data from In Progress to In Code Review on the Analytics-Kanban board.
Thu, Dec 12, 5:15 PM · Patch-For-Review, Analytics-Kanban, Research, Analytics
Milimetric moved T239565: Create reportupdater reports that execute SDC requests from In Progress to In Code Review on the Analytics-Kanban board.
Thu, Dec 12, 5:10 PM · Patch-For-Review, Analytics-Kanban, Product-Analytics, SDC General, Wikidata, Analytics
JAllemandou awarded T240413: Join slot, content, revision, and page once on load a Like token.
Thu, Dec 12, 8:11 AM · Analytics
Milimetric added a comment to T220524: Article recommender: retrieve top 50 wikipedias automatically.

@Iflorez: sorry for the confusion, edited-pages is just the root of our pages-related metrics on the api. So in this case, I pointed to the edited-pages/new metric which gives you pages created stats. The following parameters are {wiki}/{editor type}/{page type}/{granularity}/{start}/{end}. You can see that the "Edited Pages" metric is a different edited-pages/aggregate path: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats_2#Edited_Pages_2

Thu, Dec 12, 3:06 AM · Article-Recommendation
Milimetric added a comment to T239565: Create reportupdater reports that execute SDC requests .

Ok, seems like some of this confusion is getting cleared up. For my part, here's what I'm planning to do next:

Thu, Dec 12, 2:57 AM · Patch-For-Review, Analytics-Kanban, Product-Analytics, SDC General, Wikidata, Analytics

Wed, Dec 11

Milimetric updated subscribers of T239571: Check home leftovers of dfoy.

I tracked down the other zero files I saved for @DFoy:

Wed, Dec 11, 5:28 PM · Analytics
Milimetric moved T236941: Dashiki: Read multiple wikis from single file from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Wed, Dec 11, 5:03 PM · Analytics-Kanban, Analytics
Milimetric moved T225578: EventLogging needs to enque events to avoid draining users' battery on mobile from In Code Review to Paused on the Analytics-Kanban board.
Wed, Dec 11, 5:03 PM · Performance-Team, Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, Analytics
Milimetric added a comment to T238878: Data about how many file pages on Commons contain at least one structured data element .

I translated T238878#5708511 to Hive to familiarize myself with it and get ahead of productionizing it. I got similar numbers first as a sanity check and then grouped the numbers by the month of the page_latest revision's timestamp. I was wondering if the numbers increase in some nice way that we could report on regardless of the overall total. There's a fairly clear trend towards more structured data. And this is not a clear way to show the trend because we're just looking at the latest revision not all revisions, but maybe it's useful to @Abit as she thinks about this metric:

Wed, Dec 11, 2:40 AM · Product-Analytics, SDC General, Analytics, Wikidata
Milimetric created T240413: Join slot, content, revision, and page once on load.
Wed, Dec 11, 2:14 AM · Analytics

Mon, Dec 9

Milimetric added a comment to T190700: Automate creation of sqoop list of wikis to import data for from sitematrix.

Tasked this briefly, want to coordinate with T239589 and make T239136 not necessary. To that end,

Mon, Dec 9, 5:52 PM · Analytics, Analytics-Wikistats

Fri, Dec 6

Milimetric added a comment to T236586: "dashiki" Cloud VPS project jessie deprecation.

I'll take care of this either tomorrow or early next week. I was going to do it tonight but it's rejecting my ssh connect, probably either have to wait for puppet or something else is wrong.

Fri, Dec 6, 5:16 AM · Patch-For-Review, Analytics-Kanban, Analytics, Cloud-VPS (Debian Jessie Deprecation)
Milimetric moved T236586: "dashiki" Cloud VPS project jessie deprecation from Next Up to In Progress on the Analytics-Kanban board.
Fri, Dec 6, 5:01 AM · Patch-For-Review, Analytics-Kanban, Analytics, Cloud-VPS (Debian Jessie Deprecation)
Milimetric moved T236586: "dashiki" Cloud VPS project jessie deprecation from Incoming to Ops Week on the Analytics board.
Fri, Dec 6, 5:01 AM · Patch-For-Review, Analytics-Kanban, Analytics, Cloud-VPS (Debian Jessie Deprecation)
Milimetric added projects to T236586: "dashiki" Cloud VPS project jessie deprecation: Analytics, Analytics-Kanban.
Fri, Dec 6, 5:00 AM · Patch-For-Review, Analytics-Kanban, Analytics, Cloud-VPS (Debian Jessie Deprecation)
Milimetric added a comment to T239685: Analytics: Some pages/page requests are not reflected in statistics.

I'm not exactly sure what's going on but I only see the piwik script on the root of the site. Looking at the piwik report for the site, 99% of requests it tracked are indeed to /index. I see the piwik.js code referenced from the other pages, but it doesn't look like the tracker code is present. I'm not at all familiar with how you instrumented this, just looking at it briefly seems like matomo isn't tracking subpages.

Fri, Dec 6, 4:42 AM · Wikimedia Design Style Guide, Analytics

Thu, Dec 5

Milimetric moved T239217: Degraded RAID on dbstore1003 from Incoming to Operational Excellence on the Analytics board.
Thu, Dec 5, 6:29 PM · Analytics, ops-eqiad, Operations
Milimetric assigned T239672: Many special pages missing from pageview_hourly dataset starting on July 23, 2019 to Nuria.
Thu, Dec 5, 6:29 PM · Analytics, Product-Analytics
Milimetric moved T239672: Many special pages missing from pageview_hourly dataset starting on July 23, 2019 from Incoming to Data Quality on the Analytics board.
Thu, Dec 5, 6:28 PM · Analytics, Product-Analytics
Milimetric moved T239127: Import slots/slots_roles and wikibase.wbc_entity_usage through scoop from In Code Review to Done on the Analytics-Kanban board.
Thu, Dec 5, 6:17 PM · Analytics-Kanban, Analytics

Wed, Dec 4

Milimetric moved T238106: "Wikidata Query Service Updater" should have 'bot' in the user agent to indicate is a tool from Ready to Deploy to Done on the Analytics-Kanban board.
Wed, Dec 4, 10:39 PM · Analytics-Kanban, Patch-For-Review, Discovery-Search (Current work)
Milimetric moved T239848: Delay cassandra mediarequest-per-file daily job one hour so that it doesn't colide with pageview-per-article from Ready to Deploy to Done on the Analytics-Kanban board.
Wed, Dec 4, 10:38 PM · Analytics-Kanban, Analytics
Milimetric moved T238106: "Wikidata Query Service Updater" should have 'bot' in the user agent to indicate is a tool from Next Up to Ready to Deploy on the Analytics-Kanban board.
Wed, Dec 4, 8:17 PM · Analytics-Kanban, Patch-For-Review, Discovery-Search (Current work)
Milimetric moved T239848: Delay cassandra mediarequest-per-file daily job one hour so that it doesn't colide with pageview-per-article from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Wed, Dec 4, 8:16 PM · Analytics-Kanban, Analytics
Milimetric moved T118517: [RFC] Use <figure> for media from Implemented to In progress on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 6:49 PM · TechCom-RFC (TechCom-Approved), Accessibility, Parsing-Team, Wikipedia-Android-App-Backlog, MediaWiki-Parser
Milimetric moved T200297: Review Jade data storage and architecture proposal [RFC] from Implemented to In progress on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 6:48 PM · TechCom-RFC (TechCom-Approved), MW-1.33-notes (1.33.0-wmf.14; 2019-01-22), Patch-For-Review, Scoring-platform-team (Current), DBA, Operations, Jade
Milimetric moved T589: RFC: image and oldimage tables from Implemented to Backlog on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 6:45 PM · Wikimedia-Rdbms, TechCom-RFC (TechCom-Approved)
Milimetric moved T220056: MediaWiki database policy and/or guidelines (2019) from Backlog to Last Call on the TechCom-RFC board.

Moving to last call, @Krinkle or @Nikerabbit to incorporate comments from T220056#5644705.

Wed, Dec 4, 6:35 PM · DBA, TechCom-RFC
Milimetric moved T237618: Amendments to the Gerrit Privilege policy from Inbox to In progress on the TechCom board.

@tstarling we wanted to assign this to you in today's techcom meeting, but not in absentia, so just pinging you. Are you ok with driving this forward? We only had four people today but we generally agreed with you and Gergo that forking when maintainers leave is a bad idea.

Wed, Dec 4, 6:33 PM · TechCom
Milimetric moved T589: RFC: image and oldimage tables from Backlog to Implemented on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 4:30 PM · Wikimedia-Rdbms, TechCom-RFC (TechCom-Approved)
Milimetric moved T190063: Tracking dependencies for multiple Content objects per page (MCR) from In progress to Implemented on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 4:30 PM · Multi-Content-Revisions (MCR-SDC Statement Support - phase 3), Epic, TechCom-RFC (TechCom-Approved), User-Daniel
Milimetric moved T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps from In progress to Implemented on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 4:29 PM · Structured-Data-Backlog, CPT Initiatives (MCR), Multi-Content-Revisions, Multimedia, Core Platform Team Workboards (Done with CPT), TechCom-RFC (TechCom-Approved), Structured Data Engineering, Dumps-Generation, User-ArielGlenn, User-Daniel, Wikidata
Milimetric moved T200297: Review Jade data storage and architecture proposal [RFC] from In progress to Implemented on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 4:29 PM · TechCom-RFC (TechCom-Approved), MW-1.33-notes (1.33.0-wmf.14; 2019-01-22), Patch-For-Review, Scoring-platform-team (Current), DBA, Operations, Jade
Milimetric moved T118517: [RFC] Use <figure> for media from In progress to Implemented on the TechCom-RFC (TechCom-Approved) board.
Wed, Dec 4, 4:29 PM · TechCom-RFC (TechCom-Approved), Accessibility, Parsing-Team, Wikipedia-Android-App-Backlog, MediaWiki-Parser

Mon, Dec 2

Milimetric moved T239565: Create reportupdater reports that execute SDC requests from Next Up to In Progress on the Analytics-Kanban board.
Mon, Dec 2, 8:39 PM · Patch-For-Review, Analytics-Kanban, Product-Analytics, SDC General, Wikidata, Analytics
Milimetric added a comment to T239565: Create reportupdater reports that execute SDC requests .

Yay, I get to work with @mpopov :) Ok, questions:

Mon, Dec 2, 8:39 PM · Patch-For-Review, Analytics-Kanban, Product-Analytics, SDC General, Wikidata, Analytics
Milimetric added a comment to T238878: Data about how many file pages on Commons contain at least one structured data element .

I noticed that three folks used rev_deleted = 0 to mean "revision not deleted", but this field means something completely different (https://www.mediawiki.org/wiki/Manual:Revision_table#rev_deleted). Basically, it's about parts of the revision that have been suppressed from different types of users in the interface (like hiding revision comments). Good news is that revisions in the revision table are by definition not deleted. When a page is deleted, its revisions all get inserted into the archive table and removed from the revision table. So when you did rev_deleted = 0 you were just filtering out revisions that had any kind of suppression applied. I think that's a small number and didn't seem to add to the confusion above, but should be included just in case.

Mon, Dec 2, 8:32 PM · Product-Analytics, SDC General, Analytics, Wikidata
Milimetric added a comment to T238360: Hourly Feature extraction for bot detection from webrequest .

doc looks great, copy-edited a bit as I went through it

Mon, Dec 2, 7:38 PM · Patch-For-Review, Analytics-Kanban, Analytics

Wed, Nov 20

Milimetric added a comment to T213505: RfC: OpenGraph descriptions in wiki pages.

We looked at this in tech com and honestly we've lost track of where this is at. Is it still being discussed? Is there a preference for a way forward and resourcing and this is waiting for us? We can help set up discussions and move this forward, but we're not sure which way forward is.

Wed, Nov 20, 10:53 PM · Core Platform Team, Product-Infrastructure-Team-Backlog, MediaWiki-General, TechCom-RFC
Milimetric added a comment to T206789: Modern Event Platform: Schema Registry: Implementation.

I'm also liking the idea of experimental, and I think experiment is actually really nice and concise. It is fun and easy and kind of the same thing as instrument but less specific so it stays more flexible.

Wed, Nov 20, 8:42 PM · Analytics-Kanban, CPT Initiatives (Modern Event Platform (TEC2)), Services (watching), Analytics-EventLogging, Event-Platform, Analytics
Milimetric moved T236941: Dashiki: Read multiple wikis from single file from In Progress to In Code Review on the Analytics-Kanban board.
Wed, Nov 20, 11:15 AM · Analytics-Kanban, Analytics

Tue, Nov 19

Milimetric created T238668: Output schema with mediawiki_history snapshots dumps.
Tue, Nov 19, 4:16 PM · Analytics
Milimetric added a comment to T235189: Prototype client to log errors in vagrant.

I started out thinking Sentry was the gold standard and I now prefer the way you're going about it, Jason. Because even if we want all those weird corner cases it makes much more sense for us to build up to them as we need them than to not understand exactly what we're seeing and why. Also love that this is much easier to maintain and get through code review, and would bring value to core.

Tue, Nov 19, 4:09 PM · Patch-For-Review, Product-Infrastructure-Team-Backlog (Kanban), Performance-Team (Radar), Better Use Of Data, Epic, Analytics
Milimetric added a comment to T235189: Prototype client to log errors in vagrant.

Will comment on architecture next, but just jotting down thoughts about code as I read it:

Tue, Nov 19, 4:07 PM · Patch-For-Review, Product-Infrastructure-Team-Backlog (Kanban), Performance-Team (Radar), Better Use Of Data, Epic, Analytics
Milimetric added a comment to T237752: Make stats.wikimedia.org point to wikistats2 by default .

Sold!

Tue, Nov 19, 2:03 PM · Analytics-Kanban, Analytics
Milimetric added a comment to T237752: Make stats.wikimedia.org point to wikistats2 by default .

That’s the ugly/cute part, since we’re copying and not moving / to /v1, all the urls will work, relative or absolute. This is totally fine with v2 because that’s a client-side single page app that won’t overwrite anything except index.html

Tue, Nov 19, 11:49 AM · Analytics-Kanban, Analytics
Milimetric moved T236941: Dashiki: Read multiple wikis from single file from Next Up to In Progress on the Analytics-Kanban board.
Tue, Nov 19, 4:33 AM · Analytics-Kanban, Analytics
Milimetric moved T223931: Switch mw.user.sessionId back to session-cookie persistence from Next Up to In Progress on the Analytics-Kanban board.
Tue, Nov 19, 4:33 AM · Product-Infrastructure-Team-Backlog, Product-Analytics (Kanban), Better Use Of Data, Analytics, Performance-Team
Milimetric added a comment to T237752: Make stats.wikimedia.org point to wikistats2 by default .

@elukey - we can talk more tomorrow but this solution hides Wikistats 1's index.html, which is how most people navigated the old site. I think we should preserve it. My idea seemed simple to me, let's see what I'm missing:

Tue, Nov 19, 4:31 AM · Analytics-Kanban, Analytics
Milimetric closed T113695: Clean the code review queue of analytics/wikistats as Declined.

Wikistats 1 is no longer maintained.

Tue, Nov 19, 4:16 AM · DevRel-February-2016, Analytics, DevRel-January-2016, DevRel-December-2015, DevRel-November-2015, Analytics-Wikistats, DevRel-October-2015
Milimetric closed T153923: vet edit data on the data lake as Declined.

In the time since we made this task, the Product Analytics and Analytics Engineering teams have been working closely on this dataset. We made some quality improvements and continue to vet together. This task is therefore outdated.

Tue, Nov 19, 4:15 AM · Analytics
Milimetric closed T153923: vet edit data on the data lake , a subtask of T152035: Productionize Edit History Reconstruction and Extraction, as Declined.
Tue, Nov 19, 4:15 AM · Analytics-Kanban
Milimetric added a comment to T230136: Tune Wikistats 2 Varnish caching.

@DannyS712: could you tweak the batch to not leave a comment? Otherwise subscribers get notified

Tue, Nov 19, 4:13 AM · Analytics-Kanban, Analytics
Milimetric created T238615: Adapt mediawiki history for MCR.
Tue, Nov 19, 3:21 AM · Analytics, Multi-Content-Revisions (Tech Debt)
Milimetric added a comment to T215466: Remove revision_comment_temp and revision_actor_temp.

I'm sorry I thought I confirmed this - yes, we were blissfully ignorant on top of the cloud db views. Our other private sqoops don't use these tables either.

Tue, Nov 19, 3:16 AM · Patch-For-Review, Analytics, CPT Initiatives (Revision Storage Schema Improvements), Technical-Debt, Epic
Milimetric added a comment to T234188: Taxonomy of new user reading patterns.

Most sessions start in ns=1 (article), but for new users the percentage is slightly smaller.

Tue, Nov 19, 3:12 AM · Analytics, Research
Milimetric added a comment to T236223: Setup Config:Dashiki:WMCSEdits on meta wiki .

@srishakatux what's left to do here? Need any support updating the config.yaml for example?

Tue, Nov 19, 3:09 AM · Analytics, Developer-Advocacy (Oct-Dec 2019), Cloud-Services
Milimetric added a comment to T211881: graphoid: Code stewardship request.

I guess its not clear to me what exactly the decision to kill the server side component means. Are we replacing it with a different server side component? If so, how do we know the new component won't suffer from the same concerns that Graphoid has.

No, the idea is that we will no longer have any server-side component dependency. This decision is based on the fact that we don't have engineering capacity to support a dedicated server-side component. It's obviously not ideal, and may have various problems (as mentioned by @Yurik and @Yair_rand), but the alternative would be to retire the Graph extension entirely, which would be worse.

Tue, Nov 19, 2:52 AM · Release-Engineering-Team-TODO (201908), Release-Engineering-Team (Code Health), Core Platform Team Legacy (Watching / External), Services (watching), Operations, Code-Stewardship-Reviews, Graphoid
Milimetric added a comment to T206789: Modern Event Platform: Schema Registry: Implementation.

I'm fine with just two repos, and the way you outline using them. Two thoughts on naming:

Tue, Nov 19, 2:41 AM · Analytics-Kanban, CPT Initiatives (Modern Event Platform (TEC2)), Services (watching), Analytics-EventLogging, Event-Platform, Analytics

Mon, Nov 18

Milimetric added a comment to T235143: Wikistats API for legacy pagecounts does not have mobile data before October 2014.

Yes, definitely, his aggregated data is available there and in CSV files,

Do we know where are these files?

Mon, Nov 18, 6:45 PM · Product-Analytics, Analytics
Milimetric updated subscribers of T237752: Make stats.wikimedia.org point to wikistats2 by default .

We have archived all the old geowiki (old name for geoeditors data) data to the archive hive database, tables are:

Mon, Nov 18, 4:35 PM · Analytics-Kanban, Analytics
Milimetric added a comment to T159046: Track page views by page ID rather than title (handles moved pages).

Thanks for the context, @Pine. This issue is in our backlog, meaning it's behind all of our other priorities. It has very little chance of being picked up by itself. We do have the data now to group pageviews by page id, and to figure out all names that a particular page had over its history. So someone could pick this up at a hackathon and make a workable prototype of collecting pageviews by title or someone has to argue that this should be higher priority for our team to pick it up.

Mon, Nov 18, 3:58 PM · Pageviews-API, Analytics

Fri, Nov 15

Milimetric updated the task description for T238264: Unconference: JS Framework Experience Sharing.
Fri, Nov 15, 10:10 PM · Wikimedia-Technical-Conference-2019
Milimetric updated the task description for T238264: Unconference: JS Framework Experience Sharing.
Fri, Nov 15, 10:09 PM · Wikimedia-Technical-Conference-2019

Wed, Nov 13

Milimetric updated subscribers of T238264: Unconference: JS Framework Experience Sharing.
Wed, Nov 13, 9:02 PM · Wikimedia-Technical-Conference-2019
Milimetric created T238264: Unconference: JS Framework Experience Sharing.
Wed, Nov 13, 9:02 PM · Wikimedia-Technical-Conference-2019

Nov 6 2019

Milimetric moved T235200: Create HDFS /tmp/ cleaner from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Nov 6 2019, 8:23 PM · Analytics-Kanban, Analytics-Cluster, Analytics
Milimetric added a comment to T117279: [EPIC] Core should provide inline diffs as well as side by side (Move InlineDifferenceEngine into core / remove MobileDiff).

Just trying to channel help here, @tstarling said he could help review a couple years ago, and @Tgr knows this code. Would either of you be able to help now?

Nov 6 2019, 5:21 PM · MW-1.35-notes (1.35.0-wmf.10; 2019-12-10), User-Jdlrobson, Core Platform Team Workboards (Clinic Duty Team), Desktop Improvements, TechCom, MobileFrontend (MobileFrontend Special Pages), Multi-Content-Revisions, Readers-Web-Backlog (Tracking), Technical-Debt (RW-Tech-Debt)

Nov 5 2019

Milimetric moved T131280: Make aggregate data on editors per country per wiki publicly available from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Nov 5 2019, 2:05 PM · Product-Analytics, Analytics-Kanban
Milimetric moved T237072: Correct namespace zero editor counts on geoeditors_monthly table on hive and druid from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Nov 5 2019, 2:04 PM · Product-Analytics, Analytics, Patch-For-Review, Analytics-Kanban
Milimetric added a comment to T237072: Correct namespace zero editor counts on geoeditors_monthly table on hive and druid.

Status update:

Nov 5 2019, 2:03 PM · Product-Analytics, Analytics, Patch-For-Review, Analytics-Kanban

Nov 4 2019

Milimetric moved T237072: Correct namespace zero editor counts on geoeditors_monthly table on hive and druid from In Progress to In Code Review on the Analytics-Kanban board.
Nov 4 2019, 4:02 PM · Product-Analytics, Analytics, Patch-For-Review, Analytics-Kanban

Nov 1 2019

Milimetric added a comment to T237072: Correct namespace zero editor counts on geoeditors_monthly table on hive and druid.

Plan to correct, with draft of scripts needed (EDITED using feedback below):

Nov 1 2019, 7:30 PM · Product-Analytics, Analytics, Patch-For-Review, Analytics-Kanban
Milimetric updated subscribers of T237072: Correct namespace zero editor counts on geoeditors_monthly table on hive and druid.

@Ijon, this is only an issue for the 1 to 4 activity level, but it raises another question we should've asked before:

Nov 1 2019, 5:32 AM · Product-Analytics, Analytics, Patch-For-Review, Analytics-Kanban
Milimetric triaged T237072: Correct namespace zero editor counts on geoeditors_monthly table on hive and druid as High priority.
Nov 1 2019, 4:18 AM · Product-Analytics, Analytics, Patch-For-Review, Analytics-Kanban
Milimetric moved T237072: Correct namespace zero editor counts on geoeditors_monthly table on hive and druid from Next Up to In Progress on the Analytics-Kanban board.
Nov 1 2019, 4:18 AM · Product-Analytics, Analytics, Patch-For-Review, Analytics-Kanban

Oct 30 2019

Milimetric created T236941: Dashiki: Read multiple wikis from single file.
Oct 30 2019, 6:54 PM · Analytics-Kanban, Analytics

Oct 25 2019

Milimetric updated subscribers of T131280: Make aggregate data on editors per country per wiki publicly available.

review from @Ottomata is appreciated. And btw, why is the mediawiki_history fetch disabled? Is there some (problem/decision to be made) with the (rsync/labs servers) that would affect this as well?

Oct 25 2019, 12:00 AM · Product-Analytics, Analytics-Kanban

Oct 24 2019

Milimetric triaged T236403: Remove references in doc to mysql storage for EL data as High priority.
Oct 24 2019, 4:20 PM · Analytics-Kanban, Analytics-EventLogging

Oct 23 2019

Milimetric updated the task description for T234907: RFC: Where to implement Desktop Improvements project.
Oct 23 2019, 8:43 PM · Readers-Web-Backlog (Kanbanana-2019-20-Q2), Desktop Improvements, TechCom-RFC
Milimetric added a comment to T201970: RfC: ParallelMaintenance helper class for multi-process maintenance scripts.

Moving to last call per decision in today's TechCom meeting, looking to approve in 3 weeks.

Oct 23 2019, 8:39 PM · TechCom-RFC (TechCom-Approved), Patch-For-Review, MediaWiki-Maintenance-scripts
Milimetric moved T201970: RfC: ParallelMaintenance helper class for multi-process maintenance scripts from Inbox to Last Call on the TechCom-RFC board.
Oct 23 2019, 8:39 PM · TechCom-RFC (TechCom-Approved), Patch-For-Review, MediaWiki-Maintenance-scripts
Milimetric removed a project from T236150: Write an RFC describing in detail possible solutions for checking user-provided regexes in constraints: TechCom-RFC.

Removing TechCom-RFC for now, just add it back when this task is ready to go through the RFC process.

Oct 23 2019, 7:38 PM · Wikidata-Campsite, Wikibase-Quality-Constraints, Wikibase-Quality, Wikidata
Milimetric added a comment to T236223: Setup Config:Dashiki:WMCSEdits on meta wiki .

I've a few questions:

  • Is metrics-by-layout choice the right one for us?
Oct 23 2019, 7:06 PM · Analytics, Developer-Advocacy (Oct-Dec 2019), Cloud-Services

Oct 22 2019

Milimetric added a comment to T193613: Establish stable interface policy for PHP code (was: strategy for PHP interface changes).

Per the policy, public methods are considered stable (safe to call) per default. However, public constructors are not considered stable. And public methods are not automatically considered fixed (safe to override).

Oct 22 2019, 9:33 PM · Core Platform Team, Discovery-Search, TechCom, TechCom-RFC, MediaWiki-General
Milimetric added a comment to T235143: Wikistats API for legacy pagecounts does not have mobile data before October 2014.
  • Erik compiled data from sampled logs which are no longer available
Oct 22 2019, 9:22 PM · Product-Analytics, Analytics
Milimetric added a comment to T226071: Having trouble setting up MobileFrontend for development.

I tried to enable the vagrant role again to test and I got this error, so I figure either my vagrant is messed up or this will prevent me from having a clean test. I think if it works for other folks on a clean vagrant, it's fine, I'll try to test again when I look at the unit testing RFC next.

Oct 22 2019, 9:19 PM · Readers-Web-Backlog, MobileFrontend
Milimetric added a comment to T234649: Wikimedia Technical Conference 2019 Session: Front-end modernization and standardization.

Something missing in the announcements about the FAWG and this task is the possibility that none of the available options are good trade-offs/long-term investments. There is a maturity of the environment required to do such a migration and I don't know if the current frontend framework landscape has reached that stage yet.
For example, it's possible that the most useful parts of these frameworks are things that should be made part of the web platform itself and that resources and efforts in terms of long-term investments would be better made in the standards space (W3C, WHATWG, etc.).
By framing the discussion on "which framework should we pick", it might lock the discussion into a close-minded view in that respect.
If there is a need so common that most websites end up using similar frameworks to achieve it, it might be something that should be part of JS/CSS. We've seen that process from beginning to end with jQuery being less and less useful as features have made their way into JS itself. Now we're still paying the price of jQuery's weight because we're locked into it, it's completely baked into all our code, even though we use very little of it.
We might be early in such a cycle with frontend frameworks. It would be unfortunate to invest a lot of resources in the jQuery-like phase when we could be helping and pushing towards the endgame. I.e. that it becomes part of the web platform and using those features becomes a non-issue. Worse, we could make a heavy frontend framework a page load requirement that will be very difficult to undo, with a potential usefulness lifespan of a few years.

Oct 22 2019, 9:15 PM · International-Developer-Events, Wikimedia-Technical-Conference-2019
Milimetric added a comment to T131280: Make aggregate data on editors per country per wiki publicly available.

anticipating release, docs here: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Geoeditors/Public

Oct 22 2019, 3:20 PM · Product-Analytics, Analytics-Kanban

Oct 17 2019

Milimetric moved T225578: EventLogging needs to enque events to avoid draining users' battery on mobile from In Progress to In Code Review on the Analytics-Kanban board.
Oct 17 2019, 3:56 PM · Performance-Team, Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, Analytics

Oct 15 2019

Milimetric added a comment to T233432: Figure out how to $ref common schema across schema repositories.

I wouldn't worry too much about how others are using this in their own mediawiki installs. Not that it's not important, just that we can't possibly guess as to how they might want to do that. Just having three repos with some common stuff will allow for plenty of flexibility and refactoring later on.

Oct 15 2019, 8:39 PM · Analytics-Kanban, CPT Initiatives (Modern Event Platform (TEC2)), Services (watching), Analytics-EventLogging, Event-Platform, Analytics
Milimetric added a comment to T233432: Figure out how to $ref common schema across schema repositories.

I like a single namespace, especially because having "common" as a root would be too vague. This might be useful:

Oct 15 2019, 3:58 PM · Analytics-Kanban, CPT Initiatives (Modern Event Platform (TEC2)), Services (watching), Analytics-EventLogging, Event-Platform, Analytics
Milimetric changed the status of T234461: Sudden drop in WikipediaPortal events from Declined to Resolved.
Oct 15 2019, 3:37 PM · Analytics-Kanban, Analytics, Analytics-EventLogging, Wikimedia-Portals

Oct 11 2019

Milimetric moved T131280: Make aggregate data on editors per country per wiki publicly available from In Progress to In Code Review on the Analytics-Kanban board.
Oct 11 2019, 3:10 AM · Product-Analytics, Analytics-Kanban

Oct 10 2019

Milimetric added a comment to T234188: Taxonomy of new user reading patterns.

I looked at this and it's a clever way to get some rough information to answer the main question. But I just wanted to point out: it's not by accident that this kind of connection is hard to make. We made a conscious decision a while back that we should not optimize accessing reading patterns of specific users. Being able to get this data is problematic for privacy reasons, and more so if we can get this data quickly. So, nice work, but I would hesitate before optimizing it too much more.

Oct 10 2019, 5:17 PM · Analytics, Research