Nuria (Nuria)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Nov 26 2014, 3:04 AM (216 w, 4 d)
Availability
Available
LDAP User
Nuria
MediaWiki User
NRuiz (WMF) [ Global Accounts ]

Recent Activity

Today

Nuria added a comment to T205744: Deprecation Information for EventLogging ResourceLoader modules.

Tagging some people in this ticket that should know about these changes.

Sun, Jan 20, 2:23 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Analytics-Kanban, Analytics-EventLogging, Analytics
Nuria updated subscribers of T205744: Deprecation Information for EventLogging ResourceLoader modules.
Sun, Jan 20, 2:23 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Analytics-Kanban, Analytics-EventLogging, Analytics

Fri, Jan 18

Nuria added a comment to T211173: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset.

The code to ingest this data already exists but it does not work well due to number of dimensions and how hard it is to understand the dimensions and measures (at least for me) in the fully denomalized dataset, see:

Fri, Jan 18, 7:33 PM · Product-Analytics, Analytics
Nuria added a comment to T213566: Transferring data from Hadoop to production MySQL database.

Finding a separate place to run the import scripts is probably the path of least resistance.

+1

Fri, Jan 18, 4:19 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria updated subscribers of T211173: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset.

That is quite easy, it just will take a few hours to load data with different transformations and see how the data looks in turnilo, probably @mforns will be doing this work

Fri, Jan 18, 3:54 PM · Product-Analytics, Analytics

Thu, Jan 17

Nuria updated subscribers of T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.

Confirmed with @MusikAnimal that he can see usage dashboard cc @jmatazzoni Closing ticket

Thu, Jan 17, 10:38 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria closed T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool as Resolved.
Thu, Jan 17, 10:37 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria added a comment to T213566: Transferring data from Hadoop to production MySQL database.

@bmansurov how do handle deleting data in your storage when you have reached capacity or when that dataset is bad? There must be a daemon that takes care of that work right ? as older versions of data are no longer useful and should not occupy space. That process that should be automated is not dissimilar to the one that would load the data, right? One is concern with deletion of older versions (language pairs might have been added or removed) other is concern with loading of newer versions.

Thu, Jan 17, 8:46 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria removed a project from T213996: New MongoDB version is not DFSG-compatible, dropped by Debian: Analytics.
Thu, Jan 17, 8:37 PM · VisualEditor, Software-Licensing, Performance-Team, Operations
Nuria added a comment to T213566: Transferring data from Hadoop to production MySQL database.

I think one telling use case the ilustrates why we want to decouple data loading from hadoop is a rollback. Say that you have RecomendationsData1.tsv and RecomendationsData2.tsv and you have currently loaded in prod RecomendationsData1.tsv. You produce a new data set in cluster and load it (RecomendationsData2.tsv). This dataset, despite your data guards, has bad data . You want an immediate rollback and that process should not involve the cluster in any case since you have that data (might need reloading as it might no longer be on mysql) . You would go to the mount where the data exists and reload it as needed using service code, not code that runs inside an oozie job or similar.

Thu, Jan 17, 8:31 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria added a comment to T213566: Transferring data from Hadoop to production MySQL database.

It has been abandoned after Analytics said to not use stat hosts and use Hadoop instead.

To clarify: stats machines should not be in the path to update a production database, that has not changed. The current debate (on this ticket) is whether it makes sense to do it from hadoop (yours truly disagrees) or rather (my preferred option) to have a daemon on the mysql hosts that gets notified that data is available for consumption, fetches a file from an accessible mount and does whatever needs doing: inserting data, droping data , creating databases...

Thu, Jan 17, 8:21 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria added a comment to T213566: Transferring data from Hadoop to production MySQL database.

Unless...unless that is we can actually write to the MySQL db from Hadoop.

I do not think we shoudl consider this an option. We should ahve a clear separation of concerns and while the hadoop cluster is in charge of computing the task of updating the db needs to live in the service side.

Thu, Jan 17, 7:46 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria added a comment to T210687: Bug: can't make a YoY time series chart in Superset.

Just try again, teh tiem granularity needs to be set to "day"

Thu, Jan 17, 7:44 PM · Analytics-Kanban, Product-Analytics, Analytics
Nuria triaged T214043: Make edit data lake data available as a snapshot on dump hosts that can be sourced by Presto as High priority.
Thu, Jan 17, 2:30 PM · Analytics-Kanban
Nuria set the point value for T211609: Create Office Hours for Team Analytics to 1.
Thu, Jan 17, 2:26 PM · Analytics-Kanban, Analytics
Nuria closed T207321: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster as Resolved.
Thu, Jan 17, 2:26 PM · Analytics-Kanban, netops, Operations, Analytics
Nuria closed T207321: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster, a subtask of T204951: Presto cluster online and usable with test data pushed from analytics prod infrastructure accessible by Cloud (labs) users, as Resolved.
Thu, Jan 17, 2:26 PM · Patch-For-Review, Analytics, Analytics-Kanban
Nuria closed T211000: Failure while refining webrequest upload 2018-12-01-14. Upgrade alarms as Resolved.
Thu, Jan 17, 2:26 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria closed T211717: Clickstream job failing due to change of types of namespace column as Resolved.
Thu, Jan 17, 2:25 PM · Patch-For-Review, Analytics-Kanban
Nuria closed T212420: Move KafkaSSE development from Differential to Github as Resolved.
Thu, Jan 17, 2:18 PM · Patch-For-Review, Analytics-Kanban, Analytics, Wikimedia-Stream, Phabricator
Nuria closed T212420: Move KafkaSSE development from Differential to Github, a subtask of T191182: Stop using Differential for code review, as Resolved.
Thu, Jan 17, 2:18 PM · Release-Engineering-Team (Backlog), Phabricator, Gerrit
Nuria closed T206943: JVM pauses cause Yarn master to failover as Resolved.
Thu, Jan 17, 2:18 PM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
Nuria added a comment to T213716: Alarms for virtualpageview should exist (probably in oozie) for jobs that have been idle too long.

The hue dashboard for workflows displays SLAS (probably not big news) . Looked at docs and i saw we can add an alarm for durantion of job which "seems" to be different than sla for end time. Now, using oozie's sla bindings there is no way to send repeated alarms .

Thu, Jan 17, 1:54 PM · Analytics
Nuria closed T205846: Move users from stat1005 to stat1007 as Resolved.
Thu, Jan 17, 1:51 PM · Analytics-Kanban, Patch-For-Review, Analytics
Nuria closed T212778: Add is_pageview as a dimension to the 'webrequest_sampled_128' Druid dataset as Resolved.
Thu, Jan 17, 1:50 PM · Analytics-Kanban, Patch-For-Review, Analytics
Nuria added a comment to T212778: Add is_pageview as a dimension to the 'webrequest_sampled_128' Druid dataset.

Nice to know that druid admin interface displayed all dimensions: webrequest_source hostname time_firstbyte ip http_status response_size http_method uri_host uri_path uri_query content_type referer user_agent x_cache continent country_code isp as_number is_pageview

Thu, Jan 17, 1:50 PM · Analytics-Kanban, Patch-For-Review, Analytics
Nuria added a comment to T212778: Add is_pageview as a dimension to the 'webrequest_sampled_128' Druid dataset.

mmmm .. both jobs were restarted on the 1/7

Thu, Jan 17, 12:35 AM · Analytics-Kanban, Patch-For-Review, Analytics
Nuria added a comment to T212778: Add is_pageview as a dimension to the 'webrequest_sampled_128' Druid dataset.

In neither turnilo nor superset does is_pageview appear as a dimension. I think we might need a job restart.

Thu, Jan 17, 12:05 AM · Analytics-Kanban, Patch-For-Review, Analytics

Wed, Jan 16

Nuria added a comment to T210687: Bug: can't make a YoY time series chart in Superset.

Trying link again: https://superset.wikimedia.org/superset/explore/?form_data=%7B%22datasource%22%3A%22352__druid%22%2C%22viz_type%22%3A%22time_pivot%22%2C%22slice_id%22%3A144%2C%22granularity%22%3A%22P1M%22%2C%22druid_time_origin%22%3Anull%2C%22since%22%3A%222016-01-01T00%3A00%3A00%22%2C%22until%22%3A%222018-12-31T00%3A00%3A00%22%2C%22metric%22%3A%22count%22%2C%22adhoc_filters%22%3A%5B%7B%22comparator%22%3A%22user%22%2C%22operator%22%3A%22%3D%3D%22%2C%22fromFormData%22%3Atrue%2C%22clause%22%3A%22WHERE%22%2C%22sqlExpression%22%3Anull%2C%22subject%22%3A%22agent_type%22%2C%22expressionType%22%3A%22SIMPLE%22%2C%22filterOptionName%22%3A%22filter_s6qulnxa2o_bre8xqzho8g%22%7D%5D%2C%22freq%22%3A%22AS%22%2C%22show_legend%22%3Atrue%2C%22line_interpolation%22%3A%22monotone%22%2C%22color_picker%22%3A%7B%22a%22%3A1%2C%22r%22%3A123%2C%22g%22%3A0%2C%22b%22%3A81%7D%2C%22x_axis_label%22%3A%22%22%2C%22bottom_margin%22%3A%22auto%22%2C%22x_axis_showminmax%22%3Afalse%2C%22x_axis_format%22%3A%22smart_date%22%2C%22y_axis_label%22%3A%22Pageviews+Daily+%28all+projects%29%22%2C%22left_margin%22%3A%22auto%22%2C%22y_axis_showminmax%22%3Afalse%2C%22y_log_scale%22%3Afalse%2C%22y_axis_format%22%3A%22.3s%22%2C%22y_axis_bounds%22%3A%5Bnull%2Cnull%5D%2C%22having%22%3A%22%22%2C%22filters%22%3A%5B%7B%22op%22%3A%22%3D%3D%22%2C%22val%22%3A%22user%22%2C%22col%22%3A%22agent_type%22%7D%5D%2C%22where%22%3A%22%22%2C%22having_filters%22%3A%5B%5D%2C%22url_params%22%3A%7B%7D%7D

Wed, Jan 16, 11:05 PM · Analytics-Kanban, Product-Analytics, Analytics
Nuria closed T210687: Bug: can't make a YoY time series chart in Superset, a subtask of T211706: Superset Updates , as Resolved.
Wed, Jan 16, 10:20 PM · Analytics-Kanban, Product-Analytics
Nuria closed T210687: Bug: can't make a YoY time series chart in Superset as Resolved.
Wed, Jan 16, 10:20 PM · Analytics-Kanban, Product-Analytics, Analytics
Nuria added a comment to T210687: Bug: can't make a YoY time series chart in Superset.

See graph For YoY pageviews: https://bit.ly/2QRRiCw in superset cc @JKatzWMF

Wed, Jan 16, 10:20 PM · Analytics-Kanban, Product-Analytics, Analytics
Nuria added a subtask for T213976: Workflow to be able to move data files computed in jobs from analytics cluster to production : T213566: Transferring data from Hadoop to production MySQL database.
Wed, Jan 16, 10:00 PM · Discovery, Analytics
Nuria added a parent task for T213566: Transferring data from Hadoop to production MySQL database: T213976: Workflow to be able to move data files computed in jobs from analytics cluster to production .
Wed, Jan 16, 10:00 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria created T213976: Workflow to be able to move data files computed in jobs from analytics cluster to production .
Wed, Jan 16, 9:59 PM · Discovery, Analytics
Nuria added a comment to T172410: Replace the current multisource analytics-store setup.

I definitely understand that you can't make any ironclad guarantees when you're under a hard end-of-life constraint (that's why I wrote that y'all agreed to work on it 😁).

Right, no iron clad guarantees. I clarifying that our top priority is to replace the hardware that is failing continuously (https://phabricator.wikimedia.org/T213670) and other requests we can consider after that work is done.

Wed, Jan 16, 7:49 PM · Product-Analytics, Analytics, WMDE-Analytics-Engineering, User-Addshore, User-Elukey, Research
Nuria moved T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool from Ready to Deploy to Done on the Analytics-Kanban board.
Wed, Jan 16, 5:08 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria added a comment to T206700: Create a method for putting 'Avg. daily views to pages that have files uploaded' metric into 'Event Summary' reports .

Before advertising this metric it will be good to quantify how good this approximation is to the "real" number, that can be easily done with comparing the results of this calculation with the data on the mediacounts table on hive.

Wed, Jan 16, 4:01 PM · Community-Tech-Sprint, Grant-Metrics, Community-Tech, Event Metrics
Nuria updated the task description for T213923: Create staging environment for superset .
Wed, Jan 16, 2:31 PM · Analytics-Kanban, Analytics
Nuria triaged T213923: Create staging environment for superset as High priority.
Wed, Jan 16, 2:30 PM · Analytics-Kanban, Analytics
Nuria added a comment to T213566: Transferring data from Hadoop to production MySQL database.

@bmansurov sounds good, let's start working on the code for oozie jobs, that work has to be done before we could transfer data into mySQL anyways.

Wed, Jan 16, 2:17 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria added a comment to T172410: Replace the current multisource analytics-store setup.

Analytics agreed to work on providing such a tool before the shutdown of dbstore1002 (based of course on our help clearly defining how we'd like the tool to work).

Let me clarify: our biggest priority is that the hardware gets replaced before end of life. We are going to try to install in the replicas the tools that we now have in production that work across shards, we think those will work but we are not 100% sure until we install them. It might very well be that there is a period in which the hardware has been moved to the multi host setup but yet there is no tool to access shards in a shard agnostic way. Let's please keep this is mind. We will have an update for you guys probably by mid February, until then we will just be working on the replacement.

Wed, Jan 16, 1:45 PM · Product-Analytics, Analytics, WMDE-Analytics-Engineering, User-Addshore, User-Elukey, Research
Nuria added a comment to T213875: Explore alternatives to browser fingerprinting for anti-abuse efforts.

Rather than focusing on the bad actor why not focus in the target giving them tools to protect interactions? For example: they can protect their talk pages such to post you need to know the "secret" word (similar to a captcha), this way interactions are "whitelisted"

Wed, Jan 16, 12:14 AM · MediaWiki-User-management, Anti-Harassment

Tue, Jan 15

Nuria added a comment to T213566: Transferring data from Hadoop to production MySQL database.

The first thing we need to do is to oozie-fy the data creation workflow that produces the files you would be loading into mysql (likely tsv), let's work on that while Analytics works on designing a workflow for you to transfer data into MySQL.

Tue, Jan 15, 11:18 PM · User-Marostegui, Operations, Article-Recommendation, Analytics, Research
Nuria added a comment to T213351: Timeboxed investigation into browser fingerprinting for anti-abuse report to WMF Board.

@dbarratt I think you need to do due diligence when replying, From the docs you sent "the value {UUID] changes when the user deletes all of that vendor’s apps from the device and subsequently reinstalls one or more of them". A UUID identifies an install of the app, that value is equivalent to the appinstallId already present on Wikipedia's app. There is a difference with app install Id and a fingerprint.

Tue, Jan 15, 7:49 PM · Anti-Harassment (Bet — ב)
Nuria added a comment to T212885: NLP contractor set up and access.

Request access for Julia to Stats machines (after NDA)

Please let me know if there is issues with NDA, we will need a date by which this contract is over in order to expire access. Confirming that data never leaves our systems, we expect collaborators to do all the work on our cluster/stats machines.

Tue, Jan 15, 5:51 PM · Discovery-Search (Current work)
Nuria claimed T213716: Alarms for virtualpageview should exist (probably in oozie) for jobs that have been idle too long.
Tue, Jan 15, 4:21 PM · Analytics
Nuria moved T206267: Create labeled dataset for bot identification from In Code Review to Done on the Analytics-Kanban board.
Tue, Jan 15, 4:01 PM · Analytics-Kanban, Research, Analytics
Nuria moved T211359: POC More efficient Bot filtering on pageview data from In Code Review to Done on the Analytics-Kanban board.
Tue, Jan 15, 4:01 PM · Analytics-Kanban, Research, Analytics
Nuria added a comment to T209732: Wire ORES recent_score events into Hadoop.

So i understand, the expectation here will be that files are written for all hours but empty for those of which there was no data?

Tue, Jan 15, 3:00 PM · Patch-For-Review, Scoring-platform-team (Current), Analytics, ORES
Nuria added a comment to T213602: virtualpageview_hourly lacks data from December 17 on.

It finished by midnite the December data, which is the one you needed for the report.

Tue, Jan 15, 1:42 PM · Analytics

Mon, Jan 14

Nuria added a comment to T213602: virtualpageview_hourly lacks data from December 17 on.

I think is taking about 3 minutes per hour, which means it should finish by midnite today, January 14th.

Mon, Jan 14, 10:20 PM · Analytics
Nuria added a comment to T213602: virtualpageview_hourly lacks data from December 17 on.

Data is present now up to the 22nd.

Mon, Jan 14, 10:14 PM · Analytics
Nuria added a comment to T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.

Ok, let me know when you do and we can verify there is traffic coming in.

Mon, Jan 14, 9:59 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria closed T153641: How to use Wikipedia EventLogging schemas in Vagrant setup? as Resolved.
Mon, Jan 14, 5:55 PM · Analytics-Kanban, Analytics, Analytics-EventLogging, MediaWiki-Vagrant
Nuria set the point value for T153641: How to use Wikipedia EventLogging schemas in Vagrant setup? to 3.
Mon, Jan 14, 5:54 PM · Analytics-Kanban, Analytics, Analytics-EventLogging, MediaWiki-Vagrant
Nuria closed T210099: druid ingestion should calculate 1/sample rate to be able to normalize event counts as Resolved.
Mon, Jan 14, 5:54 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria closed T209929: Decommission old Hadoop worker nodes and add newer ones as Resolved.
Mon, Jan 14, 5:54 PM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
Nuria closed T168477: Update html language for per-domain uniques, a subtask of T167539: Final steps to expose project family unique devices data , as Resolved.
Mon, Jan 14, 5:53 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria closed T168477: Update html language for per-domain uniques as Resolved.
Mon, Jan 14, 5:53 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria closed T209103: unique devices monthly should be configured with default "monthly" granularity in turnilo as Resolved.
Mon, Jan 14, 5:52 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria added a comment to T209103: unique devices monthly should be configured with default "monthly" granularity in turnilo.

While split option is not "very" configurable i have shorcuted it such for monthly datasets only monthly splits are available by default. I think things make more sense now. Closing ticket

Mon, Jan 14, 5:52 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria moved T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool from Next Up to Ready to Deploy on the Analytics-Kanban board.
Mon, Jan 14, 5:44 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria set the point value for T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool to 1.
Mon, Jan 14, 5:44 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria added a project to T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool: Analytics-Kanban.
Mon, Jan 14, 5:43 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria created T213741: Use MaxMind DB in piwik geo-location .
Mon, Jan 14, 5:40 PM · Analytics
Nuria added a comment to T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.

Geographical location is based on user's language so actually not very accurate but better than nothing.

Mon, Jan 14, 5:38 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria added a comment to T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.

Tracking code:

Mon, Jan 14, 5:37 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria updated the task description for T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.
Mon, Jan 14, 5:33 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria added a comment to T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.
Mon, Jan 14, 5:32 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria claimed T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.
Mon, Jan 14, 5:04 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria created T213735: Add site to piwik.wikimedia.org for Event Metrics so we can measure traffic to tool.
Mon, Jan 14, 5:02 PM · Community-Tech-Sprint, Event Metrics, Community-Tech, Analytics-Kanban, Analytics
Nuria added a comment to T213602: virtualpageview_hourly lacks data from December 17 on.

@Tbayer we will be working on seeing why this job stopped (it is missing a few hours arround september 17th) in the meantime to approximate data if this is urgent you can use eventlogging requests which are there as you mentioned.

Mon, Jan 14, 4:24 PM · Analytics
Nuria added a comment to T213602: virtualpageview_hourly lacks data from December 17 on.

There is no gap on incoming data: https://grafana.wikimedia.org/d/000000018/eventlogging-schema?orgId=1&from=now-34d&to=now

Mon, Jan 14, 4:22 PM · Analytics
Nuria created T213716: Alarms for virtualpageview should exist (probably in oozie) for jobs that have been idle too long.
Mon, Jan 14, 4:20 PM · Analytics

Fri, Jan 11

Nuria added a comment to T213219: Reportupdater queries jobs failing.

Thanks to @Milimetric for the fast turnaround here

Fri, Jan 11, 2:47 PM · Analytics-Kanban, Analytics
Nuria closed T213219: Reportupdater queries jobs failing as Resolved.
Fri, Jan 11, 2:47 PM · Analytics-Kanban, Analytics
Nuria added a comment to T211950: Add partial blocks to mediawiki history tables.

Super, super thanks to @nettrom_WMF for flagging this issue so we can incorporate changes to mw history

Fri, Jan 11, 2:12 PM · Product-Analytics, Anti-Harassment, Analytics
Nuria added a comment to T212386: Provide tools for querying MediaWiki replica databases without having to specify the shard.

@jcrespo it seems we should be able to deploy (out of the box with a new config) the tool existing in prod to the new and upcoming analytics replicas right? Am I missing something why this would not be possible?

Fri, Jan 11, 2:00 PM · Analytics, WMDE-Analytics-Engineering, User-Addshore, User-Elukey, Research
Nuria closed T1384: Capture Javascript support level for web users as Resolved.
Fri, Jan 11, 1:48 PM · MediaWiki-extensions-WikimediaEvents, MediaWiki-General-or-Unknown, FINCH
Nuria added a comment to T1384: Capture Javascript support level for web users.

Docs or it didn't happen! ;-)

Indeed. Super thanks

Fri, Jan 11, 1:48 PM · MediaWiki-extensions-WikimediaEvents, MediaWiki-General-or-Unknown, FINCH
Nuria added a comment to T212386: Provide tools for querying MediaWiki replica databases without having to specify the shard.

@Marostegui I am a bit lost, I though jaime was talking about "prod"databases, but your post is about the replicas on cloud correct?

Fri, Jan 11, 1:13 PM · Analytics, WMDE-Analytics-Engineering, User-Addshore, User-Elukey, Research
Nuria added a project to T206894: Set up automated email to report completion of mediawiki_history snapshot and Druid loading: Analytics-Kanban.
Fri, Jan 11, 1:03 PM · Patch-For-Review, Analytics-Kanban, Analytics, Contributors-Analysis, Product-Analytics
Nuria added a comment to T206894: Set up automated email to report completion of mediawiki_history snapshot and Druid loading.

Ok, moving to kanban and assigning to fdans as background work, e-mail will be sent to product-analytics@wikimedia.org when data is available

Fri, Jan 11, 1:03 PM · Patch-For-Review, Analytics-Kanban, Analytics, Contributors-Analysis, Product-Analytics
Nuria assigned T206894: Set up automated email to report completion of mediawiki_history snapshot and Druid loading to fdans.
Fri, Jan 11, 1:02 PM · Patch-For-Review, Analytics-Kanban, Analytics, Contributors-Analysis, Product-Analytics
Nuria moved T168477: Update html language for per-domain uniques from In Progress to Done on the Analytics-Kanban board.
Fri, Jan 11, 1:01 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria added a comment to T213290: Add Chinese Wikiversity edit-related metrics to Wikistats 2.

Nice, there should be metrics up to January 2019 when the February snapshot is computed.

Fri, Jan 11, 1:00 PM · Chinese-Sites, Analytics-Kanban, Patch-For-Review, Analytics
Nuria added a comment to T211835: Sunset Wikimetrics .

@MaxSem Noted, still, is teh best replacement that exists for a tool that sees very few use if any.

Fri, Jan 11, 12:57 PM · Analytics-Wikimetrics, Analytics
Nuria added a comment to T213351: Timeboxed investigation into browser fingerprinting for anti-abuse report to WMF Board.

Leaving here comments that I also sent via e-mail cause although others have made similar point on this ticket I feel they are worth reiterating.

Fri, Jan 11, 12:29 PM · Anti-Harassment (Bet — ב)

Tue, Jan 8

Nuria updated subscribers of T209103: unique devices monthly should be configured with default "monthly" granularity in turnilo.

ping @JKatzWMF take a loot at http://turnilo.wikimedia.org , monthly datasets display now with monthly defaults. The "split" option is not configurable as far as I can see.

Tue, Jan 8, 5:39 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria moved T209103: unique devices monthly should be configured with default "monthly" granularity in turnilo from Ready to Deploy to In Code Review on the Analytics-Kanban board.
Tue, Jan 8, 3:29 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria moved T209103: unique devices monthly should be configured with default "monthly" granularity in turnilo from Next Up to Ready to Deploy on the Analytics-Kanban board.
Tue, Jan 8, 3:29 PM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria closed T212862: Update IP addresses of cloud labs to mark internal traffic on refinery code as Resolved.
Tue, Jan 8, 12:01 PM · Analytics-Kanban, Patch-For-Review, Analytics
Nuria closed T153821: wikitech.wikimedia.org missing from pageviews API as Resolved.
Tue, Jan 8, 12:01 PM · Patch-For-Review, Pageviews-API, Analytics, wikitech.wikimedia.org
Nuria closed T153821: wikitech.wikimedia.org missing from pageviews API, a subtask of T161859: Make Wikitech an SUL wiki, as Resolved.
Tue, Jan 8, 12:01 PM · Epic, wikitech.wikimedia.org
Nuria added a comment to T153821: wikitech.wikimedia.org missing from pageviews API.

Wikitech pageviews are now available as of yesterday, closing: https://wikimedia.org/api/rest_v1/metrics/pageviews/aggregate/wikitech.wikimedia.org/all-access/user/daily/2019010700/2019010800

Tue, Jan 8, 12:00 PM · Patch-For-Review, Pageviews-API, Analytics, wikitech.wikimedia.org
Nuria closed T212958: Create staging domain for turnilo to test config changes, a subtask of T209103: unique devices monthly should be configured with default "monthly" granularity in turnilo, as Resolved.
Tue, Jan 8, 9:08 AM · Patch-For-Review, Analytics-Kanban, Analytics
Nuria closed T212958: Create staging domain for turnilo to test config changes as Resolved.
Tue, Jan 8, 9:08 AM · User-Elukey, Analytics-Kanban, Analytics
Nuria added a comment to T212958: Create staging domain for turnilo to test config changes.

Documented process, will tune as I test today but closing ticket: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Turnilo-Pivot#Test_config_changes

Tue, Jan 8, 9:08 AM · User-Elukey, Analytics-Kanban, Analytics
Nuria added a comment to T1384: Capture Javascript support level for web users.

@Quiddity the mediawiki page you listed predates the browser stats: https://analytics.wikimedia.org/dashboards/browsers/#all-sites-by-os with those and our support matrix is actually easy to figure out the percentage of requests w/o javascript support (example: IE9 supports some js but we do not serve js to such an old browser). This applies to both users and editors so at this time, we have quite a lot of data to triage bugs based on browser usage. On my opinion this task can be closed.

Tue, Jan 8, 8:50 AM · MediaWiki-extensions-WikimediaEvents, MediaWiki-General-or-Unknown, FINCH