elukey (Luca Toscano)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Jan 5 2016, 9:54 PM (119 w, 5 d)
Availability
Available
LDAP User
Unknown
MediaWiki User
LToscano (WMF)

Recent Activity

Today

elukey updated the task description for T192640: Reimage stat1004 with Debian Stretch.
Mon, Apr 23, 1:48 PM · Analytics, User-Elukey
elukey moved T164008: Update druid to 0.10 from Paused to In Code Review on the Analytics-Kanban board.
Mon, Apr 23, 1:03 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey added a project to T164008: Update druid to 0.10: Analytics-Kanban.
Mon, Apr 23, 1:03 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey renamed T164008: Update druid to 0.10 from Update druid to latest release (0.11) to Update druid to 0.10.
Mon, Apr 23, 1:03 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey added a comment to T164008: Update druid to 0.10.

true
false
false
false

Mon, Apr 23, 10:02 AM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey updated subscribers of T164008: Update druid to 0.10.

Me and the JS master @fdans checked the Pivot's code this morning, and after a lot of tests we identified what returns the error:

Mon, Apr 23, 9:49 AM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey added a comment to T164008: Update druid to 0.10.

Pivot deployed on d-1, usable via:

Mon, Apr 23, 8:10 AM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review

Fri, Apr 20

elukey added a comment to T164008: Update druid to 0.10.

Upgraded d[1-3] in labs to druid 0.10, adding manual hiera config as replacement for https://gerrit.wikimedia.org/r/#/c/355471.

Fri, Apr 20, 2:56 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey updated the task description for T192557: Reimage the Debian Jessie Analytics worker nodes to Stretch..
Fri, Apr 20, 12:07 PM · Analytics-Kanban, Patch-For-Review, Analytics
elukey updated the task description for T192557: Reimage the Debian Jessie Analytics worker nodes to Stretch..
Fri, Apr 20, 12:07 PM · Analytics-Kanban, Patch-For-Review, Analytics
elukey updated the task description for T192557: Reimage the Debian Jessie Analytics worker nodes to Stretch..
Fri, Apr 20, 11:32 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey updated the task description for T192557: Reimage the Debian Jessie Analytics worker nodes to Stretch..
Fri, Apr 20, 11:32 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey added a parent task for T182924: Refresh zookeeper nodes in eqiad: T192642: Upgrade Analytics infrastructure to Debian Stretch.
Fri, Apr 20, 11:24 AM · Patch-For-Review, User-Elukey, Analytics-Kanban
elukey added subtasks for T192642: Upgrade Analytics infrastructure to Debian Stretch: T192639: Upgrade Archiva (meitnerium) to Debian Stretch, T182924: Refresh zookeeper nodes in eqiad, T192636: Upgrade Druid nodes (1001->1006) to Debian Stretch, T192557: Reimage the Debian Jessie Analytics worker nodes to Stretch., T192640: Reimage stat1004 with Debian Stretch, T192641: Reimage thorium to Debian Stretch.
Fri, Apr 20, 11:24 AM · Analytics-Kanban, Analytics
elukey added a parent task for T192557: Reimage the Debian Jessie Analytics worker nodes to Stretch.: T192642: Upgrade Analytics infrastructure to Debian Stretch.
Fri, Apr 20, 11:24 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey added a parent task for T192636: Upgrade Druid nodes (1001->1006) to Debian Stretch: T192642: Upgrade Analytics infrastructure to Debian Stretch.
Fri, Apr 20, 11:24 AM · Analytics, User-Elukey
elukey added a parent task for T192639: Upgrade Archiva (meitnerium) to Debian Stretch: T192642: Upgrade Analytics infrastructure to Debian Stretch.
Fri, Apr 20, 11:24 AM · User-Elukey, Analytics
elukey triaged T192642: Upgrade Analytics infrastructure to Debian Stretch as Normal priority.
Fri, Apr 20, 11:23 AM · Analytics-Kanban, Analytics
elukey created T192641: Reimage thorium to Debian Stretch.
Fri, Apr 20, 11:22 AM · Analytics
elukey added a project to T192640: Reimage stat1004 with Debian Stretch: Analytics.
Fri, Apr 20, 11:20 AM · Analytics, User-Elukey
elukey created T192640: Reimage stat1004 with Debian Stretch.
Fri, Apr 20, 11:19 AM · Analytics, User-Elukey
elukey created T192639: Upgrade Archiva (meitnerium) to Debian Stretch.
Fri, Apr 20, 10:36 AM · User-Elukey, Analytics
elukey created T192636: Upgrade Druid nodes (1001->1006) to Debian Stretch.
Fri, Apr 20, 10:08 AM · Analytics, User-Elukey
elukey added a comment to T164008: Update druid to 0.10.

First step of testing confirmed on labs with druid 0.9.2:

  • Indexation from hadoop
  • Realtime indexation with tranquility
Fri, Apr 20, 7:45 AM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review

Thu, Apr 19

elukey triaged T192557: Reimage the Debian Jessie Analytics worker nodes to Stretch. as Normal priority.
Thu, Apr 19, 3:04 PM · Analytics-Kanban, Patch-For-Review, Analytics
elukey moved T164341: Decommission old memcached hosts - mc1001->mc1018 from Done to Keep an eye on it on the User-Elukey board.
Thu, Apr 19, 2:49 PM · Patch-For-Review, User-Elukey, Operations, ops-eqiad
elukey added a comment to T164341: Decommission old memcached hosts - mc1001->mc1018.

ping - status :)

Thu, Apr 19, 2:49 PM · Patch-For-Review, User-Elukey, Operations, ops-eqiad
elukey removed projects from T166081: rack/setup/install conf1004-conf1006: Patch-For-Review, User-Elukey.
Thu, Apr 19, 2:48 PM · User-Joe, Operations
elukey moved T166081: rack/setup/install conf1004-conf1006 from Stalled to Keep an eye on it on the User-Elukey board.
Thu, Apr 19, 2:48 PM · User-Joe, Operations
elukey moved T181036: Pull netflow data in realtime from Kafka via Tranquillity/Spark from Stalled to Keep an eye on it on the User-Elukey board.
Thu, Apr 19, 2:48 PM · User-Elukey, monitoring, netops, Operations
elukey moved T189464: Fix Mirror Maker erratic behavior when replicating from main-eqiad to jumbo from Backlog to Keep an eye on it on the User-Elukey board.
Thu, Apr 19, 2:46 PM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics, Analytics-Cluster
elukey moved T159584: Secure hue and other private data access sites with 2FA from Backlog to Analytics Backlog on the User-Elukey board.
Thu, Apr 19, 2:45 PM · User-Elukey, Analytics
elukey moved T164008: Update druid to 0.10 from Analytics Backlog to In Progress on the User-Elukey board.
Thu, Apr 19, 2:44 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey added a comment to T164008: Update druid to 0.10.

After a chat with the team we decided to proceed with Druid 0.10 for the moment, since we have basically everything that we need ready to go.

Thu, Apr 19, 1:57 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review

Wed, Apr 18

elukey added a comment to T164008: Update druid to 0.10.

Restarting to work on this after the Hadoop cluster has been migrated to Java 8. The latest stable release is currently 0.12, meanwhile we are running 0.9.2. The previous attempt was targeting 0.10.

Wed, Apr 18, 1:45 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey added a comment to T136732: Puppetize job that saves old versions of Maxmind geoIP database.

The total size of the archive as it is right now is 17Gb. @Ottomata @elukey is it sustainable to sync 17Gb+ weekly between volatile and /usr/share/GeoIP?

Wed, Apr 18, 1:07 PM · Puppet, Patch-For-Review, Analytics-Kanban
elukey added a comment to T192348: SparkR on Spark 2.3.0 - Testing on Large Data Sets.

Ok! I am going to chat with Andrew about https://gerrit.wikimedia.org/r/427159, since adding r-base-core to the packages pinned to jessie-backports seems to work (tested this morning with `sudo apt-get install r-base-core -t jessie-backports on an1028 before cleaning up).

Wed, Apr 18, 10:05 AM · User-GoranSMilovanovic, Analytics-Kanban, Patch-For-Review, WMDE-Analytics-Engineering
elukey added a comment to T192348: SparkR on Spark 2.3.0 - Testing on Large Data Sets.

This morning I removed the old apt config for jessie backports (since after https://gerrit.wikimedia.org/r/427170 it seemed not needed and puppet was broken on Jessie hosts) but now this is the situation for the Hadoop workers:

Wed, Apr 18, 9:10 AM · User-GoranSMilovanovic, Analytics-Kanban, Patch-For-Review, WMDE-Analytics-Engineering

Tue, Apr 17

elukey awarded T187014: Proxies information gone from Zero portal. Opera mini pageviews geolocating to wrong country a Love token.
Tue, Apr 17, 9:46 AM · Zero, Patch-For-Review, Analytics-Data-Quality, Analytics-Kanban, Operations, Traffic, Analytics, Readers-Web-Backlog (Tracking), Mobile, New-Readers

Mon, Apr 16

elukey moved T182993: TLS security review of the Kafka stack from Stalled to Keep an eye on it on the User-Elukey board.
Mon, Apr 16, 4:40 PM · Patch-For-Review, Traffic, User-Elukey, Analytics-Kanban, Analytics-Cluster, Operations
elukey moved T164008: Update druid to 0.10 from Backlog to Analytics Backlog on the User-Elukey board.
Mon, Apr 16, 4:39 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey moved T192298: Update piwik to latest stable from Backlog to Analytics Backlog on the User-Elukey board.
Mon, Apr 16, 4:39 PM · User-Elukey, Analytics
elukey added a project to T164008: Update druid to 0.10: User-Elukey.
Mon, Apr 16, 4:26 PM · Analytics-Kanban, User-Elukey, Analytics, Patch-For-Review
elukey renamed T192298: Update piwik to latest stable from update piwik to latest stable to Update piwik to latest stable.
Mon, Apr 16, 3:42 PM · User-Elukey, Analytics
elukey added a comment to T182924: Refresh zookeeper nodes in eqiad.

Tested in labs a migration with two stretch hosts running zk 3.4.9 and one jessie host running zk 3.4.9 (Moritz's backport) and the host swap happened without any issue (no host in LOOKING state).

Mon, Apr 16, 2:32 PM · Patch-For-Review, User-Elukey, Analytics-Kanban
elukey added a comment to T182924: Refresh zookeeper nodes in eqiad.

Interesting discovery today while testing zookeeper on stretch. I tried to clean up /etc/zookeeper/conf and ran puppet to check if everything was going to be restored or not, and with my great surprise, the zookeeper systemd unit wasn't able to "start". After a big of digging, the culprit seems to be the following:

Mon, Apr 16, 1:09 PM · Patch-For-Review, User-Elukey, Analytics-Kanban

Fri, Apr 13

elukey added a comment to T190566: Decrease the request from iOS app to bohrium.

@chelsyx so now the infrastructure that runs the bohrium host (and hence piwik) is much more stable, we hope to have solved the issues that were causing the host to frequently freeze and not archive data. If I have understood it correctly, the last remaining step is on your side to work on a wider dispatch interval; is my understanding correct? Are there pending actions for Analytics?

Fri, Apr 13, 1:17 PM · Analytics-Kanban, Analytics, Wikipedia-iOS-App-Backlog
elukey added a comment to T182924: Refresh zookeeper nodes in eqiad.

But then zk1-1's state is not, since it keeps repeating the following:

2018-04-12 10:07:55,347 - INFO  [WorkerReceiver[myid=101]:FastLeaderElection@542] - Notification: 102 (n.leader), 0x700010d13 (n.zxid), 0x1 (n.round), LEADING (n.state), 102 (n.sid), 0x7 (n.peerEPoch), LOOKING (my state)
Fri, Apr 13, 10:04 AM · Patch-For-Review, User-Elukey, Analytics-Kanban
elukey moved T189105: Expand the Hadoop Journal nodes from 3 to 5 to improve resiliency from Backlog to Analytics Backlog on the User-Elukey board.
Fri, Apr 13, 7:52 AM · Analytics, User-Elukey
elukey moved T192071: Upgrade deployment-prep appserver fleet to Debian Stretch (using HHVM) from Backlog to Keep an eye on it on the User-Elukey board.
Fri, Apr 13, 7:52 AM · Patch-For-Review, Beta-Cluster-Infrastructure, User-Joe, User-Elukey, HHVM, Operations
elukey moved T191871: Report updater setting log ownership incorrectly (leading to cronspam) from Backlog to Analytics Backlog on the User-Elukey board.
Fri, Apr 13, 7:52 AM · User-Elukey, Analytics
elukey moved T189051: Add trash folder to hadoop from In Progress to Done on the User-Elukey board.
Fri, Apr 13, 7:51 AM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
elukey moved T189691: [EL sanitization] Ensure presence of EL YAML whitelist in analytics1003 from In Progress to Done on the User-Elukey board.
Fri, Apr 13, 7:51 AM · Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, User-Elukey, Analytics
elukey added a comment to T189692: [EL sanitization] Modify mysql purging script to read from the new YAML whitelist.
elukey@db1108:~$ sudo -u eventlogcleaner crontab -l
0 11 * * * /usr/bin/flock --verbose -n /var/lock/eventlogging_cleaner /usr/local/bin/eventlogging_cleaner --whitelist /etc/analytics/sanitization/eventlogging_purging_whitelist.yaml --yaml --older-than 90 --start-ts-file /var/run/eventlogging_cleaner --batch-size 10000 --sleep-between-batches 2  >> /var/log/eventlogging_cleaner/eventlogging_cleaner.log
Fri, Apr 13, 7:51 AM · Patch-For-Review, Analytics-Kanban, Analytics
elukey moved T189692: [EL sanitization] Modify mysql purging script to read from the new YAML whitelist from In Code Review to Done on the Analytics-Kanban board.
Fri, Apr 13, 7:47 AM · Patch-For-Review, Analytics-Kanban, Analytics
elukey added a comment to T189691: [EL sanitization] Ensure presence of EL YAML whitelist in analytics1003.
elukey@analytics1003:~$ ls -l /etc/analytics/sanitization/eventlogging_purging_whitelist.yaml
-r--r--r-- 1 root root 26165 Apr 13 07:44 /etc/analytics/sanitization/eventlogging_purging_whitelist.yaml
Fri, Apr 13, 7:46 AM · Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, User-Elukey, Analytics
elukey moved T189691: [EL sanitization] Ensure presence of EL YAML whitelist in analytics1003 from In Code Review to Done on the Analytics-Kanban board.
Fri, Apr 13, 7:46 AM · Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, User-Elukey, Analytics

Thu, Apr 12

elukey moved T189691: [EL sanitization] Ensure presence of EL YAML whitelist in analytics1003 from Next Up to In Code Review on the Analytics-Kanban board.
Thu, Apr 12, 3:34 PM · Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, User-Elukey, Analytics
elukey claimed T189691: [EL sanitization] Ensure presence of EL YAML whitelist in analytics1003.
Thu, Apr 12, 3:34 PM · Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, User-Elukey, Analytics
elukey added a comment to T182924: Refresh zookeeper nodes in eqiad.

From the consumers point of view (two kafka clusters and one hadoop cluster) I have observed only some non critical logs in one of the hadoop masters when zookeeper broke its session:

Thu, Apr 12, 11:24 AM · Patch-For-Review, User-Elukey, Analytics-Kanban
elukey added a comment to T182924: Refresh zookeeper nodes in eqiad.

Re-tested today the swap of one node in labs (analytics project) to verify again logs and things that might break. Some details about the procedure:

Thu, Apr 12, 11:22 AM · Patch-For-Review, User-Elukey, Analytics-Kanban
elukey moved T189051: Add trash folder to hadoop from Ready to Deploy to Done on the Analytics-Kanban board.
Thu, Apr 12, 8:25 AM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
elukey added a comment to T189051: Add trash folder to hadoop.

Added documentation to https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster#recover_files_deleted_by_mistake_using_the_hdfs_CLI_rm_command?, last step is to send a mail to analytics@ (and possibly research, engineering?) to announce the new feature.

Thu, Apr 12, 8:25 AM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics

Tue, Apr 10

elukey added a comment to T189691: [EL sanitization] Ensure presence of EL YAML whitelist in analytics1003.

@mforns sorry I don't find the whitelist, can you add it in here? Moreover, do you need that puppet also pushes it to hdfs?

Tue, Apr 10, 9:12 AM · Patch-For-Review, Analytics-Kanban, Analytics-EventLogging, User-Elukey, Analytics
elukey moved T188991: Eventlogging mysql consumers inserted rows on the analytics slave (db1108) for two hours from In Progress to Done on the Analytics-Kanban board.
Tue, Apr 10, 9:01 AM · Patch-For-Review, Operations, Analytics-Kanban
elukey added a comment to T188991: Eventlogging mysql consumers inserted rows on the analytics slave (db1108) for two hours.

Created https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/Administration#Mysql_insertion_rate_dropping_to_zero_due_to_db_failures

Tue, Apr 10, 8:58 AM · Patch-For-Review, Operations, Analytics-Kanban
elukey added a comment to T188991: Eventlogging mysql consumers inserted rows on the analytics slave (db1108) for two hours.

A bit of historic context about the why db1108 is not read-only:

Tue, Apr 10, 8:11 AM · Patch-For-Review, Operations, Analytics-Kanban
elukey moved T188991: Eventlogging mysql consumers inserted rows on the analytics slave (db1108) for two hours from Next Up to In Progress on the Analytics-Kanban board.
Tue, Apr 10, 8:02 AM · Patch-For-Review, Operations, Analytics-Kanban
elukey moved T177460: Add the prometheus jmx exporter to all the Zookeeper daemons from In Progress to Done on the User-Elukey board.
Tue, Apr 10, 7:46 AM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics
elukey moved T188719: Upgrade Kafka Burrow to 1.0 from Analytics Backlog to Done on the User-Elukey board.
Tue, Apr 10, 7:46 AM · Analytics-Kanban, Services (watching), User-Elukey, Analytics
elukey added a comment to T191308: Access to stat100x and notebook1003.eqiad.wmnet for Jonas Kress.

So Jonas (user: jk) is already in analytics-privatedata-users, and as far as I can see access is already granted for notebook1003, stat1004 and stat1006. The only one that is not included is stat1005, but it is probably redundant for Jonas' use case (let me know otherwise).

Tue, Apr 10, 7:23 AM · Analytics, Operations, Ops-Access-Requests
elukey updated the task description for T191871: Report updater setting log ownership incorrectly (leading to cronspam).
Tue, Apr 10, 6:39 AM · User-Elukey, Analytics
elukey created T191871: Report updater setting log ownership incorrectly (leading to cronspam).
Tue, Apr 10, 6:38 AM · User-Elukey, Analytics

Mon, Apr 9

elukey changed the point value for T188719: Upgrade Kafka Burrow to 1.0 from 13 to 8.
Mon, Apr 9, 1:29 PM · Analytics-Kanban, Services (watching), User-Elukey, Analytics
elukey moved T188719: Upgrade Kafka Burrow to 1.0 from Next Up to Done on the Analytics-Kanban board.
Mon, Apr 9, 1:29 PM · Analytics-Kanban, Services (watching), User-Elukey, Analytics
elukey edited projects for T188719: Upgrade Kafka Burrow to 1.0, added: Analytics-Kanban; removed Patch-For-Review.
Mon, Apr 9, 1:29 PM · Analytics-Kanban, Services (watching), User-Elukey, Analytics
elukey moved T177460: Add the prometheus jmx exporter to all the Zookeeper daemons from In Progress to Done on the Analytics-Kanban board.
Mon, Apr 9, 11:35 AM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics
elukey updated the task description for T177460: Add the prometheus jmx exporter to all the Zookeeper daemons.
Mon, Apr 9, 11:35 AM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics

Fri, Apr 6

elukey added a comment to T187014: Proxies information gone from Zero portal. Opera mini pageviews geolocating to wrong country.

Nothing varnish-related happened on Feb 6th as far as I can see from the ops SAL: https://tools.wmflabs.org/sal/production?p=0&q=&d=2018-02-06

Fri, Apr 6, 8:18 AM · Zero, Patch-For-Review, Analytics-Data-Quality, Analytics-Kanban, Operations, Traffic, Analytics, Readers-Web-Backlog (Tracking), Mobile, New-Readers
elukey edited projects for T172410: Phase out and replace analytics-store (multisource), added: Analytics; removed Analytics-Kanban.
Fri, Apr 6, 7:12 AM · Analytics, WMDE-Analytics-Engineering, User-Addshore, User-Elukey, Research
elukey moved T189051: Add trash folder to hadoop from Done to Ready to Deploy on the Analytics-Kanban board.
Fri, Apr 6, 7:12 AM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
elukey moved T189051: Add trash folder to hadoop from In Progress to Done on the Analytics-Kanban board.
Fri, Apr 6, 7:12 AM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics

Thu, Apr 5

elukey added a comment to T157435: Review ACLs for the Analytics VLAN.

Since this task has been open for a long time, I'll open a new one when we'll be ready to create the analytics-in6 filter.

Thu, Apr 5, 5:06 PM · Analytics, User-Elukey, Operations, netops

Wed, Apr 4

elukey added a comment to T188719: Upgrade Kafka Burrow to 1.0.

Ok mistery solved after checking with Andrew. The version of https://github.com/julienschmidt/httprouter in Debian is stuck at the 1.1 tag (from 2015), and since then a ton of things changed. https://github.com/julienschmidt/httprouter/issues/207 is open since last year to ask for a 1.2 release (that would probably kick off a new Debian pkg release) but no traction since then, so I think that we'd probably need to backtrack into packing all the Burrow dependencies in our package rather than relying on Debian upstream :(

Wed, Apr 4, 5:04 PM · Analytics-Kanban, Services (watching), User-Elukey, Analytics
elukey closed T181728: Stop using jmx_exporter deployed via scap in favour of Debian package as Resolved.
Wed, Apr 4, 3:23 PM · Patch-For-Review, User-Elukey, User-fgiunchedi, Goal, Operations
elukey closed T181728: Stop using jmx_exporter deployed via scap in favour of Debian package, a subtask of T177197: Export Prometheus-compatible JVM metrics from JVMs in production, as Resolved.
Wed, Apr 4, 3:23 PM · User-Elukey, User-fgiunchedi, Goal, Operations
elukey added a comment to T188719: Upgrade Kafka Burrow to 1.0.

Tried to build burrow 1.0 using all debian dependencies (and not godeps added to the package) but this is what I get:

Wed, Apr 4, 1:45 PM · Analytics-Kanban, Services (watching), User-Elukey, Analytics
elukey added a comment to T181728: Stop using jmx_exporter deployed via scap in favour of Debian package.

After removing /srv/deployment/prometheus I don't see any trace of the jmx exporter jar contained in the dir in lsof -Xd DEL on rdb2001/1007.

Wed, Apr 4, 12:04 PM · Patch-For-Review, User-Elukey, User-fgiunchedi, Goal, Operations
elukey moved T177460: Add the prometheus jmx exporter to all the Zookeeper daemons from Analytics Backlog to In Progress on the User-Elukey board.
Wed, Apr 4, 11:26 AM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics
elukey closed T184794: Fix outstanding bugs preventing the use of prometheus jmx agent for Hive/Oozie as Resolved.

Sadly after a long battle there seems not to be a good way to add prometheus for the hive/oozie jvms, we'll revisit later on when upgrading to newer cdh versions.

Wed, Apr 4, 11:25 AM · Patch-For-Review, Analytics-Kanban, User-Elukey
elukey closed T184794: Fix outstanding bugs preventing the use of prometheus jmx agent for Hive/Oozie, a subtask of T175344: Move away from jmxtrans in favor of prometheus jmx_exporter, as Resolved.
Wed, Apr 4, 11:25 AM · Analytics-Kanban, Patch-For-Review, User-Elukey
elukey added a comment to T183145: Refresh SWAP notebook hardware.

I have been using impyla on notebook1001 to run Hive queries, but this no longer works on notebook1003. Any ideas what might be wrong? See error message below (these two lines work without problem on notebook1001).

Wed, Apr 4, 11:17 AM · Patch-For-Review, Analytics-Kanban

Tue, Apr 3

elukey set the point value for T177460: Add the prometheus jmx exporter to all the Zookeeper daemons to 8.
Tue, Apr 3, 8:31 AM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics
elukey updated the task description for T177460: Add the prometheus jmx exporter to all the Zookeeper daemons.
Tue, Apr 3, 8:31 AM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics

Fri, Mar 30

elukey removed projects from T166081: rack/setup/install conf1004-conf1006: Analytics-Kanban, Patch-For-Review.
Fri, Mar 30, 3:35 PM · User-Joe, Operations
elukey moved T177460: Add the prometheus jmx exporter to all the Zookeeper daemons from Next Up to In Progress on the Analytics-Kanban board.
Fri, Mar 30, 3:34 PM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics
elukey added a project to T177460: Add the prometheus jmx exporter to all the Zookeeper daemons: Analytics-Kanban.
Fri, Mar 30, 3:34 PM · Analytics-Kanban, Patch-For-Review, User-Elukey, Analytics
elukey moved T189051: Add trash folder to hadoop from Next Up to In Progress on the Analytics-Kanban board.
Fri, Mar 30, 3:34 PM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
elukey added a comment to T189051: Add trash folder to hadoop.

Tested the two values that I've set in the above patch in labs:

Fri, Mar 30, 1:31 PM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
elukey moved T185195: Apache reload fails on stretch-based app servers from Ops Backlog to Keep an eye on it on the User-Elukey board.
Fri, Mar 30, 12:12 PM · Patch-For-Review, Operations, User-Elukey