Page MenuHomePhabricator

elukey (Luca Toscano)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Jan 5 2016, 9:54 PM (193 w, 6 d)
Availability
Available
LDAP User
Unknown
MediaWiki User
LToscano (WMF) [ Global Accounts ]

Recent Activity

Today

elukey moved T204736: Move Analytics Report Updater to Python 3 from Next Up to In Code Review on the Analytics-Kanban board.
Tue, Sep 24, 6:58 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey claimed T204736: Move Analytics Report Updater to Python 3.
Tue, Sep 24, 6:58 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey added a comment to T204736: Move Analytics Report Updater to Python 3.

https://gerrit.wikimedia.org/r/#/c/analytics/reportupdater/+/537268/ is ready for a first review, testing with real data has not been done yet.

Tue, Sep 24, 6:58 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey added a comment to T232349: Disable production EventLogging analytics MySQL consumers.

Something strange happened today: I noticed in icinga that the eventlogging mysql insertion rate alarm was in UNKNOWN state, so I checked the graphs and the m4 consumer stopped sending data around the 21st:

Tue, Sep 24, 5:56 AM · Analytics, Analytics-EventLogging
elukey moved T204735: Move the Analytics Refinery to Python 3 from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Tue, Sep 24, 5:43 AM · Analytics-Kanban, Analytics
elukey set the point value for T204735: Move the Analytics Refinery to Python 3 to 8.
Tue, Sep 24, 5:42 AM · Analytics-Kanban, Analytics

Yesterday

elukey added a comment to T232069: analytics1045 - RAID failure and /var/lib/hadoop/data/j can't be mounted.

This host can keep running with one disk less, fixed it with https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/538303/

Mon, Sep 23, 3:23 PM · ops-eqiad, DC-Ops, Analytics, Operations, Analytics-Cluster
elukey added a comment to T226698: Allow all Analytics tools to work with Kerberos auth.

Now I get it thanks, this is what we already do for other datasets, so it is not a new use case. The rsync should be from the dumps hosts, that pulls data to them from stat1007. Eventually the data will be accessible via datasets.w.o or dumps.w.o right?

Mon, Sep 23, 1:59 PM · Patch-For-Review, Analytics-Kanban, User-Elukey, Analytics
elukey added a comment to T231067: Install Debian Buster on Hadoop.

T233604 tracks the work to import the openjdk-8 package to a special component for Debian Buster, thanks Moritz!

Mon, Sep 23, 12:47 PM · Analytics
elukey added a subtask for T231067: Install Debian Buster on Hadoop: T233604: Create OpenJDK 8 packages for Buster.
Mon, Sep 23, 12:46 PM · Analytics
elukey added a parent task for T233604: Create OpenJDK 8 packages for Buster: T231067: Install Debian Buster on Hadoop.
Mon, Sep 23, 12:46 PM · Operations
elukey added a comment to T231858: Archive data on eventlogging MySQL to analytics replica before decomisioning .

Yes I am aware, but in case of a big disaster (like host completely broken) we could think about an alternative starting from the db dumps taken before. If one of the three dbstore node goes down indefinitely now we'll have to either order hardware or ask for a spare replacement, I see the same problem but in a different context. Given the fact that the past 1.5y of eventlogging data is already in Hadoop and people are using it, I don't see a huge problem in keeping only one db host.

Mon, Sep 23, 12:44 PM · Analytics-Kanban, Analytics, Analytics-EventLogging
elukey added a comment to T233604: Create OpenJDK 8 packages for Buster.

Work already done to keep archives happy:

Mon, Sep 23, 10:57 AM · Operations
elukey awarded T233604: Create OpenJDK 8 packages for Buster a Love token.
Mon, Sep 23, 10:56 AM · Operations
elukey added a comment to T231858: Archive data on eventlogging MySQL to analytics replica before decomisioning .

There is also another option that we could think about to avoid spending a ton of time on this. Since we want to allow people to query the log database for historical purposes, it seems a waste of human resources to move it from one db host to another one (reducing space available for other important databases like the wiki replicas). We could simply stop pushing data to db1107, set it to spare to allow the Data Persistence team to repurpose it in other ways and keep db1108 untouched. Our users will still have a "known" place to query for historical data, and we'll not have to advertise to them how to migrate their scripts etc.. We'll also don't spend time on migrating databases (involving Manuel as well).

Mon, Sep 23, 10:49 AM · Analytics-Kanban, Analytics, Analytics-EventLogging
elukey added a comment to T231858: Archive data on eventlogging MySQL to analytics replica before decomisioning .

Only on db1107 there are _log_sql_aff* files for up 507G, that are not on db1108. No idea what they are about.

Mon, Sep 23, 10:42 AM · Analytics-Kanban, Analytics, Analytics-EventLogging
elukey added a comment to T233591: Eventlogging is not compatible with python3?.

Thanks a lot for the task :)

Mon, Sep 23, 9:40 AM · Analytics, Technical-Debt (Deprecation), Analytics-EventLogging
elukey added a comment to T204735: Move the Analytics Refinery to Python 3.
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:   File "/srv/deployment/analytics/refinery/bin/refinery-drop-older-than", line 595, in <module>
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:     main(args)
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:   File "/srv/deployment/analytics/refinery/bin/refinery-drop-older-than", line 347, in main
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:     execute is not None)
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:   File "/srv/deployment/analytics/refinery/bin/refinery-drop-older-than", line 195, in remove_directories
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:     candidate_paths = hdfs.ls(glob, include_children=False)
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:   File "/srv/deployment/analytics/refinery/python/refinery/hdfs.py", line 59, in ls
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:     check_return_code=False
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:   File "/srv/deployment/analytics/refinery/python/refinery/hdfs.py", line 60, in <listcomp>
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]:     ).splitlines() if not line.startswith(b'Found ')
Sep 23 08:33:35 an-coord1001 refinery-drop-older-than[40168]: TypeError: startswith first arg must be str or a tuple of str, not bytes
Mon, Sep 23, 9:24 AM · Analytics-Kanban, Analytics
elukey moved T204735: Move the Analytics Refinery to Python 3 from Ready to Deploy to In Code Review on the Analytics-Kanban board.
Mon, Sep 23, 8:40 AM · Analytics-Kanban, Analytics

Fri, Sep 20

elukey added a comment to T226698: Allow all Analytics tools to work with Kerberos auth.

@elukey
Yes! MediaWiki history dumps need to be rsync'd from HDFS mount.

Fri, Sep 20, 2:02 PM · Patch-For-Review, Analytics-Kanban, User-Elukey, Analytics
elukey moved T204737: Verify what Python 2 packages deployed to Analytics hosts are needed from Next Up to Done on the Analytics-Kanban board.
Fri, Sep 20, 1:34 PM · Analytics-Kanban, Analytics
elukey added a project to T204737: Verify what Python 2 packages deployed to Analytics hosts are needed: Analytics-Kanban.
Fri, Sep 20, 1:34 PM · Analytics-Kanban, Analytics
elukey set the point value for T204737: Verify what Python 2 packages deployed to Analytics hosts are needed to 3.
Fri, Sep 20, 1:34 PM · Analytics-Kanban, Analytics
elukey added a comment to T204737: Verify what Python 2 packages deployed to Analytics hosts are needed.

I think that the task can be closed, please re-open if necessary!

Fri, Sep 20, 1:34 PM · Analytics-Kanban, Analytics
elukey updated subscribers of T226698: Allow all Analytics tools to work with Kerberos auth.

@mforns you have another use case that needs rsync from the HDFS mount right?

Fri, Sep 20, 1:28 PM · Patch-For-Review, Analytics-Kanban, User-Elukey, Analytics
elukey added a project to T231208: Version analytics meta mysql database backup : User-Elukey.
Fri, Sep 20, 1:23 PM · User-Elukey, Patch-For-Review, Analytics-Kanban, Analytics
elukey moved T204735: Move the Analytics Refinery to Python 3 from In Progress to Ready to Deploy on the Analytics-Kanban board.
Fri, Sep 20, 1:23 PM · Analytics-Kanban, Analytics
elukey added a comment to T231858: Archive data on eventlogging MySQL to analytics replica before decomisioning .

Some data:

Fri, Sep 20, 1:22 PM · Analytics-Kanban, Analytics, Analytics-EventLogging
elukey moved T227257: Move refinery to hive 2 actions from Kerberos to Done on the User-Elukey board.
Fri, Sep 20, 11:35 AM · Patch-For-Review, Analytics-Kanban, User-Elukey, Analytics
elukey added a comment to T204735: Move the Analytics Refinery to Python 3.

After checking the code I think that the problem is in the util.sh() function, that with python2 returns a text string meanwhile with Python3 a bytes string. Tested the behavior of https://gerrit.wikimedia.org/r/538235 with python 3 and it works as expected.

Fri, Sep 20, 10:57 AM · Analytics-Kanban, Analytics
elukey moved T227257: Move refinery to hive 2 actions from In Progress to Done on the Analytics-Kanban board.
Fri, Sep 20, 9:31 AM · Patch-For-Review, Analytics-Kanban, User-Elukey, Analytics
elukey added a comment to T227257: Move refinery to hive 2 actions.

To keep archives happy, Joseph also followed up with:

Fri, Sep 20, 9:31 AM · Patch-For-Review, Analytics-Kanban, User-Elukey, Analytics
elukey set the point value for T227257: Move refinery to hive 2 actions to 21.
Fri, Sep 20, 9:29 AM · Patch-For-Review, Analytics-Kanban, User-Elukey, Analytics

Thu, Sep 19

elukey added a comment to T204735: Move the Analytics Refinery to Python 3.

@mforns another interesting use case:

Thu, Sep 19, 10:41 AM · Analytics-Kanban, Analytics
elukey added a comment to T232615: Enable gzip compression for interface icon SVGs served by MediaWiki.

Reporting a discussion happened on IRC: the change looks good, but it seems that Varnish/ATS explicitly unsed the Accept-Encoding request header before calling mediawiki:

Thu, Sep 19, 9:27 AM · Operations, Traffic, Patch-For-Review, Performance-Team
elukey added a comment to T225128: Move cloudvirtan* hardware out of CloudVPS back into production Analytics VLAN..

@Cmjohnson @Jclark-ctr there is one last problem - an-presto1005:

Thu, Sep 19, 8:33 AM · Analytics-Kanban, ops-eqiad, Operations, netops, Analytics
elukey added a comment to T225128: Move cloudvirtan* hardware out of CloudVPS back into production Analytics VLAN..
elukey@asw2-b-eqiad# show | compare
[edit interfaces xe-4/0/5]
-   description cloudvirtan1004;
+   description an-presto1004;
Thu, Sep 19, 8:17 AM · Analytics-Kanban, ops-eqiad, Operations, netops, Analytics
elukey added a comment to T225128: Move cloudvirtan* hardware out of CloudVPS back into production Analytics VLAN..
elukey@asw2-a-eqiad# show | compare
[edit interfaces xe-2/0/24]
-   description cloudvirtan1002;
+   description an-presto1002;
[edit interfaces xe-4/0/37]
-   description cloudvirtan1005;
+   description an-presto1005;
Thu, Sep 19, 8:12 AM · Analytics-Kanban, ops-eqiad, Operations, netops, Analytics
elukey added a comment to T204735: Move the Analytics Refinery to Python 3.

Nice! Thanks a lot @mforns.. I think that we might be able to quickly deploy refinery today (that wouldn't even require a deploy to hdfs) and close this task.

Thu, Sep 19, 7:48 AM · Analytics-Kanban, Analytics
elukey added a comment to T232349: Disable production EventLogging analytics MySQL consumers.

This look related,
https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=eventlog1002&service=Check+systemd+state
CRITICAL - degraded: The system is operational but one or more units failed.

eventlog1002:~$ sudo systemctl
[...]
● eventlogging-consumer@mysql-eventbus.service not-found failed failed    eventl
Thu, Sep 19, 7:44 AM · Analytics, Analytics-EventLogging
elukey updated subscribers of T231858: Archive data on eventlogging MySQL to analytics replica before decomisioning .

I asked some info to @Marostegui today and we might need to do some work to make this happen.

Thu, Sep 19, 7:42 AM · Analytics-Kanban, Analytics, Analytics-EventLogging

Wed, Sep 18

elukey added a comment to T204735: Move the Analytics Refinery to Python 3.

help(bytes) in python2 leads to help(str), meanwhile on python3:

Wed, Sep 18, 5:20 PM · Analytics-Kanban, Analytics
elukey added a comment to T204735: Move the Analytics Refinery to Python 3.
>>> hash_args = {'a': 'a', 'b': 'b'}
>>> hash_message = bytes(sorted(hash_args.items()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object cannot be interpreted as an integer
Wed, Sep 18, 4:54 PM · Analytics-Kanban, Analytics
elukey added a comment to T204735: Move the Analytics Refinery to Python 3.

So it seems that the refinery-drop-older-than script has some issues with python3:

Wed, Sep 18, 4:44 PM · Analytics-Kanban, Analytics
elukey moved T204735: Move the Analytics Refinery to Python 3 from Ready to Deploy to In Progress on the Analytics-Kanban board.
Wed, Sep 18, 4:41 PM · Analytics-Kanban, Analytics
elukey moved T204735: Move the Analytics Refinery to Python 3 from In Code Review to Ready to Deploy on the Analytics-Kanban board.
Wed, Sep 18, 3:58 PM · Analytics-Kanban, Analytics
elukey created T233231: Upgrade eventlogging to Python 3.
Wed, Sep 18, 3:57 PM · Analytics
elukey added a comment to T227025: (Need By: August 31) rack/setup/install (3) new zookeeper nodes.

I followed up with @Jclark-ctr and it seems that there is only one serial port available for the server (so we can't really change any cabling). @Cmjohnson the serial console on all hosts is still broken, let's sync up when you have time / are back :)

Wed, Sep 18, 2:43 PM · User-Elukey, Operations, ops-eqiad
elukey added a comment to T225128: Move cloudvirtan* hardware out of CloudVPS back into production Analytics VLAN..

Ok so current status:

Wed, Sep 18, 2:41 PM · Analytics-Kanban, ops-eqiad, Operations, netops, Analytics
elukey added a comment to T225128: Move cloudvirtan* hardware out of CloudVPS back into production Analytics VLAN..
elukey@asw2-b-eqiad# show | compare
[edit interfaces interface-range vlan-cloud-hosts1-b-eqiad]
-    member xe-4/0/5;
[edit interfaces interface-range vlan-analytics1-b-eqiad]
     member ge-5/0/7 { ... }
+    member xe-4/0/5;
Wed, Sep 18, 2:02 PM · Analytics-Kanban, ops-eqiad, Operations, netops, Analytics
elukey added a comment to T225128: Move cloudvirtan* hardware out of CloudVPS back into production Analytics VLAN..

Proposed fix for asw2-b:

Wed, Sep 18, 1:47 PM · Analytics-Kanban, ops-eqiad, Operations, netops, Analytics
elukey added a comment to T225128: Move cloudvirtan* hardware out of CloudVPS back into production Analytics VLAN..

Thanks to the awesome work of @Jclark-ctr an-presto1001 and an-presto1003 are now reimaged, but an-presto1004 is still not working. I think the problem is:

Wed, Sep 18, 1:46 PM · Analytics-Kanban, ops-eqiad, Operations, netops, Analytics
elukey added a comment to T199736: Help accessing SWAP for research collaborators.

done, followed up in the task :)

Wed, Sep 18, 12:21 PM · Research
elukey added a comment to T232707: Requesting access to analytics cluster for Martin Gerlach.

@MGerlach done! Please take a moment to review https://wikitech.wikimedia.org/wiki/LDAP/Groups#wmf_group, since your account is now able to see a lot of sensitive material. Use all these new powers with responsibility :)

Wed, Sep 18, 12:20 PM · Analytics, Operations, SRE-Access-Requests
elukey added a comment to T220639: Show IPs matching a list of IP subnets in Webrequest data.

@ayounsi should we keep this task open?

Wed, Sep 18, 9:38 AM · User-Elukey, Analytics
elukey moved T225297: Create a spicerack recipe to reboot the hadoop worker nodes from In Progress to Done on the User-Elukey board.
Wed, Sep 18, 9:38 AM · Analytics-Kanban, Analytics, User-Elukey
elukey moved T233141: setup/install krb1001/WMF5173 from Backlog to Keep an eye on it on the User-Elukey board.
Wed, Sep 18, 9:38 AM · Operations, User-Elukey, Analytics
elukey moved T233142: setup/install codfw kerbos node WMF6577 from Backlog to Keep an eye on it on the User-Elukey board.
Wed, Sep 18, 9:38 AM · Operations, User-Elukey, Analytics
elukey reassigned T233142: setup/install codfw kerbos node WMF6577 from elukey to RobH.

Thanks a lot!

Wed, Sep 18, 7:57 AM · Operations, User-Elukey, Analytics
elukey reassigned T233141: setup/install krb1001/WMF5173 from elukey to RobH.

Thanks a lot!

Wed, Sep 18, 7:56 AM · Operations, User-Elukey, Analytics
elukey moved T225297: Create a spicerack recipe to reboot the hadoop worker nodes from In Code Review to Done on the Analytics-Kanban board.
Wed, Sep 18, 7:46 AM · Analytics-Kanban, Analytics, User-Elukey
elukey removed a watcher for MediaWiki-Cache: elukey.
Wed, Sep 18, 7:26 AM
elukey changed the point value for T225297: Create a spicerack recipe to reboot the hadoop worker nodes from 0 to 8.
Wed, Sep 18, 6:28 AM · Analytics-Kanban, Analytics, User-Elukey
elukey moved T204735: Move the Analytics Refinery to Python 3 from Next Up to In Code Review on the Analytics-Kanban board.
Wed, Sep 18, 6:28 AM · Analytics-Kanban, Analytics
elukey claimed T204735: Move the Analytics Refinery to Python 3.
Wed, Sep 18, 6:27 AM · Analytics-Kanban, Analytics
elukey added a project to T204735: Move the Analytics Refinery to Python 3: Analytics-Kanban.
Wed, Sep 18, 6:27 AM · Analytics-Kanban, Analytics
elukey added a comment to T233073: Test if Hue can run with Python3.

From upstream, it seems that the fixed version of Hue (4.3) will only be available in CDH6.

Wed, Sep 18, 6:23 AM · Analytics-Kanban

Tue, Sep 17

elukey closed T232821: Check home leftovers of atgomez as Resolved.

Done!

Tue, Sep 17, 11:36 AM · Analytics
elukey triaged T232821: Check home leftovers of atgomez as Normal priority.
Tue, Sep 17, 11:34 AM · Analytics
elukey moved T232821: Check home leftovers of atgomez from Operational Excellence to Ops Week on the Analytics board.
Tue, Sep 17, 11:34 AM · Analytics
elukey added a comment to T215775: Check home leftovers of ISI researchers.

Ping @leila @Isaac :)

Tue, Sep 17, 11:34 AM · Research, Analytics
elukey added a comment to T227025: (Need By: August 31) rack/setup/install (3) new zookeeper nodes.

Today I tested Redirection after boot set to enabled on an-coord1001's bios but I didn't resolve the problem, the mgmt console is not available. Given the workaround that I used to make it work, is it possible that the serial cabling for the an-conf nodes is inverted between com1/com2? Just trying to think about what could be the problem..

Tue, Sep 17, 11:03 AM · User-Elukey, Operations, ops-eqiad
elukey added a comment to T227288: eqiad: 1 misc node for the Kerberos KDC service.

@RobH let me know if I can help with the host repurpose (also with the codfw one), I can take care of the DNS/puppet/DHCP/etc.. steps :)

Tue, Sep 17, 10:45 AM · hardware-requests, Operations, User-Elukey, Analytics
elukey moved T226035: Dropping data from druid takes down aqs hosts from In Progress to Stalled on the User-Elukey board.
Tue, Sep 17, 10:26 AM · Patch-For-Review, User-Elukey, Analytics-Kanban, Analytics
elukey assigned T233080: Decommission analytics1032 to RobH.
Tue, Sep 17, 9:56 AM · decommission, Operations
elukey updated the task description for T233080: Decommission analytics1032.
Tue, Sep 17, 9:56 AM · decommission, Operations
elukey created T233080: Decommission analytics1032.
Tue, Sep 17, 8:04 AM · decommission, Operations
elukey renamed T227485: Decommission analytics10[28-31,33-41] from Decommission analytics10[28-41] to Decommission analytics10[28-31,33-41].
Tue, Sep 17, 8:03 AM · decommission, Operations
elukey raised the priority of T204736: Move Analytics Report Updater to Python 3 from Low to Normal.
Tue, Sep 17, 7:33 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey added a comment to T204736: Move Analytics Report Updater to Python 3.

This is a bit harder than expected, since tests will need some adjustment to:

Tue, Sep 17, 7:33 AM · Analytics-Kanban, Patch-For-Review, Analytics
elukey added a comment to T233073: Test if Hue can run with Python3.

Opened https://github.com/cloudera/hue/issues/968

Tue, Sep 17, 6:41 AM · Analytics-Kanban
elukey updated the task description for T233073: Test if Hue can run with Python3.
Tue, Sep 17, 6:37 AM · Analytics-Kanban
elukey updated the task description for T233073: Test if Hue can run with Python3.
Tue, Sep 17, 6:34 AM · Analytics-Kanban
elukey created T233073: Test if Hue can run with Python3.
Tue, Sep 17, 6:27 AM · Analytics-Kanban
elukey updated the task description for T227543: b8-eqiad pdu refresh (Thursday 10/31 @11am UTC).
Tue, Sep 17, 6:04 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227542: b7-eqiad pdu refresh (Tuesday 11/5 @11am UTC).
Tue, Sep 17, 6:03 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227541: b6-eqiad pdu refresh (Tuesday 9/10 @11am UTC).
Tue, Sep 17, 6:02 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227540: b4-eqiad pdu refresh (Thursday 10/24 @11am UTC).
Tue, Sep 17, 6:02 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227539: b3-eqiad pdu refresh (Tuesday 9/17 @11am UTC).
Tue, Sep 17, 6:01 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227539: b3-eqiad pdu refresh (Tuesday 9/17 @11am UTC).
Tue, Sep 17, 6:00 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227538: b2-eqiad pdu refresh (Tuesday 10/29 @11am UTC).
Tue, Sep 17, 5:58 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227538: b2-eqiad pdu refresh (Tuesday 10/29 @11am UTC).
Tue, Sep 17, 5:58 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227536: b1-eqiad pdu refresh (Thursday 10/10 @11am UTC).
Tue, Sep 17, 5:58 AM · DC-Ops, Operations, ops-eqiad
elukey added a comment to T227142: a6-eqiad pdu refresh (Tuesday 10/22 @11am UTC).

I also added the info about analytics hosts and flipped the requirement of depooling for memcached to "no", since we should do it only if things go on fire :)

Tue, Sep 17, 5:57 AM · DC-Ops, Operations, ops-eqiad
elukey updated the task description for T227142: a6-eqiad pdu refresh (Tuesday 10/22 @11am UTC).
Tue, Sep 17, 5:56 AM · DC-Ops, Operations, ops-eqiad
elukey added a comment to T227142: a6-eqiad pdu refresh (Tuesday 10/22 @11am UTC).

@elukey for labsdb1012 your Team would need to let us know if MySQL can be stopped for this maintenance (just in case there is powerloss, better to have MySQL stopped, as labs hosts do not have GTID enabled and the risk of corruption can be higher).

Tue, Sep 17, 5:53 AM · DC-Ops, Operations, ops-eqiad

Mon, Sep 16

elukey added a comment to T204735: Move the Analytics Refinery to Python 3.

@mforns checked puppet without a simple grep, and:

Mon, Sep 16, 2:19 PM · Analytics-Kanban, Analytics
elukey added a comment to T204737: Verify what Python 2 packages deployed to Analytics hosts are needed.

From the response in the mailing list it seems that all our users have already migrated to python 3 a while ago. Let's keep this task open for a bit more to allow people to chime in if necessary.

Mon, Sep 16, 12:43 PM · Analytics-Kanban, Analytics
elukey updated subscribers of T204735: Move the Analytics Refinery to Python 3.

@mforns can you check in the task's description the seems unused in puppet? IIUC those got replaced by refinery-drop-older-than, so we could possibly delete some of those? Also, can we run refinery-drop-older-than with python3? :)

Mon, Sep 16, 10:26 AM · Analytics-Kanban, Analytics
elukey updated the task description for T204735: Move the Analytics Refinery to Python 3.
Mon, Sep 16, 10:23 AM · Analytics-Kanban, Analytics