Upgrade python-kafka to 1.4.7
Closed, ResolvedPublic8 Estimated Story Points
Actions

Assigned To

Authored By

	elukey
	Oct 7 2019, 10:22 AM

Description

In T222941 we are testing python-kafka 1.4.7 on eventlog1002, that seems working well. The new version doesn't need the custom patch that we added to the deb since it is already included, so it would be great to upgrade all consumers of the package:

https://debmonitor.wikimedia.org/packages/python-kafka

The Analytics hosts are the predominant set of hosts to upgrade, but there are also webperf ones that @Gilles upgraded the last time (T221848). It would also be great to know what services are relying on python-kafka (as opposed to python3-kafka) since it would be great to move to python3 everywhere (see https://www.python.org/doc/sunset-python-2/).

Eventlogging is moving to Python3 in T233231.

Related Objects

Mentioned Here: T233231: Upgrade eventlogging to Python 3
T221848: Upgrade python-kafka
T222941: Eventlogging processors are frequently failing heartbeats causing consumer group rebalances

Event Timeline

elukey triaged this task as Medium priority.Oct 7 2019, 10:22 AM

elukey created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 7 2019, 10:22 AM

elukey updated the task description. (Show Details)Oct 7 2019, 10:24 AM

MoritzMuehlenhoff subscribed.Oct 7 2019, 12:44 PM

root@install1002:/srv/wikimedia# reprepro lsbycomponent python-kafka
python-kafka | 1.4.3-1~jessie1 |  jessie-wikimedia | main | amd64, i386, source
python-kafka |         1.4.7-1 | stretch-wikimedia | main | amd64, i386, source

root@install1002:/srv/wikimedia# reprepro lsbycomponent python3-kafka
python3-kafka | 1.4.3-1~jessie1 |  jessie-wikimedia | main | amd64, i386
python3-kafka |         1.4.7-1 | stretch-wikimedia | main | amd64, i386

root@install1002:/srv/wikimedia# reprepro lsbycomponent python3-kafka
python3-kafka | 1.4.3-1~jessie1 |  jessie-wikimedia | main | amd64, i386
python3-kafka |         1.4.7-1 | stretch-wikimedia | main | amd64, i386
python3-kafka |         1.4.7-1 |  buster-wikimedia | main | amd64, i386

Milimetric raised the priority of this task from Medium to High.Oct 7 2019, 4:05 PM

Milimetric moved this task from Incoming to Operational Excellence on the Analytics board.

Milimetric added a project: Analytics-Kanban.

• Gilles moved this task from Inbox, needs triage to Radar on the Performance-Team board.Oct 7 2019, 7:57 PM

• Gilles edited projects, added Performance-Team (Radar); removed Performance-Team.

Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.Oct 8 2019, 12:23 AM

elukey moved this task from Next Up to In Progress on the Analytics-Kanban board.Oct 8 2019, 5:40 AM

@Gilles how should we coordinate for the deployment?

elukey moved this task from In Progress to Next Up on the Analytics-Kanban board.Oct 14 2019, 1:55 PM

@elukey what does your Monday look like next week (21st)? Preferably during EU morning for me.

In T234808#5584214, @Gilles wrote:

@elukey what does your Monday look like next week (21st)? Preferably during EU morning for me.

I'll be out next week for ApacheCon, maybe the week after? Or tomorrow morning :)

I can't tomorrow and the week after I'm on vacation. @Krinkle could you take care of this the week of Oct 27?

Ack. @elukey Let me know what day/time works best next week. I'll be on US Eastern Time that week.

Krinkle claimed this task.Oct 25 2019, 12:03 AM

In T234808#5605049, @Krinkle wrote:

Ack. @elukey Let me know what day/time works best next week. I'll be on US Eastern Time that week.

In the EU afternoon any time is good, just ping me on IRC when you are caffeinated and ready to work and I'll upgrade the package :)

Mentioned in SAL (#wikimedia-operations) [2019-10-29T14:29:45Z] <elukey> upgrade python-kafka on webperf[12]001 - T234808

elukey set the point value for this task to 8.Oct 29 2019, 2:42 PM

elukey moved this task from Next Up to Done on the Analytics-Kanban board.

elukey claimed this task.Oct 29 2019, 2:45 PM

I've restarted coal, navtiming and stats on webperf1001 and webperf2001 and verified in the journals that the restarts happened and that they resumed their duties without issues.

Restart procedure now documented at https://wikitech.wikimedia.org/wiki/Performance/Runbook/Webperf-processor_services.

Upgrade python-kafka to 1.4.7Closed, ResolvedPublic8 Estimated Story PointsActions

Description

Related Objects

Event Timeline

Upgrade python-kafka to 1.4.7
Closed, ResolvedPublic8 Estimated Story Points
Actions