Package udplog for Buster
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	taavi
	Mar 4 2021, 8:10 AM

Description

To allow updating mwlog servers to Buster (T224565 and T276419), the udplog package should be available for Debian Buster.

Related Objects
Search...

Status	Subtype	Assigned	Task
Invalid		None	T197804 Puppet: forbid new Python2 code
Open		None	T218426 Upgrade various Cloud VPS Python 2 scripts to Python 3
Resolved	BUG REPORT	• Bstorm	T218423 Add python 3 packages to openstack::clientpackages::common
Resolved		MoritzMuehlenhoff	T232677 Remove support for Debian Jessie in Cloud Services
Duplicate		None	T236575 "deployment-prep" Cloud VPS project jessie deprecation
Resolved		None	T218729 Migrate deployment-prep away from Debian Jessie to Debian Stretch/Buster
Resolved		taavi	T276419 Replace deployment-fluorine02 with a Buster host
Resolved		MoritzMuehlenhoff	T224549 Track remaining jessie systems in production
Resolved		herron	T224565 Migrate mwlog/udp2log servers to Buster
Resolved		taavi	T276421 Package udplog for Buster
Resolved		hashar	T276422 Unarchive analytics/udplog repository

Event Timeline

taavi created this task.Mar 4 2021, 8:10 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 4 2021, 8:10 AM

taavi mentioned this in T276419: Replace deployment-fluorine02 with a Buster host.Mar 4 2021, 8:11 AM

Legoktm added a subtask: T276422: Unarchive analytics/udplog repository.Mar 4 2021, 8:12 AM

Legoktm added parent tasks: T224565: Migrate mwlog/udp2log servers to Buster, T276419: Replace deployment-fluorine02 with a Buster host.

The Debian packaging needs a refresh, and it'll need a few code tweaks to compile on Buster:

diff --git a/srcmisc/packet-loss.cpp b/srcmisc/packet-loss.cpp
index 01e13a2..861cfdc 100644
--- a/srcmisc/packet-loss.cpp
+++ b/srcmisc/packet-loss.cpp
@@ -1,5 +1,5 @@
 #include <iostream>
-#include <boost/tr1/unordered_map.hpp>
+#include <boost/unordered/unordered_map.hpp>
 #include <map>
 #include <stdint.h>
 #include <boost/lexical_cast.hpp>
@@ -10,7 +10,7 @@
 
 using std::strtok;
 
-typedef std::tr1::unordered_map<std::string, uint64_t>::iterator SeqIterator;
+typedef boost::unordered::unordered_map<std::string, uint64_t>::iterator SeqIterator;
 
 struct HostData {
        HostData() : received(1), sent(0) {}
@@ -21,7 +21,7 @@ struct HostData {
        // Number of packets sent, estimated from sequence numbers (unsampled)
root@013a033d2822:/udplog# git diff
diff --git a/srcmisc/packet-loss.cpp b/srcmisc/packet-loss.cpp
index 01e13a2..861cfdc 100644
--- a/srcmisc/packet-loss.cpp
+++ b/srcmisc/packet-loss.cpp
@@ -1,5 +1,5 @@
 #include <iostream>
-#include <boost/tr1/unordered_map.hpp>
+#include <boost/unordered/unordered_map.hpp>
 #include <map>
 #include <stdint.h>
 #include <boost/lexical_cast.hpp>
@@ -10,7 +10,7 @@
 
 using std::strtok;
 
-typedef std::tr1::unordered_map<std::string, uint64_t>::iterator SeqIterator;
+typedef boost::unordered::unordered_map<std::string, uint64_t>::iterator SeqIterator;
 
 struct HostData {
        HostData() : received(1), sent(0) {}
@@ -21,7 +21,7 @@ struct HostData {
        // Number of packets sent, estimated from sequence numbers (unsampled)
        int64_t sent;
 };
-typedef std::tr1::unordered_map<std::string, HostData>::iterator HostIterator;
+typedef boost::unordered::unordered_map<std::string, HostData>::iterator HostIterator;
 
 struct SampleData {
        SampleData() : total(0), outOfOrder(0), invalid(0) {}
@@ -35,7 +35,7 @@ struct SampleData {
        // Number of sampled lines which were invalid
        int64_t invalid;
 
-       std::tr1::unordered_map<std::string, HostData> hosts;
+       boost::unordered::unordered_map<std::string, HostData> hosts;
 
        void Report(const struct tm * timeStruct, int sampleRate);
        void PrintRatio(int64_t numerator, int64_t denominator, double numeratorError);
@@ -50,7 +50,7 @@ int main(int argc, char** argv)
        using namespace std;
 
        const size_t bufSize = 65536;
-       std::tr1::unordered_map<string, uint64_t> seqs;
+       boost::unordered::unordered_map<string, uint64_t> seqs;
        char buffer[bufSize];
        int sampleRate;

Also the init script should be rewritten as a systemd service.

Ah, I hadn't see Kunal's update on the task until now, but my code changes to fix with current Boost are practically the same (sans the that is still uses an init script), I've just uploaded 1.8.5+deb10u1 to buster-wikimedia.org

JMeybohm triaged this task as Medium priority.Mar 4 2021, 9:25 AM

hashar changed the status of subtask T276422: Unarchive analytics/udplog repository from Open to Stalled.Mar 4 2021, 9:37 AM

Peachey88 added a project: Packaging.Mar 4 2021, 11:50 AM

Mentioned in SAL (#wikimedia-operations) [2021-03-05T06:17:16Z] <legoktm> uploaded udplog 1.9 (buster-wikimedia) to apt.wikimedia.org (T276421)

Legoktm changed the status of subtask T276422: Unarchive analytics/udplog repository from Stalled to Open.Mar 5 2021, 6:17 AM

@Majavah refreshed the packaging (https://gerrit.wikimedia.org/r/c/analytics/udplog/+/668451), switching to dh, a native package and got it to pass lintian with no errors. I uploaded that as 1.9, it works fine on beta cluster, but updating produced some warnings:

Preparing to unpack .../archives/udplog_1.9_amd64.deb ...
Unpacking udplog (1.9) over (1.8-5+deb10u1) ...
dpkg: warning: udplog: conffile '/etc/udp2log' is not a plain file or symlink (= '/etc/udp2log')
Setting up udplog (1.9) ...
dpkg: warning: udplog: conffile '/etc/udp2log' is not a plain file or symlink (= '/etc/udp2log')
Processing triggers for systemd (241-7~deb10u6) ...

Looking into that, the package creates /etc/udp2log as a config file, while puppet creates it as a directory and puts the config as individual files:

This is a pre-existing problem though, so I think we can file a separate task for that and move on.