Page MenuHomePhabricator
Create Task
Maniphest T174577

Request for python package csvsort on stat1005.eqiad.wmnet
Closed, ResolvedPublic
Actions

Description

I would like to request the installation of the python package csvsort on stat1005.eqiad.wmnet. We generate a lot of different .tsv-files while analysing the SPARQL-Logs and they are sometimes too large to sort in memory.

Event Timeline

Adrian_Bielefeldt created this task.Aug 30 2017, 4:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 30 2017, 4:08 PM
faidon renamed this task from Request for python package csvsort on stat1005.equiad.wmnet to Request for python package csvsort on stat1005.eqiad.wmnet.Aug 30 2017, 4:09 PM
faidon edited projects, added Analytics; removed SRE.
faidon updated the task description. (Show Details)
Ottomata subscribed.Aug 31 2017, 4:04 PM

I'd have to create a python debian package for this.

Q: can you used Hadoop? You wouldn't have memory problems then! :)

• Nuria subscribed.Aug 31 2017, 4:04 PM

+1 @Ottomata hadoop seems best choice

• Nuria moved this task from Incoming to Dashiki on the Analytics board.Aug 31 2017, 4:05 PM
• Nuria moved this task from Dashiki to Incoming on the Analytics board.
Adrian_Bielefeldt added a comment.Sep 22 2017, 12:12 PM

Then we'll use hadoop.

• Nuria closed this task as Resolved.Sep 22 2017, 3:55 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL