Page MenuHomePhabricator
Create Task
Maniphest T128407

Remove Client IP from Eventlogging capsule {mole}
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

We decided to remove ClientIP's altogether from Eventlogging capsule altogether on T126366: Add IP field only to schemas that need it. Remove it from EL capsule and do not collect it by default {mole}. @leila let's block this task on the completion of QuickSurvey data collection.

In this task, we'll only make configuration changes, and may be fix tests(?) - and drop the field from the capsule. There's a subtask that we'll use to clean up and refactor existing code to remove the IP hasher related stuff from EL.

Details

SubjectRepoBranchLines +/-
Remove etcd usage from eventlogging processoroperations/puppetproduction+0 -27
Remove clientIp from EventLogging varnishkafka formatoperations/puppetproduction+3 -1
Update EventCapsule version after dropping clientIPeventloggingmaster+1 -1
Add a format specifier that indicates to parser to ignore the valueeventloggingmaster+38 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

• madhuvishy created this task.Feb 29 2016, 8:45 PM
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 29 2016, 8:45 PM
• madhuvishy created subtask T128408: Clean up Client IP and hashing related code on Eventlogging {oryx} {mole}.Feb 29 2016, 8:46 PM
leila added a subtask: T125946: Large scale QuickSurvey for reader segmentation research.Feb 29 2016, 10:27 PM

@madhuvishy I added the blocking task T125946. It's marked as resolved but it's not. It will be on March 8 or at the latest on March 14. I'll update this thread.

leila added a project: Research.Feb 29 2016, 10:29 PM
leila moved this task from Backlog to Radar on the Research board.
• madhuvishy added a comment.Feb 29 2016, 10:36 PM

@leila Thanks!

gerritbot subscribed.Mar 1 2016, 5:54 PM

Change 274152 had a related patch set uploaded (by Madhuvishy):
[WIP] Ignore client IP while parsing events

https://gerrit.wikimedia.org/r/274152

gerritbot added a project: Patch-For-Review.Mar 1 2016, 5:54 PM
• madhuvishy moved this task from Next Up to In Progress on the Analytics-Kanban board.Mar 2 2016, 4:35 PM
Tgr mentioned this in T108757: Allow opting out from logging some of the default EventLogging fields on a schema-by-schema basis.Mar 2 2016, 8:16 PM
• madhuvishy mentioned this in T126366: Add IP field only to schemas that need it. Remove it from EL capsule and do not collect it by default {mole}.Mar 2 2016, 8:44 PM
gerritbot added a comment.Mar 2 2016, 8:46 PM

Change 274152 merged by Ottomata:
Add a format specifier that indicates to parser to ignore the value

https://gerrit.wikimedia.org/r/274152

Milimetric set the point value for this task to 5.Mar 3 2016, 5:19 PM
Nemo_bis awarded a token.Mar 3 2016, 5:33 PM
• madhuvishy moved this task from In Progress to Ready to Deploy on the Analytics-Kanban board.Mar 4 2016, 4:40 PM
leila reopened subtask T125946: Large scale QuickSurvey for reader segmentation research as Open.Mar 4 2016, 5:08 PM
leila added a comment.Mar 4 2016, 5:11 PM

@madhuvishy FYI, our survey tracked in T125946 is scheduled to stop on Monday, 2016-03-07 in the 16:00-17:00 PST SWAT Deploy time slot.

• madhuvishy added a comment.Mar 7 2016, 11:29 PM

Thanks @leila :) @Nuria @Ottomata should we start this deployment process tomorrow (Tuesday)? The plan is here - https://etherpad.wikimedia.org/p/el-clientips-drop

• Nuria added a comment.Mar 8 2016, 3:35 AM

I am for deploying, let's check at every step that things are proceeding as they should.

Ottomata added a comment.Mar 8 2016, 2:53 PM

Yeah! let’s do it! After standup?

leila closed subtask T125946: Large scale QuickSurvey for reader segmentation research as Resolved.Mar 8 2016, 4:33 PM
• madhuvishy removed a subtask: T128408: Clean up Client IP and hashing related code on Eventlogging {oryx} {mole}.Mar 8 2016, 6:59 PM
• madhuvishy added a parent task: T128408: Clean up Client IP and hashing related code on Eventlogging {oryx} {mole}.
gerritbot added a comment.Mar 8 2016, 7:41 PM

Change 275892 had a related patch set uploaded (by Ottomata):
Remove clientIp from EventLogging varnishkafka format

https://gerrit.wikimedia.org/r/275892

gerritbot added a comment.Mar 8 2016, 7:42 PM

Change 275893 had a related patch set uploaded (by Madhuvishy):
Update EventCapsule version after dropping clientIP

https://gerrit.wikimedia.org/r/275893

gerritbot added a comment.Mar 8 2016, 7:45 PM

Change 275893 merged by Ottomata:
Update EventCapsule version after dropping clientIP

https://gerrit.wikimedia.org/r/275893

gerritbot added a comment.Mar 9 2016, 7:09 PM

Change 275892 merged by Ottomata:
Remove clientIp from EventLogging varnishkafka format

https://gerrit.wikimedia.org/r/275892

gerritbot added a comment.Mar 9 2016, 7:25 PM

Change 276244 had a related patch set uploaded (by Ottomata):
Remove etcd usage from eventlogging processor

https://gerrit.wikimedia.org/r/276244

gerritbot added a comment.Mar 9 2016, 7:28 PM

Change 276244 merged by Ottomata:
Remove etcd usage from eventlogging processor

https://gerrit.wikimedia.org/r/276244

• Nuria moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Mar 9 2016, 8:29 PM
• ggellerman edited projects, added Research-Freezer; removed Research.Mar 17 2016, 10:25 PM
• ggellerman moved this task from Backlog to Radar on the Research-Freezer board.Mar 17 2016, 10:27 PM
• Nuria closed this task as Resolved.Mar 22 2016, 7:14 PM
• schana mentioned this in T165364: Test data quality of the test surveys.May 16 2017, 6:36 PM
leila mentioned this in T150325: Move first paragraph before infobox on stable.May 17 2017, 4:35 PM
leila mentioned this in T165678: clientIP needs to be collected as part of the schema or ....May 18 2017, 4:07 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL