Page MenuHomePhabricator
Create Task
Maniphest T106257

Send raw server side events to Kafka using a PHP Kafka Client {oryx}
Closed, ResolvedPublic0 Estimated Story Points
Actions

Description

This task is technically part of T102225, but is a low priority and may require a lot of work due to the potential difficulty of T106256.

We all think this will be somewhere between 21 to 34 points, the notes from tasking etherpad will be copied into here and this should be split up into subtasks.

Related Objects
Search...

Event Timeline

Ottomata created this task.Jul 19 2015, 6:07 AM
Ottomata renamed this task from Send raw server side events to Kafka using a PHP Kafka Client to Send raw server side events to Kafka using a PHP Kafka Client {stag}.
Ottomata raised the priority of this task from to Low.
Ottomata updated the task description. (Show Details)
Ottomata added projects: MediaWiki-extensions-EventLogging, Analytics-Kanban.
Ottomata added a subtask: T106256: Kafka Client for MediaWiki.
Ottomata set Security to None.
Ottomata added subscribers: • kevinator, Aklapper, Ottomata.
Ottomata moved this task from Backlog to {stag} EventLogging on Kafka on the MediaWiki-extensions-EventLogging board.Jul 19 2015, 6:10 AM
Ottomata edited projects, added Analytics-Backlog; removed Analytics-Kanban.Jul 22 2015, 3:52 PM
EBernhardson closed subtask T106256: Kafka Client for MediaWiki as Resolved.Sep 23 2015, 11:12 PM
Ottomata added a comment.Sep 24 2015, 1:51 PM

Hey a-team, we can do this now! :D

• kevinator renamed this task from Send raw server side events to Kafka using a PHP Kafka Client {stag} to Send raw server side events to Kafka using a PHP Kafka Client.Oct 1 2015, 5:15 PM
• kevinator removed a parent task: T102225: {stag} EventLogging on Kafka.

removing this as blocking the Stag project.

Milimetric moved this task from Incoming to Backlog on the Analytics-Backlog board.Oct 1 2015, 5:25 PM
bd808 mentioned this in T114733: Determine proper encoding for structured log data sent to Kafka by MediaWiki.Oct 6 2015, 3:27 AM
• ggellerman edited projects, added Analytics; removed Analytics-Backlog.Jan 12 2016, 7:41 PM
Milimetric moved this task from Incoming to Event Platform on the Analytics board.Jan 12 2016, 7:41 PM
• Nuria raised the priority of this task from Low to Medium.Jan 26 2016, 7:30 PM
• Nuria edited projects, added Analytics-Kanban; removed Analytics.
Milimetric renamed this task from Send raw server side events to Kafka using a PHP Kafka Client to Send raw server side events to Kafka using a PHP Kafka Client {oryx} [0 pts].Feb 8 2016, 6:34 PM
Milimetric updated the task description. (Show Details)
• Nuria subscribed.EditedFeb 8 2016, 6:35 PM

Substaks:

This is likely between 21 and 34.

Substasks:

  • make sure we can publish json text with mediawiki mononlog (right now search team is only publishing avro)
  • update eventlogging extension to use mediawiki monolog
  • update puppet code so server side eventlogging publishes to kafka
  • update eventlogging extension unit tests
  • deploy to beta labs
  • test test
Ottomata added a comment.Feb 8 2016, 6:38 PM

Not sure if using monolog is necessary or the best solution. It might be, but you might be able to just update the EventLogging extension to produce to Kafka instead of UDP.

• Nuria edited projects, added Analytics; removed Analytics-Kanban.Feb 8 2016, 6:47 PM
• Nuria raised the priority of this task from Medium to High.Feb 8 2016, 6:51 PM
• Nuria moved this task from Event Platform to Analytics Query Service on the Analytics board.
• Nuria moved this task from Analytics Query Service to Operational Excellence Future on the Analytics board.
Milimetric merged a task: T124813: Server side eventlogging should publish to kafka and not use udp {stag}.Feb 8 2016, 6:52 PM
Milimetric renamed this task from Send raw server side events to Kafka using a PHP Kafka Client {oryx} [0 pts] to Send raw server side events to Kafka using a PHP Kafka Client {oryx}.Feb 22 2016, 9:08 PM
Milimetric set the point value for this task to 0.
• Nuria added a comment.Feb 25 2016, 5:34 PM

This should not be needed if these set of chnages work: https://phabricator.wikimedia.org/T127209

• Nuria closed this task as Resolved.Mar 9 2016, 8:36 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits