Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Declined | None | T112846 Display automata and humans separately on zero results rate graph | |||
| Resolved | EBernhardson | T103505 Create analytics-centric Cirrus logs and have them import into HDFS | |||
| Resolved | None | T106257 Send raw server side events to Kafka using a PHP Kafka Client {oryx} | |||
| Resolved | EBernhardson | T106256 Kafka Client for MediaWiki | |||
| Resolved | mpopov | T110590 Add breakdown of zero results rate by language/project pair to dashboard | |||
| Resolved | • csteipp | T109384 Security review of apache/avro and nmred/kafka-php | |||
| Resolved | bd808 | T111851 Package the Avro PHP library for easier Composer usage |
Event Timeline
This is a dependency for being able to easily analyze the logs generated by search, currently they are ~30GB per day. This will allow us to put them into hadoop which is better equipped to handle the volume of data.
There is no hard timeline on this. It's not blocking anything in particular, it will just allow us more visibility into the data we already collect and allow the two search analysts to do more in less time.
Could you give an estimate on where this fits into the security team's timeline so we can plan appropriately?
@csteipp: My understanding is that there was some follow-up work related to this. Can you mention those tasks here so I can follow them?
@ksmith, I -1'ed https://gerrit.wikimedia.org/r/#/c/232075/ for bundling potentially dangerous example code with the library. How the team wants to handle that is up to them-- they should add blockers here as needed.