Change Details

ORES draftopic scores pushed into Kafka via {T240549} need to get to HDFS somehow, and new scores for the same page need to override old scores. Presumably EventGate can handle this. Open questions: * Index via title or page ID? The first is less efficient and less robust, but would allow cross-wiki lookups, which might or might not be needed while drafttopic is enwiki-only. * Do we need a mechanism to get rid of data for deleted pages?

ORES articletopic scores pushed into Kafka via {T240549} need to get to HDFS somehow, and new scores for the same page need to override old scores. Presumably EventGate can handle this. Open questions: * Index via title or page ID? The first is less efficient and less robust, but would allow cross-wiki lookups, which might or might not be needed while articletopic is enwiki-only. * Do we need a mechanism to get rid of data for deleted pages?

ORES drafarticletopic scores pushed into Kafka via {T240549} need to get to HDFS somehow, and new scores for the same page need to override old scores. Presumably EventGate can handle this. Open questions: * Index via title or page ID? The first is less efficient and less robust, but would allow cross-wiki lookups, which might or might not be needed while draftarticletopic is enwiki-only. * Do we need a mechanism to get rid of data for deleted pages?