Maniphest T214074

Productionize article recommender systems
Open, MediumPublic
Actions

Assigned To

None

Authored By

	• bmansurov
	Jan 17 2019, 7:01 PM

Tags

Referenced Files

None

Subscribers

Description

Production Pipeline overview

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T214074 Productionize article recommender systems
		Resolved		Ottomata	T213566 Transferring data from Hadoop to production MySQL database
		Resolved		Ottomata	T213976 Workflow to be able to move data files computed in jobs from analytics cluster to production
		Resolved		Ottomata	T219544 Make hadoop cluster able to push to swift
		Resolved		ayounsi	T220081 Allow swift https access from analytics to prod
		Resolved		Ottomata	T227896 Make oozie swift upload emit event to Kafka about swift object upload complete
		Resolved		elukey	T228882 Allow analytics VLAN to reach eventgate-analytics.discovery.wmnet:31192
		Resolved		EBernhardson	T227364 Adjust mjolnir bulk_daemon to import glent swift uploads
		Resolved		MatthewVernon	T229584 Run swift-object-expirer as part of the swift cluster
		Resolved		None	T211980 'morelike' recommendation API: Bulk import data to MySQL in chunks
		Declined		None	T215421 Site: 1 VM request for recommender-systems
		Resolved		• bmansurov	T213866 Filter out newly created articles
		Open		None	T213761 Verify newly generated data before pushing it to production
		Resolved		• bmansurov	T210844 Generate article recommendations in Hadoop for use in production
		Resolved		• bmansurov	T211981 Improve article-recommender scripts
		Resolved		jbond	T217447 Add bmansurov to archiva-deployers LDAP group
		Invalid		None	T218955 Some worker nodes don't seem to have numpy installed
		Resolved		Dzahn	T219309 Create research-alerts mailing list
		Resolved		JAllemandou	T209655 Copy Wikidata dumps to HDFS + parquet
		Resolved		JAllemandou	T243832 Fix hdfs-rsync`prune-empty-dirs` feature

Event Timeline

• bmansurov created this task.Jan 17 2019, 7:01 PM

• bmansurov added a subtask: T213566: Transferring data from Hadoop to production MySQL database.

• bmansurov updated the task description. (Show Details)

• bmansurov added a subtask: T213866: Filter out newly created articles.Jan 17 2019, 7:05 PM

• bmansurov added a subtask: T213761: Verify newly generated data before pushing it to production.

• bmansurov added a subtask: T210844: Generate article recommendations in Hadoop for use in production.Jan 17 2019, 7:08 PM

• bmansurov added a subtask: T209655: Copy Wikidata dumps to HDFS + parquet.Jan 17 2019, 7:11 PM

• bmansurov moved this task from Backlog to In Progress on the Research board.Jan 17 2019, 7:45 PM

• bmansurov triaged this task as Medium priority.Jan 18 2019, 9:14 PM

• bmansurov moved this task from Backlog to In-progress on the Article-Recommendation board.

• Mholloway subscribed.Feb 1 2019, 12:17 AM

akosiaris changed the status of subtask T213566: Transferring data from Hadoop to production MySQL database from Open to Stalled.Feb 20 2019, 2:11 PM

Ottomata mentioned this in T213976: Workflow to be able to move data files computed in jobs from analytics cluster to production .Feb 21 2019, 2:33 PM

• bmansurov closed subtask T213866: Filter out newly created articles as Resolved.Feb 21 2019, 3:09 PM

leila moved this task from In Progress to Backlog on the Research board.Jul 11 2019, 12:38 AM

leila edited projects, added Research-Freezer; removed Research.Jul 11 2019, 3:49 PM

leila removed a project: Research-Freezer.Nov 20 2019, 12:26 AM

Ottomata closed subtask T213566: Transferring data from Hadoop to production MySQL database as Resolved.Nov 20 2019, 7:49 PM

• Nuria closed subtask T209655: Copy Wikidata dumps to HDFS + parquet as Resolved.Feb 27 2020, 7:30 PM

leila closed subtask T210844: Generate article recommendations in Hadoop for use in production as Resolved.Jun 2 2021, 10:42 PM

obviously not "in progress"

• Mholloway unsubscribed.Jul 20 2021, 12:49 PM

leila removed • bmansurov as the assignee of this task.Jul 20 2021, 7:06 PM