Page MenuHomePhabricator

Refactor and update our internal R codebase
Closed, ResolvedPublic6 Story Points

Description

We need to consolidate some functionality in Ironholds/olivr to wikimedia-research/wmf, as well as remove or update any deprecated.

For example, we don't use the outdated wmf::hive_query() in our golden retriever package but instead have a query_hive() function that lives inside golden::common.R.

This task is complete when the hive/mysql-querying functionality currently split between golden::common.R and olivr instead lives just in wmf and when our internal packages don't rely on Oliver's personal package.

Event Timeline

mpopov created this task.Feb 24 2016, 8:40 PM
mpopov created this object with visibility "Discovery-Analysis (Current work) (Project)".
mpopov changed the point value for this task from 20 to 4.
mpopov changed the visibility from "Discovery-Analysis (Current work) (Project)" to "All Users".Feb 25 2016, 7:09 PM
Krenair changed the visibility from "All Users" to "Public (No Login Required)".Feb 25 2016, 7:11 PM
Krenair added a subscriber: Krenair.

@mpopov: Please remember that Discovery-Analysis-Sprint is a publicly-joinable project which should therefore never be used in visibility policies (it's not secure). All-Users should also generally be avoided in favour of Public. Thanks.

@Ironholds I just realized that since golden relies on polloi's databases of projects and languages, that if whoever is running data collection scripts (currently you) doesn't have the most up-to-date databases (the languages are scraped off a wiki table), the new languages are going to be missed. It's not a big problem, but something to think about, at least.

Ironholds changed the point value for this task from 4 to 6.Feb 26 2016, 10:06 PM
Ironholds claimed this task.Mar 1 2016, 2:30 PM
Ironholds moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.

First set: https://github.com/wikimedia-research/wmf/pull/6 @mpopov ?

(We should REALLY move this to gerrit)

Merged.

(We should REALLY move this to gerrit)

I agree. I'm so used to checking gerrit for incoming CR. Shall we poke Erik to make a wikimedia/discovery/wmf or wikimedia/discovr repo on gerrit?

mpopov added a comment.Mar 2 2016, 5:04 PM

Okay, I submitted a request (https://www.mediawiki.org/wiki/Git/New_repositories/Requests) for a gerrit repo wikimedia/discovery/wmf and merged PR#7. When the repo is created, I believe the commit history will also be carried over from GitHub. I suggest refactoring golden until the transfer is complete :)

Uh. We should refactor the code before we have the dependency refactored?

mpopov added a comment.Mar 2 2016, 6:56 PM

Yeah, good point.

Change 274723 had a related patch set uploaded (by OliverKeyes):
Switch set_proxies call over

https://gerrit.wikimedia.org/r/274723

mpopov added a comment.Mar 3 2016, 6:53 PM

@Ironholds: okay, wikimedia-research/wmf is up on gerrit now: https://git.wikimedia.org/summary/?r=wikimedia/discovery/wmf.git but I'll leave deleting it from GitHub up to you

Change 274723 merged by Bearloga:
Switch set_proxies call over

https://gerrit.wikimedia.org/r/274723

Deskana added a subscriber: Deskana.Mar 3 2016, 9:11 PM

Removing Patch-For-Review since the only patch which was in review has been merged. Further work remains.

Change 274944 had a related patch set uploaded (by OliverKeyes):
Switch golden over to use the new wmf version

https://gerrit.wikimedia.org/r/274944

Change 274944 merged by Bearloga:
Switch golden over to use the new wmf version

https://gerrit.wikimedia.org/r/274944

@Ironholds oh! Did you update your installed version of ortiz too?

Awesome! *thumbs up* Good job! :D

Deskana closed this task as Resolved.Mar 28 2016, 10:27 PM