Page MenuHomePhabricator

Refactor and update our internal R codebase
Closed, ResolvedPublic6 Estimated Story Points


We need to consolidate some functionality in Ironholds/olivr to wikimedia-research/wmf, as well as remove or update any deprecated.

For example, we don't use the outdated wmf::hive_query() in our golden retriever package but instead have a query_hive() function that lives inside golden::common.R.

This task is complete when the hive/mysql-querying functionality currently split between golden::common.R and olivr instead lives just in wmf and when our internal packages don't rely on Oliver's personal package.

Event Timeline

mpopov created this object with visibility "Discovery-Analysis (Current work) (Project)".
mpopov changed the point value for this task from 20 to 4.
Krenair changed the visibility from "All Users" to "Public (No Login Required)".Feb 25 2016, 7:11 PM
Krenair added a subscriber: Krenair.

@mpopov: Please remember that Discovery-Analysis-Sprint is a publicly-joinable project which should therefore never be used in visibility policies (it's not secure). All-Users should also generally be avoided in favour of Public. Thanks.

@Ironholds I just realized that since golden relies on polloi's databases of projects and languages, that if whoever is running data collection scripts (currently you) doesn't have the most up-to-date databases (the languages are scraped off a wiki table), the new languages are going to be missed. It's not a big problem, but something to think about, at least.

Ironholds changed the point value for this task from 4 to 6.Feb 26 2016, 10:06 PM


(We should REALLY move this to gerrit)

I agree. I'm so used to checking gerrit for incoming CR. Shall we poke Erik to make a wikimedia/discovery/wmf or wikimedia/discovr repo on gerrit?

Okay, I submitted a request ( for a gerrit repo wikimedia/discovery/wmf and merged PR#7. When the repo is created, I believe the commit history will also be carried over from GitHub. I suggest refactoring golden until the transfer is complete :)

Uh. We should refactor the code before we have the dependency refactored?

Change 274723 had a related patch set uploaded (by OliverKeyes):
Switch set_proxies call over

@Ironholds: okay, wikimedia-research/wmf is up on gerrit now: but I'll leave deleting it from GitHub up to you

Change 274723 merged by Bearloga:
Switch set_proxies call over

Removing Patch-For-Review since the only patch which was in review has been merged. Further work remains.

Change 274944 had a related patch set uploaded (by OliverKeyes):
Switch golden over to use the new wmf version

Change 274944 merged by Bearloga:
Switch golden over to use the new wmf version

@Ironholds oh! Did you update your installed version of ortiz too?

Awesome! *thumbs up* Good job! :D