Page MenuHomePhabricator
Create Task
Maniphest T128016

Refactor and update our internal R codebase
Closed, ResolvedPublic6 Estimated Story Points
Actions

Description

We need to consolidate some functionality in Ironholds/olivr to wikimedia-research/wmf, as well as remove or update any deprecated.

For example, we don't use the outdated wmf::hive_query() in our golden retriever package but instead have a query_hive() function that lives inside golden::common.R.

This task is complete when the hive/mysql-querying functionality currently split between golden::common.R and olivr instead lives just in wmf and when our internal packages don't rely on Oliver's personal package.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Switch golden over to use the new wmf versionwikimedia/discovery/goldenmaster+171 -269
Switch set_proxies call overwikimedia/discovery/goldenmaster+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

mpopov created this task.Feb 24 2016, 8:40 PM
mpopov created this object with visibility "Discovery-Analysis (Current work) (Project)".
mpopov changed the point value for this task from 20 to 4.
mpopov changed the visibility from "Discovery-Analysis (Current work) (Project)" to "All Users".Feb 25 2016, 7:09 PM
Krenair changed the visibility from "All Users" to "Public (No Login Required)".Feb 25 2016, 7:11 PM
Krenair subscribed.

@mpopov: Please remember that Discovery-Analysis-Sprint is a publicly-joinable project which should therefore never be used in visibility policies (it's not secure). All-Users should also generally be avoided in favour of Public. Thanks.

mpopov added a comment.Feb 26 2016, 5:43 PM

@Ironholds I just realized that since golden relies on polloi's databases of projects and languages, that if whoever is running data collection scripts (currently you) doesn't have the most up-to-date databases (the languages are scraped off a wiki table), the new languages are going to be missed. It's not a big problem, but something to think about, at least.

Ironholds changed the point value for this task from 4 to 6.Feb 26 2016, 10:06 PM
Ironholds claimed this task.Mar 1 2016, 2:30 PM
Ironholds moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.
Ironholds added a comment.Mar 1 2016, 2:48 PM

First set: https://github.com/wikimedia-research/wmf/pull/6 @mpopov ?

(We should REALLY move this to gerrit)

Ironholds added a project: Patch-For-Review.Mar 1 2016, 2:48 PM
Ironholds moved this task from In progress to Needs review on the Discovery-Analysis (Current work) board.
mpopov added a comment.Mar 1 2016, 11:19 PM
In T128016#2075748, @Ironholds wrote:

First set: https://github.com/wikimedia-research/wmf/pull/6 @mpopov ?

Merged.

(We should REALLY move this to gerrit)

I agree. I'm so used to checking gerrit for incoming CR. Shall we poke Erik to make a wikimedia/discovery/wmf or wikimedia/discovr repo on gerrit?

Ironholds added a comment.Mar 1 2016, 11:19 PM

Totes babes!

Ironholds moved this task from Needs review to In progress on the Discovery-Analysis (Current work) board.Mar 2 2016, 4:26 PM
Ironholds added a comment.Mar 2 2016, 4:34 PM

and https://github.com/wikimedia-research/wmf/pull/7

mpopov added a comment.Mar 2 2016, 5:04 PM

Okay, I submitted a request (https://www.mediawiki.org/wiki/Git/New_repositories/Requests) for a gerrit repo wikimedia/discovery/wmf and merged PR#7. When the repo is created, I believe the commit history will also be carried over from GitHub. I suggest refactoring golden until the transfer is complete :)

Ironholds added a comment.Mar 2 2016, 5:15 PM

Uh. We should refactor the code before we have the dependency refactored?

mpopov added a comment.Mar 2 2016, 6:56 PM

Yeah, good point.

gerritbot subscribed.Mar 3 2016, 3:13 PM

Change 274723 had a related patch set uploaded (by OliverKeyes):
Switch set_proxies call over

https://gerrit.wikimedia.org/r/274723

Ironholds moved this task from In progress to Needs review on the Discovery-Analysis (Current work) board.Mar 3 2016, 3:13 PM
mpopov added a comment.Mar 3 2016, 6:53 PM

@Ironholds: okay, wikimedia-research/wmf is up on gerrit now: https://git.wikimedia.org/summary/?r=wikimedia/discovery/wmf.git but I'll leave deleting it from GitHub up to you

gerritbot added a comment.Mar 3 2016, 8:48 PM

Change 274723 merged by Bearloga:
Switch set_proxies call over

https://gerrit.wikimedia.org/r/274723

Ironholds moved this task from Needs review to In progress on the Discovery-Analysis (Current work) board.Mar 3 2016, 9:06 PM
• Deskana removed a project: Patch-For-Review.Mar 3 2016, 9:07 PM
• Deskana subscribed.Mar 3 2016, 9:11 PM

Removing Patch-For-Review since the only patch which was in review has been merged. Further work remains.

gerritbot added a comment.Mar 4 2016, 2:08 PM

Change 274944 had a related patch set uploaded (by OliverKeyes):
Switch golden over to use the new wmf version

https://gerrit.wikimedia.org/r/274944

gerritbot added a project: Patch-For-Review.Mar 4 2016, 2:08 PM
Ironholds moved this task from In progress to Needs review on the Discovery-Analysis (Current work) board.Mar 4 2016, 2:08 PM
gerritbot added a comment.Mar 7 2016, 11:17 PM

Change 274944 merged by Bearloga:
Switch golden over to use the new wmf version

https://gerrit.wikimedia.org/r/274944

Ironholds moved this task from Needs review to Done on the Discovery-Analysis (Current work) board.Mar 7 2016, 11:19 PM
mpopov added a comment.Mar 7 2016, 11:27 PM

@Ironholds oh! Did you update your installed version of ortiz too?

Ironholds added a comment.Mar 7 2016, 11:28 PM

Yup!

mpopov added a comment.Mar 7 2016, 11:29 PM

Awesome! *thumbs up* Good job! :D

• Deskana closed this task as Resolved.Mar 28 2016, 10:27 PM
• debt moved this task from Done to Resolved on the Discovery-Analysis (Current work) board.Jul 20 2016, 4:23 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits