- repository: analytics/wmf-product
- repository: analytics/wmf-product/jobs
- "analytics-product" system user for running Product Analytics-owned Oozie & Cron jobs (T255039)
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | elukey | T258970 Set up environment for Product Analytics system user | |||
| Resolved | elukey | T230743 Create a repository and user for Product Analytics Oozie jobs | |||
| Resolved | elukey | T255039 Creation of a new POSIX group and system user for the Product Analytics team |
Event Timeline
Comment Actions
Tagging Release Engineering to create the repo.
example https://github.com/wikimedia/wikimedia-discovery-analytics
Comment Actions
Release Engineering recommends and supports Gerrit code review, is that what you wanted?
https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests
Comment Actions
@mforns: Can you please reply to the previous comment? Thanks! (Also, what is a "user" in which specific system?)
Comment Actions
Can you please reply to the previous comment? Thanks! (Also, what is a "user" in which specific system?)
Sorry for the ambiguity.
Yes, we'd like a new repo in Gerrit. I'll tag and ping ProductAnalytics for them to Request one in:
https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests
Regarding the user, I meant a Hadoop user, not to be done by Release Engineering.
Thanks!
Comment Actions
@kzimmerman
Can you Product Analytics please, as Greg suggests, request a repository in Gerrit to store your team's Oozie jobs?
https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests
I can do that as well, but I thought you'd like to chose a repo name, owner and type.
Cheers
Comment Actions
Update: Gerrit repo analytics/wmf-product has been created for us to use. I've renamed the owner group to "wmf-product-analytics" and added those of us on the team who have Gerrit accounts.
Comment Actions
Change 589307 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[analytics/wmf-product@master] Add README
Comment Actions
Change 589320 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/puppet@production] Add analytics-product system user
Comment Actions
Change 589320 abandoned by Bearloga:
Add analytics-product system user
Reason:
Going to upload a different patch based on Luca's recent refactoring work
Comment Actions
@mpopov should we not use a repo like analytics/wmf-product/jobs? We'll want repos for other things as well; I'll want analytics/wmf-product/wmfdata-python, for example 🙂
Comment Actions
Change 595540 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/puppet@production] Add analytics-product system user
Comment Actions
@elukey: Is it okay if I assign you for the user creation part? (The repository part was done months ago.)
Note: we will use analytics-privatedata user for now
Comment Actions
it is yes, I'll bring this up during the next SRE meeting! Thanks for the patience :)
Comment Actions
Change 595540 merged by Elukey:
[operations/puppet@production] Add analytics-product system user
Comment Actions
Just completed the creation of the analytics-product system user + kerberos keytab, you should now be able to use it :)
Comment Actions
Thanks, Luca! Just ran sudo -u analytics-product kerberos-run-command analytics-product hdfs dfs -ls / on all the stat nodes and no problems there :)
Comment Actions
@mpopov, I'm still wondering about this.
Technically, it looks like Gerrit allows us to use analytics/wmf-product as a repo and still nest other repos within it. But it doesn't seem like it would work in GitLab (which treats "groups" and "projects" separately). And even if worked, it would still be pretty confusing for analytics/wmf-product to be a repo for jobs while, say, analytics/wmf-product/wmfdata-python is a totally separate project!
Comment Actions
@nshahquinn-wmf: That's an excellent point! I updated the task description with the analytics/wmf-product/jobs repo idea. I like that. Should we just use the analytics/wmf-product root repo as a convenient index of "sub-repos" like analytics/wmf-product/jobs and analytics/wmf-product/wmfdata-python?
Comment Actions
Using it as an index definitely makes sense. I wish Gerrit did that automatically, but since it doesn't, manual it is 😁 For now, there won't be anything to index other than the jobs repo, but no reason not to get started!