See comments https://phabricator.wikimedia.org/T294380#7524057 and below.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | MatthewVernon | T294380 Storage request for datasets published by research team | |||
| Resolved | Ottomata | T296945 Deploy research_poc Swift credidentials to Hadoop |
Event Timeline
Comment Actions
Change 743214 had a related patch set uploaded (by Ottomata; author: Ottomata):
[operations/puppet@production] Deploy research_poc thanos swift auth env file to hadoop
Comment Actions
Change 743214 merged by Ottomata:
[operations/puppet@production] Deploy research_poc thanos swift auth env file to hadoop
Comment Actions
Change 744019 had a related patch set uploaded (by Ottomata; author: Ottomata):
[operations/puppet@production] profile::analytics::cluster::secrets - run as hdfs user
Comment Actions
Change 744019 merged by Ottomata:
[operations/puppet@production] profile::analytics::cluster::secrets - run as hdfs user
Comment Actions
Change 744022 had a related patch set uploaded (by Ottomata; author: Ottomata):
[operations/puppet@production] profile::analytics::cluster::secrets - fix for research_poc swift user
Comment Actions
Change 744022 merged by Ottomata:
[operations/puppet@production] profile::analytics::cluster::secrets - fix for research_poc swift user
Comment Actions
Change 744037 had a related patch set uploaded (by Ottomata; author: Ottomata):
[operations/puppet@production] profile::analytics::cluster::secrets - fix chmod for swift research_poc env file
Comment Actions
Change 744037 merged by Ottomata:
[operations/puppet@production] profile::analytics::cluster::secrets - fix chmod for swift research_poc env file
Comment Actions
sudo -u analytics-research kerberos-run-command analytics-research hdfs dfs -cat /user/analytics-research/swift_auth_research_poc.env
Comment Actions
Hm, perhaps, although I'm not sure where. This is sort of a one off. We'd love to have more first class support for exporting to swift, but atm its a little hacky.
I added a link to this task at https://wikitech.wikimedia.org/wiki/Analytics/Systems/Exporting_from_HDFS_to_Swift.