ASSUMPTION: this task should be very similar to productionizing AQS
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | JAllemandou | T138264 Productionize Druid loader | |||
| Resolved | elukey | T138262 Productionize Pivot UI | |||
| Duplicate | None | T136640 Set up a deployment repository for Pivot | |||
| Resolved | mforns | T146389 Upload the final version of the pivot repo and load test |
Marko gave me some reference of puppet code for a new service introduced: https://gerrit.wikimedia.org/r/#/c/305256/7 (deployed via scap)
Main problem now: https://github.com/github/dmca/blob/master/2016/2016-08-31-Facet.md
I am writing to request that Github take down the project at the following link: https://github.com/implydata/pivot. This project infringes the copyright of Metamarkets Group, Inc. in the proprietary, interactive visualization software called Facet, described at https://metamarkets.com/2015/introducing-facet/. Facet was developed internally at Metamarkets and Metamarkets holds the copyright thereto. Facet is only available to Metamarkets customers who purchase a license to use Facet.
Change 309301 had a related patch set uploaded (by Elukey):
Add the pivot.wikimedia.org VHost to stat1001
Change 309312 had a related patch set uploaded (by Elukey):
Add the pivot.wikimedia.org domain
Change 309315 had a related patch set uploaded (by Elukey):
Add the pivot.w.o domain to the stat1001 misc Varnish director
Change 309315 merged by Elukey:
Add the pivot.w.o domain to the stat1001 misc Varnish director
Mentioned in SAL [2016-09-09T12:45:28Z] <elukey> running authdns-update on ns0.w.o to pick up the new domain pivot.wikimedia.org (T138262)
https://pivot.wikimedia.org/ is up and running, at the moment this is the status:
Varnish --> stat1001 VHost (Basic Auth + LDAP) --proxied--> stat1001 (nodejs run using screen)
Next steps:
OOOk, Dan and I spent some time understanding how pivot initializes itself, and what would be required to integrate it with service-runner well. Its a little nasty. pivot is written in typescript, and the init code that it has doesn't have any built in way of requireing the files without having them manually inspect things like process.argv. We could write our own version that is able to configure pivot without doing that, and/or we could upstream those changes to imply-data (but there repo is down?).
I'm inclined to drop the service-runner integration for now. If/when pivot is back up, and if we really need service-runner for something slick (like easy node-clustering), then we can revisit this. For now, I'm for just doing a systemd unit around the pivot CLI like Luca was suggesting.
For future reference, this is how Andrew worked around the problems in Pivot's current code: https://gist.github.com/ottomata/b0b85d88f7f9dd0be5524adaf1e63253
If we go that way, I'd rather upstream it to Pivot itself. And we can't / shouldn't do that until their repo is back up on github. For now, yeah, I agree a simple systemd setup will meet our goal for this quarter.
Change 312495 had a related patch set uploaded (by Elukey):
First draft of the Pivot UI's puppetization
Change 312495 had a related patch set uploaded (by Elukey):
Imply Pivot UI's puppetization
Change 315047 had a related patch set uploaded (by Elukey):
Fix wrong Pivot dependency in its puppet class
Change 315050 had a related patch set uploaded (by Elukey):
Updated scap configuration
Change 315056 had a related patch set uploaded (by Elukey):
Fix Scap name for the Analytics Pivot repository
Change 315073 had a related patch set uploaded (by Elukey):
Fix the Pivot's systemd unit with the right exec path
Change 315073 merged by Elukey:
Fix the Pivot's systemd unit with the right exec path
Change 315081 had a related patch set uploaded (by Elukey):
Add the Druid's default port to Pivot's hieradata config
Change 315081 merged by Elukey:
Add the Druid's default port to Pivot's hieradata config
Change 315083 had a related patch set uploaded (by Elukey):
Move the httpd Proxy rules from stat1002 to localhost for Pivot
Change 315083 merged by Elukey:
Move the httpd Proxy rules from stat1002 to localhost for Pivot
Change 315222 had a related patch set uploaded (by Elukey):
Set SyslogIdentifier in Pivot's systemd unit
Pivot has been deployed via Scap to stat1001 and now it is serving pivot.wikimedia.org.
I updated https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Druid with basic admin info for pivot.
The only problem left is that version 0.9.39 seems to not visualize correctly data labels on the front page, meanwhile version 0.8.4 (the one previously running on stat1002 in screen) is not affected.
Change 315315 had a related patch set uploaded (by Mforns):
[WIP] Add data cube title to config
@elukey @Milimetric
This ugly hack may solve the label problem, if nothing else does.
I haven't tested yet, but I thought it would be cool to have it here for you guys to see.
Feel free to abandon whenever we find a better solution.
it's not ugly, and should probably be upstreamed anyway if they haven't fixed it otherwise. I left a suggestion on the patch and we should totally do this if there's nothing easier in the druid config.
Change 315480 had a related patch set uploaded (by Elukey):
Add config file for the Pivot UI
Change 315481 had a related patch set uploaded (by Elukey):
Fix template render bug for the Pivot UI
Change 315486 had a related patch set uploaded (by Elukey):
Fix remaining issues with the Pivot UI
@mforns: I added a simple config file to Pivot and now the labels are clearly visible, plus we can add a description to them (afaik it is not possible to add this data to druid).
I agree with @Milimetric though that your patch is good and should be upstreamed! Thanks a lot for the work :)
Change 315315 abandoned by Mforns:
[WIP] Add data cube title to config
Reason:
We solved this by adding a config file.