[14:05] <gwicke> pchelolo has already created an aqs test module at https://github.com/wikimedia/restbase/blob/master/test/aqs_test_module.yaml, and I think we could turn that into the actual prod module
[14:06] <milimetric> gwicke: that sounds great, but maybe let us know when you have a stable new approach to the configs and you've documented everything so I can roll up my sleeves and fix the puppet configs
[14:07] <gwicke> actually.. I think https://github.com/wikimedia/restbase/blob/master/projects/aqs_default.yaml already does that
[14:09] <gwicke> this is then referenced like this: https://github.com/wikimedia/restbase/blob/master/config.test.yaml#L56
[14:09] <gwicke> you'll want to leave out the aqs_test_module
[14:12] <gwicke> milimetric: the only options you need to pass are the table ones, from what I can see: https://github.com/wikimedia/restbase/blob/master/config.test.yaml#L11
[14:14] <gwicke> petr is looking into creating an example config for you
[14:18] <milimetric> gwicke: if it's not too much trouble, I'd love just an email or a wiki page explaining what happened with the configs. The example would be very sweet if Petr can do it, but I'm juggling too many balls so I can't look into it until after the holidays.
[14:20] <gwicke> milimetric: since you are the only direct user of those configs right now it's tempting to directly talk to you / update the config
[14:20] <gwicke> from a cost / benefit perspective
[14:20] <milimetric> oh! gwicke that's cool, but my brain will literally explode if I think about this, so an email would be a nice way to help my brain not explode :)
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Update AQS config with new syntax | operations/puppet | production | +20 -88 |
Event Timeline
Comment Actions
The current puppet template is at https://github.com/wikimedia/operations-puppet/blob/production/modules/restbase/templates/config.aqs.yaml.erb.
Comment Actions
I think that a template like this will work for AQS with the newest RESTBase code.
# Analytics Query Service config
aqs_project: &aqs_project
x-modules:
/:
- path: projects/aqs_default.yaml
options: &default_options
table:
hosts: [<%= Array(@seeds).join(',') %>]
keyspace: system
localDc: <%= @cassandra_localDc %>
datacenters: [<%= @cassandra_localDc %>]
username: <%= @cassandra_user %>
password: <%= @cassandra_password %>
defaultConsistency: <%= @cassandra_defaultConsistency %>
storage_groups:
# Catch-all group
- name: default.group.local
domains: /./
# Swagger spec root.
spec: &spec
title: "The Analytics RESTBase root"
paths:
/{domain:analytics.wikimedia.org}: *aqs_project
# The main service setup. Each worker can offer one or more services.
services:
- name: aqs
module: /srv/deployment/restbase/deploy/src/lib/server
conf:
port: <%= @port %>
spec: *spec
# TODO: set this in ops/private otherwise RESTBase will refuse to start
salt: <%= @salt_key %>
default_page_size: <%= @page_size %>
# Log error messages and gracefully restart a worker if v8 reports using more
# heap (note: not RSS).
worker_heap_limit_mb: 300
logging:
name: aqs
level: <%= @logging_level %>
streams:
# XXX: Use gelf-stream -> logstash
- type: gelf
host: <%= @logstash_host %>
port: <%= @logstash_port %>
# StatsD metrics collection
metrics:
name: aqs
type: statsd # default, but lets be explicit
host: <%= @statsd_host %>
port: <%= @statsd_port %>Comment Actions
Change 268560 had a related patch set uploaded (by Milimetric):
[WIP] Update AQS config with new syntax
Comment Actions
We have just pushed out a change for RESTBase (v0.10.0) and service-runner (v1.1.1) which will require another small config change. I'll comment on the PS in Gerrit.