Create puppet config to run two instances of Blazegraph
Closed, ResolvedPublic
Actions

Description

WDQS puppet now should start two instances of Blazegraph:

One instance as before
Second instance with different data file (categories.jnl) running in different port (9990) and smaller memory size (8g).

Details

Related Gerrit Patches:

operations/puppet : production	[WIP] wdqs: create multiple instances of blazegraph
operations/puppet : production	wdqs: fixed typo in prometheus-blazegraph-exporter systemd unit
operations/puppet : production	wdqs: prometheus-blazegraph-exporter supports multi instances
operations/puppet : production	wdqs: prometheus-blazegraph-exporter supports multi instances
operations/puppet : production	wdqs: removed unused port parameter on wdqs::monitor::blazegraph
operations/puppet : production	wdqs: monitor blazegraph process per instance
wikidata/query/rdf : master	Make GC log file configurable
operations/puppet : production	wdqs: make GC log file configurable per blazegraph instance
operations/puppet : production	Create second Blazegraph instance for categories
operations/puppet : production	Make config suitable for multiple instances of Blazegraph
operations/puppet : production	Prepare for multi-instance Blazegraph
operations/puppet : production	wdqs: force creation of /etc/wdqs/RWStore.properties as a file
operations/puppet : production	wdqs: force creation of /etc/wdqs/RWStore.properties
operations/puppet : production	wdqs: preliminary work to manage multiple instances

Related Objects
Search...

Status	Assigned	Task
Resolved	Smalyshev	T213210 WDQS is hitting allocator limit on Blazegraph
Resolved	Smalyshev	T213212 Move category namespace to a separate blazegraph instance
Resolved	Smalyshev	T213234 Create puppet config to run two instances of Blazegraph
Resolved	Smalyshev	T213289 blazegraph endpoint in prometheus-blazegraph-exporter should be configureable

Event Timeline

Smalyshev triaged this task as High priority.Jan 9 2019, 12:18 AM

Smalyshev created this task.

Smalyshev added subscribers: Gehel, Mathew.onipe.

Change 483047 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/puppet@production] Make config suitable for multiple instances of Blazegraph

https://gerrit.wikimedia.org/r/483047

gerritbot added a project: Patch-For-Review.Jan 9 2019, 12:56 AM

Change 483217 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: preliminary work to manage multiple instances

https://gerrit.wikimedia.org/r/483217

Change 483234 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: create multiple instances of blazegraph

https://gerrit.wikimedia.org/r/483234

Change 483217 merged by Gehel:
[operations/puppet@production] wdqs: preliminary work to manage multiple instances

https://gerrit.wikimedia.org/r/483217

Change 483504 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: force creation of /etc/wdqs/RWStore.properties

https://gerrit.wikimedia.org/r/483504

Change 483504 merged by Gehel:
[operations/puppet@production] wdqs: force creation of /etc/wdqs/RWStore.properties

https://gerrit.wikimedia.org/r/483504

Change 483505 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: force creation of /etc/wdqs/RWStore.properties as a file

https://gerrit.wikimedia.org/r/483505

Change 483505 merged by Gehel:
[operations/puppet@production] wdqs: force creation of /etc/wdqs/RWStore.properties as a file

https://gerrit.wikimedia.org/r/483505

Gehel added a project: Discovery-Wikidata-Query-Service-Sprint.Jan 10 2019, 8:52 PM

Gehel moved this task from Backlog to In progress on the Discovery-Wikidata-Query-Service-Sprint board.

Smalyshev closed subtask T213289: blazegraph endpoint in prometheus-blazegraph-exporter should be configureable as Resolved.Jan 10 2019, 8:53 PM

Change 483529 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/puppet@production] Prepare for multi-instance Blazegraph

https://gerrit.wikimedia.org/r/483529

Change 483628 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/puppet@production] Create second Blazegraph instance for categories

https://gerrit.wikimedia.org/r/483628

Addshore moved this task from incoming to monitoring on the Wikidata board.Jan 11 2019, 10:08 AM

Change 483529 merged by Gehel:
[operations/puppet@production] Prepare for multi-instance Blazegraph

https://gerrit.wikimedia.org/r/483529

Change 483047 abandoned by Smalyshev:
Make config suitable for multiple instances of Blazegraph

Reason:
replaced with https://gerrit.wikimedia.org/r/c/operations/puppet/ /483628

https://gerrit.wikimedia.org/r/483047

Change 484246 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: prometheus-blazegraph-exporter supports multi instances

https://gerrit.wikimedia.org/r/484246

Smalyshev added a project: User-Smalyshev.Jan 14 2019, 6:08 PM

Smalyshev moved this task from Backlog to Doing on the User-Smalyshev board.

Mentioned in SAL (#wikimedia-operations) [2019-01-14T20:08:54Z] <gehel> disabling puppet on all wdqs servers to deploy T213234

Change 483628 merged by Gehel:
[operations/puppet@production] Create second Blazegraph instance for categories

https://gerrit.wikimedia.org/r/483628

Change 484287 had a related patch set uploaded (by Gehel; owner: Gehel):
[wikidata/query/rdf@master] Make GC log file configurable

https://gerrit.wikimedia.org/r/484287

Change 484288 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: make GC log file configurable per blazegraph instance

https://gerrit.wikimedia.org/r/484288

Change 484288 merged by Gehel:
[operations/puppet@production] wdqs: make GC log file configurable per blazegraph instance

https://gerrit.wikimedia.org/r/484288

Change 484287 merged by jenkins-bot:
[wikidata/query/rdf@master] Make GC log file configurable

https://gerrit.wikimedia.org/r/484287

Change 484314 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: monitor blazegraph process per instance

https://gerrit.wikimedia.org/r/484314

Change 484314 merged by Gehel:
[operations/puppet@production] wdqs: monitor blazegraph process per instance

https://gerrit.wikimedia.org/r/484314

Change 484319 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: removed unused port parameter on wdqs::monitor::blazegraph

https://gerrit.wikimedia.org/r/484319

Change 484319 merged by Gehel:
[operations/puppet@production] wdqs: removed unused port parameter on wdqs::monitor::blazegraph

https://gerrit.wikimedia.org/r/484319

Smalyshev closed this task as Resolved.Jan 15 2019, 3:45 AM

Smalyshev claimed this task.

Change 484246 merged by Gehel:
[operations/puppet@production] wdqs: prometheus-blazegraph-exporter supports multi instances

https://gerrit.wikimedia.org/r/484246

Change 484521 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: prometheus-blazegraph-exporter supports multi instances

https://gerrit.wikimedia.org/r/484521

Change 484521 merged by Gehel:
[operations/puppet@production] wdqs: prometheus-blazegraph-exporter supports multi instances

https://gerrit.wikimedia.org/r/484521

Change 484565 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs: fixed typo in prometheus-blazegraph-exporter systemd unit

https://gerrit.wikimedia.org/r/484565

Change 484565 merged by Gehel:
[operations/puppet@production] wdqs: fixed typo in prometheus-blazegraph-exporter systemd unit

https://gerrit.wikimedia.org/r/484565

Change 483234 abandoned by Gehel:
[WIP] wdqs: create multiple instances of blazegraph

Reason:
superseeded by a number of other patches