Promote initial use of new Wiki Replica servers
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	bd808
	Aug 7 2017, 3:56 PM

Description

The long process to load the new servers with data from all shards (T153743: Add and sanitize s2, s4, s5, s6 and s7 to sanitarium2 and new labsdb hosts) has finished. We need to get more usage of the new servers to shake out issues like:

missing grants
performance of new servers
evaluate need for query killers

The new hosts are reachable at:

*.analytics.db.svc.eqiad.wmflabs
*.web.db.svc.eqiad.wmflabs

The intended difference is for the "web" copies to be more responsive for interactive queries and "analytics" to be optimized for long running queries.

NOTE: These new servers will not allow tools to create their own databases/tables co-located with the replicated content. If a tool is using this functionality on the current *.labsdb hosts we are recommending that it be updated to instead keep all tool specific data on toolsdb and perform any joining of replica data and tool specific data in application space rather than with cross-database joins. See T173511 for further discussion of providing some method for 'curated' datasets to be added to the cluster.

Related Objects
Search...

Status	Assigned	Task
Resolved	jcrespo	T140788 Labs databases rearchitecture (tracking)
Resolved	bd808	T166402 Program 7 Outcome 3: data services
Resolved	bd808	T142807 Migrate all users to new Wiki Replica cluster and decommission old hardware
Resolved	bd808	T172704 Promote initial use of new Wiki Replica servers
Resolved	bd808	T174860 Define naming scheme for connecting to new wiki replica cluster

Event Timeline

bd808 created this task.Aug 7 2017, 3:56 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 7 2017, 3:56 PM

jcrespo added a parent task: T140788: Labs databases rearchitecture (tracking).Aug 7 2017, 3:58 PM

bd808 moved this task from Backlog to Wiki replicas on the Data-Services board.Aug 7 2017, 4:04 PM

bd808 moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.Aug 7 2017, 6:25 PM

bd808 updated the task description. (Show Details)Aug 10 2017, 3:14 PM

Pinging @Samwilson, @MusikAnimal, @Niharika, and @kaldari here to see if they would be interested in moving some of their personal/CommTech tools over to hit the new servers.

NOTE: the preferred service names are going to change as soon as https://gerrit.wikimedia.org/r/#/c/371055/ is merged and deployed. Probably worth waiting for that to happen.

New service names are live.

$ mysql -h wikireplica-analytics.eqiad.wmnet
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 917531
Server version: 10.1.25-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

(u3518@wikireplica-analytics.eqiad.wmnet) [(none)]> show databases;
+------------------------+
| Database               |
+------------------------+
| aawiki_p               |
| aawikibooks_p          |
| aawiktionary_p         |
| abwiki_p               |
| abwiktionary_p         |
| acewiki_p              |
...
| zuwiki_p               |
| zuwikibooks_p          |
| zuwiktionary_p         |
+------------------------+
877 rows in set (0.04 sec)

zhuyifei1999 subscribed.Aug 10 2017, 11:38 PM

Halfak mentioned this in T173511: Implement technical details and process for "datasets_p" on wikireplica hosts.Aug 17 2017, 3:18 PM

Steinsplitter subscribed.Aug 21 2017, 6:03 PM

bd808 claimed this task.Aug 25 2017, 10:24 PM

Restricted Application added a project: User-bd808. · View Herald TranscriptAug 25 2017, 10:24 PM

bd808 updated the task description. (Show Details)Aug 30 2017, 3:43 PM

bd808 triaged this task as Medium priority.Sep 2 2017, 4:14 PM

bd808 created subtask T174860: Define naming scheme for connecting to new wiki replica cluster.

bd808 edited parent tasks, added: T142807: Migrate all users to new Wiki Replica cluster and decommission old hardware; removed: T140788: Labs databases rearchitecture (tracking).Sep 5 2017, 8:05 PM

I've switched ws-cat-browser to use wikireplica-analytics.eqiad.wmnet and all seems great.

bd808 renamed this task from Promote beta test of new Wiki Replica servers to Promote initial use of new Wiki Replica servers.Sep 18 2017, 5:29 PM

bd808 updated the task description. (Show Details)

@bd808 I take it we should no longer rely on the wikireplica-web and wikireplica-analytics host names?

In T172704#3633841, @MusikAnimal wrote:

@bd808 I take it we should no longer rely on the wikireplica-web and wikireplica-analytics host names?

They will be sticking around for "a while", but yes you would be best served by changing to the new 'official' naming scheme.

bd808 closed subtask T174860: Define naming scheme for connecting to new wiki replica cluster as Resolved.Sep 26 2017, 4:51 PM

Ricordisamoa subscribed.Sep 27 2017, 2:47 AM

bd808 moved this task from Soon! to Done on the cloud-services-team (Kanban) board.Sep 28 2017, 3:54 PM

bd808 moved this task from To Do to Done on the User-bd808 board.Jul 15 2020, 9:17 PM

Promote initial use of new Wiki Replica serversClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

Promote initial use of new Wiki Replica servers
Closed, ResolvedPublic
Actions

Related Objects
Search...