Change Details

Given the growth rate of Wikidata, we need to increase storage capacity for WDQS. There are 2 possible strategies: **1) increase the storage capacity of our current servers** This is fairly straightforward, but would require adding storage in our current servers. 2) migrate from RAID1 to RAID0 Other services (elasticsearch / cassandra) manage redundancy at the cluster level and not at the server level. For this to make sense, we need to ensure that loosing a full server is a non event (the cluster is robust to loosing a node AND reimaging is cheap). In the case of WDQS, we currently have 3 nodes per clusters (4 clusters: public / internal for both eqiad and codfw), increasing it to 4 nodes would provide enough redundancy at the cluster level that loosing a node is a non issue. Note: the current implementation of WDQS is not shardable, this is addressed separately (T221631 / [[ https://wikitech.wikimedia.org/wiki/Wikidata_query_service/ScalingStrategy | Wikidata_query_service/ScalingStrategy ]])

Given the growth rate of Wikidata, we need to increase storage capacity for WDQS. There are 2 possible strategies: 1) **increase the storage capacity of our current servers** This is fairly straightforward, but would require adding storage in our current servers. 2) migrate from RAID1 to RAID0 Other services (elasticsearch / cassandra) manage redundancy at the cluster level and not at the server level. For this to make sense, we need to ensure that loosing a full server is a non event (the cluster is robust to loosing a node AND reimaging is cheap). In the case of WDQS, we currently have 3 nodes per clusters (4 clusters: public / internal for both eqiad and codfw), increasing it to 4 nodes would provide enough redundancy at the cluster level that loosing a node is a non issue. Note: the current implementation of WDQS is not shardable, this is addressed separately (T221631 / [[ https://wikitech.wikimedia.org/wiki/Wikidata_query_service/ScalingStrategy | Wikidata_query_service/ScalingStrategy ]])

Given the growth rate of Wikidata, we need to increase storage capacity for WDQS. There are 2 possible strategies: **1) 1) **increase the storage capacity of our current servers** This is fairly straightforward, but would require adding storage in our current servers. 2) migrate from RAID1 to RAID0 Other services (elasticsearch / cassandra) manage redundancy at the cluster level and not at the server level. For this to make sense, we need to ensure that loosing a full server is a non event (the cluster is robust to loosing a node AND reimaging is cheap). In the case of WDQS, we currently have 3 nodes per clusters (4 clusters: public / internal for both eqiad and codfw), increasing it to 4 nodes would provide enough redundancy at the cluster level that loosing a node is a non issue. Note: the current implementation of WDQS is not shardable, this is addressed separately (T221631 / [[ https://wikitech.wikimedia.org/wiki/Wikidata_query_service/ScalingStrategy | Wikidata_query_service/ScalingStrategy ]])