- mobileapps use cases are double-writing to the new schema, on the Cassandra 3 production cluster.
- We are collecting information that lets us confidently switch use cases in Q2.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Eevans | T169936 Services 2017/18 Q1 goal: Start gradual roll-out of Cassandra 3 & new schema to resolve storage scaling issues and OOM errors. | |||
| Resolved | Eevans | T169940 End of September milestone: Start migration of production use cases. |
Event Timeline
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-07T21:00:19Z] <urandom> T169940: Disabling changeprop in RESTBase dev environment
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-07T21:01:32Z] <urandom> T169940: Disabling puppet and restbase service in RESTBase dev
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-08T08:11:10Z] <mobrovac> restbase enabled back puppet in the dev cluster - T169940
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-08T08:11:55Z] <mobrovac@tin> Started deploy [restbase/deploy@0d39acf] (dev-cluster): Use double writing for mobileapps in the dev cluster - T169940
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-08T09:07:00Z] <mobrovac@tin> Started deploy [restbase/deploy@0d39acf] (dev-cluster): Use double writing for mobileapps in the dev cluster, take #2 - T169940
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-08T09:08:01Z] <mobrovac@tin> Finished deploy [restbase/deploy@0d39acf] (dev-cluster): Use double writing for mobileapps in the dev cluster, take #2 - T169940 (duration: 01m 01s)
Comment Actions
Change 378912 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] Config: Add the table_ng options and new storage groups
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-19T18:35:21Z] <urandom> T160570, T169940: Upgrade restbase-ng environment to Cassandra 3.11.0-wmf5
Comment Actions
Change 378912 merged by Mobrovac:
[mediawiki/services/restbase/deploy@master] Config: Add the table_ng options and new storage groups
Comment Actions
Change 379217 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] RESTBase: Separate eqiad and codfw seeds
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-20T11:51:11Z] <mobrovac@tin> Started deploy [restbase/deploy@dea2b41]: New storage schema for mobile-sections, canary deploy for schema creation - T169940
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-20T11:58:32Z] <mobrovac@tin> Finished deploy [restbase/deploy@dea2b41]: New storage schema for mobile-sections, canary deploy for schema creation - T169940 (duration: 07m 20s)
Comment Actions
Change 379217 merged by Filippo Giunchedi:
[operations/puppet@production] RESTBase: Separate eqiad and codfw seeds
Comment Actions
Change 379223 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] Cassandra: Allow extra hosts to access the cluster via CQL
Comment Actions
Change 379223 merged by Filippo Giunchedi:
[operations/puppet@production] Cassandra: Allow extra hosts to access the cluster via CQL
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-21T07:27:09Z] <mobrovac@tin> Started deploy [restbase/deploy@3e9bd9f]: New storage schema for mobile-sections, canary deploy for schema creation, take 2 - T169940
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-21T07:27:39Z] <mobrovac@tin> Finished deploy [restbase/deploy@3e9bd9f]: New storage schema for mobile-sections, canary deploy for schema creation, take 2 - T169940 (duration: 00m 30s)
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-21T07:57:44Z] <mobrovac@tin> Started deploy [restbase/deploy@3e9bd9f]: New storage schema for mobile-sections - T169940
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-21T08:08:12Z] <mobrovac@tin> Finished deploy [restbase/deploy@3e9bd9f]: New storage schema for mobile-sections - T169940 (duration: 10m 28s)
Comment Actions
Change 380443 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] MobileApps: Switch to using the next-gen storage only
Comment Actions
Change 380443 merged by Ppchelko:
[mediawiki/services/restbase/deploy@master] MobileApps: Switch to using the next-gen storage only
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-25T08:26:38Z] <mobrovac@tin> Started deploy [restbase/deploy@ab24f70]: Switch mobile-sections to next-gen storage and stop storing the current-day feed in Cassandra - T169940 T176233
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-25T08:36:41Z] <mobrovac@tin> Finished deploy [restbase/deploy@ab24f70]: Switch mobile-sections to next-gen storage and stop storing the current-day feed in Cassandra - T169940 T176233 (duration: 10m 03s)
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-25T15:57:34Z] <urandom> T169940: Converting restbase-ng data tables to size-tiered compaction
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-26T14:27:15Z] <mobrovac@tin> Started deploy [restbase/deploy@f989dd9]: Remove the old mobileapps module completely - T169940
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-26T14:47:50Z] <mobrovac@tin> Finished deploy [restbase/deploy@f989dd9]: Remove the old mobileapps module completely - T169940 (duration: 20m 35s)
Comment Actions
We are currently double-writing mobileapps to the new environment, and collecting information on performance in order to plan next-steps; Closing this issue as done.