Page MenuHomePhabricator
Create Task
Maniphest T87220

Minimize infrastructure differences between Beta Cluster and production
Open, MediumPublic
Actions

Description

Only fundamental difference between beta and prod is that:

  1. Beta has code that is continuously deployed (auto update)
  2. Beta has jenkins access
  3. Beta syncs come page resources from production

*All* other differences have to go, such as:

  1. The entire 'beta' module
  2. All ::beta roles
  3. Realm branching in any mediawiki related module
  4. Fundamental technology differences (Swift vs NFS)

This is a tracking bug for tracking this.

Related Objects
Search...

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
· · ·
OpenNoneT53494 Use Beta cluster as a true canary for code deployments (epic)
OpenNoneT87220 Minimize infrastructure differences between Beta Cluster and production
ResolvedyuvipandaT87214 Migrate beta cluster jobrunner to HAT
Resolved• mmodellT87003 New disk partition scheme for labs instances
ResolvedyuvipandaT87210 Remove beta specific mediawiki roles
DeclinedNoneT87199 Set 'cluster' salt grain appropriately for all instances in beta cluster
ResolvedyuvipandaT86644 Remove all ::beta roles in puppet
ResolvedJoeT1256 Rework beta apache config
ResolvedyuvipandaT86575 deployment-mx is its own puppetmaster
ResolvedyuvipandaT76392 Reduce [LOCAL HACK] changes on Beta Cluster to zero
ResolvedyuvipandaT87221 Move scap puppet code into a module
ResolvedyuvipandaT87223 Move deployment-prep hiera data values into ops/puppet.git repo
ResolvedyuvipandaT87616 Set up an alert for unmerged changes in deployment-prep
ResolvedyuvipandaT92367 Use keyholder for deploy key management
Resolved• AlexMonk-WMFT38891 Setup poolcounter daemon in Beta Cluster
ResolvedKrenairT84950 Thumbnail generation should happen via the same setup in the beta cluster and in production (tracking)
OpenNoneT196662 Set up LVS in beta like prod
OpenNoneT278007 Configure etcd/confd/conftool in beta/deployment-prep like production
Resolved• taaviT198673 Remove deployment.wikimedia.beta.wmflabs.org wiki (deploymentwiki)
ResolvedKrinkleT99156 Rename test.wikimedia (in Beta) to test.wikipedia (matching prod)
ResolvedJoeT196968 Re-organize the apache configuration for MediaWiki in puppet
DeclinedNoneT142255 Move mathoid to deployment-sca* hosts in Beta Cluster
OpenNoneT206158 Investigate setting up HTTPS directly on beta appservers
StalledNoneT211677 Support IPv6 in beta
OpenNoneT276650 Re-consider setting up a Kubernetes cluster on the Beta cluster
Resolved• taaviT277432 Enable CentralAuth IRC feed in beta cluster
OpenNoneT277990 Swift uses http in deployment-prep, https in production
OpenNoneT280393 Rename deployment-cache-(text|upload)0x to deployment-cp0x
Resolved• taaviT110115 Possible to run writes (e.g. UPDATE) on Beta Cluster replica
ResolvedKrinkleT281650 Move upload.beta.wmflabs.org to upload.wikimedia.beta.wmflabs.org
ResolvedZabeT318126 Migrate deployment-prep db hosts to bullseye
· · ·

Event Timeline

yuvipanda created this task.Jan 20 2015, 8:19 AM
yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added projects: Beta-Cluster-Infrastructure, Puppet, acl*sre-team.
yuvipanda added subscribers: yuvipanda, Joe.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 20 2015, 8:19 AM
yuvipanda added subtasks: T1256: Rework beta apache config, T87214: Migrate beta cluster jobrunner to HAT, T86633: Unify ::production / ::beta roles for *oid, T87210: Remove beta specific mediawiki roles.Jan 20 2015, 8:22 AM
yuvipanda added a subtask: T87199: Set 'cluster' salt grain appropriately for all instances in beta cluster.
yuvipanda edited subtasks, added: T86644: Remove all ::beta roles in puppet, T86575: deployment-mx is its own puppetmaster, T76392: Reduce [LOCAL HACK] changes on Beta Cluster to zero; removed: T86633: Unify ::production / ::beta roles for *oid.Jan 20 2015, 8:25 AM
yuvipanda added a subtask: T87221: Move scap puppet code into a module.Jan 20 2015, 8:28 AM
yuvipanda added a subtask: T87223: Move deployment-prep hiera data values into ops/puppet.git repo.Jan 20 2015, 8:30 AM
yuvipanda renamed this task from Minimize differences between beta and production to Minimize differences between beta and production (Tracking).Jan 20 2015, 8:36 AM
yuvipanda set Security to None.
yuvipanda updated the task description. (Show Details)
yuvipanda claimed this task.Jan 24 2015, 9:21 AM
• mmodell subscribed.Jan 26 2015, 10:02 PM
• demon subscribed.Jan 26 2015, 10:10 PM
greg subscribed.Jan 27 2015, 5:20 PM
Dzahn triaged this task as Medium priority.Jan 28 2015, 5:51 PM
Dzahn subscribed.
greg moved this task from To Triage to In-progress on the Beta-Cluster-Infrastructure board.Jan 28 2015, 10:36 PM
yuvipanda closed subtask T87223: Move deployment-prep hiera data values into ops/puppet.git repo as Resolved.Jan 29 2015, 1:59 AM
yuvipanda closed subtask T86575: deployment-mx is its own puppetmaster as Resolved.Jan 29 2015, 8:58 AM
yuvipanda closed subtask T87616: Set up an alert for unmerged changes in deployment-prep as Resolved.Feb 2 2015, 12:24 PM
yuvipanda added a subtask: T87003: New disk partition scheme for labs instances.Feb 3 2015, 2:15 PM
yuvipanda closed subtask T87003: New disk partition scheme for labs instances as Resolved.Feb 3 2015, 6:11 PM
yuvipanda closed subtask T87210: Remove beta specific mediawiki roles as Resolved.Feb 4 2015, 1:46 PM
yuvipanda closed subtask T87214: Migrate beta cluster jobrunner to HAT as Resolved.Feb 4 2015, 3:11 PM
Joe closed subtask T76392: Reduce [LOCAL HACK] changes on Beta Cluster to zero as Resolved.Feb 17 2015, 5:07 PM
yuvipanda closed subtask T92367: Use keyholder for deploy key management as Resolved.Mar 14 2015, 7:21 AM
yuvipanda closed subtask T87221: Move scap puppet code into a module as Resolved.Mar 14 2015, 7:25 AM
Liuxinyu970226 subscribed.Apr 18 2015, 1:31 AM
yuvipanda removed yuvipanda as the assignee of this task.Jun 7 2015, 5:14 PM
hashar moved this task from In-progress to Next: Maintenance on the Beta-Cluster-Infrastructure board.Jun 15 2015, 7:06 PM
Nemo_bis added a parent task: T53494: Use Beta cluster as a true canary for code deployments (epic).Jul 4 2015, 9:28 AM
Legoktm subscribed.Oct 24 2015, 6:11 AM
Legoktm unsubscribed.
hashar moved this task from Next: Maintenance to Backlog on the Beta-Cluster-Infrastructure board.Nov 2 2015, 9:12 PM
JanZerebecki subscribed.Jan 27 2016, 9:19 AM
Aklapper mentioned this in T86644: Remove all ::beta roles in puppet.Feb 14 2016, 3:58 PM
Danny_B added a project: Tracking-Neverending.May 5 2016, 7:38 PM
greg added a subtask: T38891: Setup poolcounter daemon in Beta Cluster.Jul 13 2016, 9:18 PM
greg added a project: Technical-Debt.
MaxSem moved this task from Unsorted to Cleanup on the Technical-Debt board.Jul 17 2016, 6:16 PM
greg moved this task from Backlog to Epics / Tracking on the Beta-Cluster-Infrastructure board.Aug 5 2016, 8:49 PM
• AlexMonk-WMF closed subtask T38891: Setup poolcounter daemon in Beta Cluster as Resolved.Aug 24 2016, 11:12 PM
Krenair added a subtask: T84950: Thumbnail generation should happen via the same setup in the beta cluster and in production (tracking).Aug 24 2016, 11:34 PM
hashar closed subtask T86644: Remove all ::beta roles in puppet as Resolved.Nov 24 2016, 8:24 AM
Krenair reopened subtask T86644: Remove all ::beta roles in puppet as Open.Nov 24 2016, 8:28 AM
• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:55 PM
hashar closed subtask T87199: Set 'cluster' salt grain appropriately for all instances in beta cluster as Declined.Jun 29 2017, 7:59 AM
Krinkle mentioned this in T174786: Provide a reliable test environment that mimics production for running integration tests.Sep 1 2017, 8:08 PM
hashar closed subtask T86644: Remove all ::beta roles in puppet as Resolved.Oct 23 2017, 11:03 PM
Krinkle added a subtask: T198673: Remove deployment.wikimedia.beta.wmflabs.org wiki (deploymentwiki).Jul 13 2018, 12:02 AM
Krinkle added a subtask: T99156: Rename test.wikimedia (in Beta) to test.wikipedia (matching prod).
Krinkle added a subtask: T196968: Re-organize the apache configuration for MediaWiki in puppet.Jul 13 2018, 12:04 AM
Krenair updated the task description. (Show Details)Jul 30 2018, 7:59 PM
Krenair updated the task description. (Show Details)
Krenair added a subtask: T142255: Move mathoid to deployment-sca* hosts in Beta Cluster.Jul 30 2018, 8:09 PM
Krenair closed subtask T84950: Thumbnail generation should happen via the same setup in the beta cluster and in production (tracking) as Resolved.Jul 30 2018, 9:17 PM
• mobrovac closed subtask T142255: Move mathoid to deployment-sca* hosts in Beta Cluster as Declined.Jul 31 2018, 8:04 AM
Krenair added a subtask: T206158: Investigate setting up HTTPS directly on beta appservers.Oct 3 2018, 7:02 PM
Joe closed subtask T196968: Re-organize the apache configuration for MediaWiki in puppet as Resolved.Oct 22 2018, 1:21 PM
Krenair closed subtask T1256: Rework beta apache config as Resolved.Oct 22 2018, 2:17 PM
Krenair added a subtask: T211677: Support IPv6 in beta.Dec 11 2018, 11:18 AM
• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 7:58 PM
AfroThundr3007730 subscribed.Jun 5 2019, 2:54 AM
Krinkle closed subtask T99156: Rename test.wikimedia (in Beta) to test.wikipedia (matching prod) as Resolved.Jun 28 2020, 10:34 PM
• taavi added a subtask: T276650: Re-consider setting up a Kubernetes cluster on the Beta cluster.Mar 6 2021, 7:34 AM
• taavi added a subtask: T277432: Enable CentralAuth IRC feed in beta cluster.Mar 18 2021, 9:46 AM
• taavi closed subtask T277432: Enable CentralAuth IRC feed in beta cluster as Resolved.Mar 18 2021, 6:11 PM
Krinkle mentioned this in T167467: Create nova service account for openstack.Mar 18 2021, 6:48 PM
Krinkle renamed this task from Minimize differences between beta and production (Tracking) to Minimize infrastructure differences between Beta Cluster and production.Mar 18 2021, 6:55 PM
Krinkle removed a subtask: T212244: Set up a process for regularly updating the Beta Cluster's copies of common.js, gadgets, and similar files.
Krinkle mentioned this in T212244: Set up a process for regularly updating the Beta Cluster's copies of common.js, gadgets, and similar files.
Krinkle removed a project: Puppet.
• taavi added a subtask: T277990: Swift uses http in deployment-prep, https in production.Mar 20 2021, 7:41 AM
• taavi mentioned this in T278007: Configure etcd/confd/conftool in beta/deployment-prep like production.Mar 20 2021, 1:13 PM
• taavi added a subtask: T278007: Configure etcd/confd/conftool in beta/deployment-prep like production.
Reedy changed the status of subtask T211677: Support IPv6 in beta from Open to Stalled.Mar 27 2021, 12:35 AM
Krinkle mentioned this in T59583: Make deployment prep have continuous replication lag.Apr 16 2021, 7:18 PM
Krinkle mentioned this in T40945: Need a way to simulate replication lag to test replag issues.
Krinkle added a subtask: T110115: Possible to run writes (e.g. UPDATE) on Beta Cluster replica.May 1 2021, 4:34 PM
• taavi closed subtask T110115: Possible to run writes (e.g. UPDATE) on Beta Cluster replica as Resolved.May 5 2021, 3:49 PM
Krinkle closed subtask T281650: Move upload.beta.wmflabs.org to upload.wikimedia.beta.wmflabs.org as Resolved.Jun 2 2022, 1:52 AM
Zabe closed subtask T318126: Migrate deployment-prep db hosts to bullseye as Resolved.Sep 22 2022, 9:22 PM
Zabe closed subtask T198673: Remove deployment.wikimedia.beta.wmflabs.org wiki (deploymentwiki) as Resolved.Mar 2 2023, 12:20 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL