Page MenuHomePhabricator
Paste P7855

(An Untitled Masterwork)
ActivePublic

Authored by Marostegui on Nov 28 2018, 10:03 AM.
Tags
None
Referenced Files
F27321428:
Nov 28 2018, 10:03 AM
Subscribers
[10:01:33] marostegui@puppetmaster1001:~$ sudo -i puppet-merge
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
remote: Counting objects: 9365, done
remote: Finding sources: 100% (4/4)
remote: Getting sizes: 100% (3/3)
remote: Compressing objects: 100% (15373/15373)
remote: Total 4 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From https://gerrit.wikimedia.org/r/p/operations/puppet
3d8e892354..4caca6dc2a production -> origin/production
HEAD is 3d8e892354585464b28ec94bd807dfb2018fd799, FETCH_HEAD is 4caca6dc2a200f40f4e4186bf1da19d9da077bb1. Reviewing changes...
diff --git a/hieradata/hosts/pc1008.yaml b/hieradata/hosts/pc1008.yaml
index db7a5e5d5a..1c0af1da89 100644
--- a/hieradata/hosts/pc1008.yaml
+++ b/hieradata/hosts/pc1008.yaml
@@ -1,2 +1 @@
mariadb::parsercache::shard: 'pc2'
-profile::base::notifications: disabled
------------------------------
Marostegui: pc1008: Enable notifications (4caca6dc2a)
Merge these changes? (yes/no)? yes
Merging 4caca6dc2a200f40f4e4186bf1da19d9da077bb1...
git merge --ff-only 4caca6dc2a200f40f4e4186bf1da19d9da077bb1
Updating 3d8e892354..4caca6dc2a
Fast-forward
hieradata/hosts/pc1008.yaml | 1 -
1 file changed, 1 deletion(-)
Running git clean to clean any untracked files.
git clean -dffx -e /private/
HEAD is now 4caca6dc2a200f40f4e4186bf1da19d9da077bb1.
/root
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
HEAD is 3d8e892354585464b28ec94bd807dfb2018fd799, FETCH_HEAD is 4caca6dc2a200f40f4e4186bf1da19d9da077bb1. Reviewing changes...
------------------------------
Merging 4caca6dc2a200f40f4e4186bf1da19d9da077bb1...
git merge --ff-only 4caca6dc2a200f40f4e4186bf1da19d9da077bb1
Updating 3d8e892354..4caca6dc2a
Fast-forward
hieradata/hosts/pc1008.yaml | 1 -
1 file changed, 1 deletion(-)
Running git clean to clean any untracked files.
git clean -dffx -e /private/
HEAD is now 4caca6dc2a200f40f4e4186bf1da19d9da077bb1.
/var/lib/git/operations/puppet
Connection to puppetmaster1002.eqiad.wmnet closed.
OK: puppet-merge on puppetmaster1002.eqiad.wmnet succeeded
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
error: cannot lock ref 'refs/remotes/origin/production': is at 4caca6dc2a200f40f4e4186bf1da19d9da077bb1 but expected d83ccdff2534a9ff335c40930c870a165ae5ea25
From https://gerrit.wikimedia.org/r/p/operations/puppet
! d83ccdff25..4caca6dc2a production -> origin/production (unable to update local ref)
Connection to rhodium.eqiad.wmnet closed.
ERROR: puppet-merge on rhodium.eqiad.wmnet failed
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
error: cannot lock ref 'refs/remotes/origin/production': is at 4caca6dc2a200f40f4e4186bf1da19d9da077bb1 but expected d83ccdff2534a9ff335c40930c870a165ae5ea25
From https://gerrit.wikimedia.org/r/p/operations/puppet
! d83ccdff25..4caca6dc2a production -> origin/production (unable to update local ref)
Connection to puppetmaster2001.codfw.wmnet closed.
ERROR: puppet-merge on puppetmaster2001.codfw.wmnet failed
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
error: cannot lock ref 'refs/remotes/origin/production': is at 4caca6dc2a200f40f4e4186bf1da19d9da077bb1 but expected d83ccdff2534a9ff335c40930c870a165ae5ea25
From https://gerrit.wikimedia.org/r/p/operations/puppet
! d83ccdff25..4caca6dc2a production -> origin/production (unable to update local ref)
Connection to puppetmaster2002.codfw.wmnet closed.
ERROR: puppet-merge on puppetmaster2002.codfw.wmnet failed
Now running conftool-merge to sync any changes to conftool data
Running conftool-sync on /etc/conftool/data
2018-11-28 10:02:29 [INFO] conftool::yaml_log_error: Error parsing yaml file /etc/conftool/etcdrc: [Errno 2] No such file or directory: '/etc/conftool/etcdrc'
2018-11-28 10:02:29 [INFO] conftool::load_files: Loading data for entity node from /etc/conftool/data
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/eqsin.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/codfw.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/ulsfo.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/esams.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/eqiad.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Loading data for entity mwconfig from /etc/conftool/data
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/mwconfig/data.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Loading data for entity discovery from /etc/conftool/data
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/discovery/mediawiki.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/discovery/services.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Loading data for entity service from /etc/conftool/data
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/mediawiki.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/services.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/cache.yaml
2018-11-28 10:02:30 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/misc.yaml
2018-11-28 10:02:30 [INFO] conftool::load: Adding objects for service
2018-11-28 10:02:30 [INFO] conftool::load: Adding objects for node
2018-11-28 10:02:30 [INFO] conftool::load: Adding objects for mwconfig
2018-11-28 10:02:30 [INFO] conftool::load: Adding objects for discovery
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for discovery
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for mwconfig
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for node
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for service
Done.

Event Timeline

banyek@puppetmaster1001:~ $ sudo -i puppet-merge
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
remote: Counting objects: 9361, done
remote: Finding sources: 100% (4/4)
remote: Getting sizes: 100% (3/3)
remote: Compressing objects: 100% (15373/15373)
remote: Total 4 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From https://gerrit.wikimedia.org/r/p/operations/puppet
   d83ccdff25..3d8e892354  production -> origin/production
HEAD is d83ccdff2534a9ff335c40930c870a165ae5ea25, FETCH_HEAD is 3d8e892354585464b28ec94bd807dfb2018fd799.  Reviewing changes...
diff --git a/hieradata/hosts/dbproxy1011.yaml b/hieradata/hosts/dbproxy1011.yaml
index 516c91da22..c30c3827aa 100644
--- a/hieradata/hosts/dbproxy1011.yaml
+++ b/hieradata/hosts/dbproxy1011.yaml
@@ -1,5 +1,5 @@
-profile::mariadb::proxy::master::primary_name: 'labsdb1009'
-profile::mariadb::proxy::master::primary_addr: '10.64.4.14:3306'
-profile::mariadb::proxy::master::secondary_name: 'labsdb1010'
-profile::mariadb::proxy::master::secondary_addr: '10.64.37.23:3306'
+profile::mariadb::proxy::master::primary_name: 'labsdb1010'
+profile::mariadb::proxy::master::primary_addr: '10.64.37.23:3306'
+profile::mariadb::proxy::master::secondary_name: 'labsdb1009'
+profile::mariadb::proxy::master::secondary_addr: '10.64.4.14:3306'
 profile::mariadb::proxy::firewall: 'disabled'


------------------------------
Banyek: wiki replicas: depool labsdb1009 for upgrades (3d8e892354)
Merge these changes? (yes/no)? yes
Merging 3d8e892354585464b28ec94bd807dfb2018fd799...
git merge --ff-only 3d8e892354585464b28ec94bd807dfb2018fd799
Updating d83ccdff25..3d8e892354
Fast-forward
 hieradata/hosts/dbproxy1011.yaml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Running git clean to clean any untracked files.
git clean -dffx -e /private/
HEAD is now 3d8e892354585464b28ec94bd807dfb2018fd799.
/root
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
remote: Counting objects: 9365, done
remote: Finding sources: 100% (8/8)
remote: Getting sizes: 100% (6/6)
remote: Compressing objects: 100% (30746/30746)
remote: Total 8 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (8/8), done.
From https://gerrit.wikimedia.org/r/p/operations/puppet
   d83ccdff25..4caca6dc2a  production -> origin/production
HEAD is d83ccdff2534a9ff335c40930c870a165ae5ea25, FETCH_HEAD is 3d8e892354585464b28ec94bd807dfb2018fd799.  Reviewing changes...


------------------------------
Merging 3d8e892354585464b28ec94bd807dfb2018fd799...
git merge --ff-only 3d8e892354585464b28ec94bd807dfb2018fd799
Updating d83ccdff25..3d8e892354
Fast-forward
 hieradata/hosts/dbproxy1011.yaml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Running git clean to clean any untracked files.
git clean -dffx -e /private/
HEAD is now 3d8e892354585464b28ec94bd807dfb2018fd799.
/var/lib/git/operations/puppet
Connection to puppetmaster1002.eqiad.wmnet closed.
OK: puppet-merge on puppetmaster1002.eqiad.wmnet succeeded
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
remote: Counting objects: 9365, done
remote: Finding sources: 100% (8/8)
remote: Getting sizes: 100% (6/6)
remote: Compressing objects: 100% (30746/30746)
remote: Total 8 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (8/8), done.
From https://gerrit.wikimedia.org/r/p/operations/puppet
   d83ccdff25..4caca6dc2a  production -> origin/production
HEAD is d83ccdff2534a9ff335c40930c870a165ae5ea25, FETCH_HEAD is 3d8e892354585464b28ec94bd807dfb2018fd799.  Reviewing changes...


------------------------------
Merging 3d8e892354585464b28ec94bd807dfb2018fd799...
git merge --ff-only 3d8e892354585464b28ec94bd807dfb2018fd799
Updating d83ccdff25..3d8e892354
Fast-forward
 hieradata/hosts/dbproxy1011.yaml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Running git clean to clean any untracked files.
git clean -dffx -e /private/
HEAD is now 3d8e892354585464b28ec94bd807dfb2018fd799.
/var/lib/git/operations/puppet
Connection to rhodium.eqiad.wmnet closed.
OK: puppet-merge on rhodium.eqiad.wmnet succeeded
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
remote: Counting objects: 9365, done
remote: Finding sources: 100% (8/8)
remote: Getting sizes: 100% (6/6)
remote: Compressing objects: 100% (30746/30746)
remote: Total 8 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (8/8), done.
From https://gerrit.wikimedia.org/r/p/operations/puppet
   d83ccdff25..4caca6dc2a  production -> origin/production
HEAD is d83ccdff2534a9ff335c40930c870a165ae5ea25, FETCH_HEAD is 3d8e892354585464b28ec94bd807dfb2018fd799.  Reviewing changes...


------------------------------
Merging 3d8e892354585464b28ec94bd807dfb2018fd799...
git merge --ff-only 3d8e892354585464b28ec94bd807dfb2018fd799
Updating d83ccdff25..3d8e892354
Fast-forward
 hieradata/hosts/dbproxy1011.yaml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Running git clean to clean any untracked files.
git clean -dffx -e /private/
HEAD is now 3d8e892354585464b28ec94bd807dfb2018fd799.
/var/lib/git/operations/puppet
Connection to puppetmaster2001.codfw.wmnet closed.
OK: puppet-merge on puppetmaster2001.codfw.wmnet succeeded
Fetching new commits from https://gerrit.wikimedia.org/r/p/operations/puppet
remote: Counting objects: 9365, done
remote: Finding sources: 100% (8/8)
remote: Getting sizes: 100% (6/6)
remote: Compressing objects: 100% (30746/30746)
remote: Total 8 (delta 3), reused 0 (delta 0)
Unpacking objects: 100% (8/8), done.
From https://gerrit.wikimedia.org/r/p/operations/puppet
   d83ccdff25..4caca6dc2a  production -> origin/production
HEAD is d83ccdff2534a9ff335c40930c870a165ae5ea25, FETCH_HEAD is 3d8e892354585464b28ec94bd807dfb2018fd799.  Reviewing changes...


------------------------------
Merging 3d8e892354585464b28ec94bd807dfb2018fd799...
git merge --ff-only 3d8e892354585464b28ec94bd807dfb2018fd799
Updating d83ccdff25..3d8e892354
Fast-forward
 hieradata/hosts/dbproxy1011.yaml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Running git clean to clean any untracked files.
git clean -dffx -e /private/
HEAD is now 3d8e892354585464b28ec94bd807dfb2018fd799.
/var/lib/git/operations/puppet
Connection to puppetmaster2002.codfw.wmnet closed.
OK: puppet-merge on puppetmaster2002.codfw.wmnet succeeded
Now running conftool-merge to sync any changes to conftool data
Running conftool-sync on /etc/conftool/data
2018-11-28 10:02:29 [INFO] conftool::yaml_log_error: Error parsing yaml file /etc/conftool/etcdrc: [Errno 2] No such file or directory: '/etc/conftool/etcdrc'
2018-11-28 10:02:29 [INFO] conftool::load_files: Loading data for entity service from /etc/conftool/data
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/mediawiki.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/services.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/cache.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/service/misc.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Loading data for entity node from /etc/conftool/data
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/eqsin.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/codfw.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/ulsfo.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/esams.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/node/eqiad.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Loading data for entity discovery from /etc/conftool/data
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/discovery/mediawiki.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/discovery/services.yaml
2018-11-28 10:02:29 [INFO] conftool::load_files: Loading data for entity mwconfig from /etc/conftool/data
2018-11-28 10:02:29 [INFO] conftool::load_files: Parsing file /etc/conftool/data/mwconfig/data.yaml
2018-11-28 10:02:29 [INFO] conftool::load: Adding objects for service
2018-11-28 10:02:29 [INFO] conftool::load: Adding objects for node
2018-11-28 10:02:30 [INFO] conftool::load: Adding objects for discovery
2018-11-28 10:02:30 [INFO] conftool::load: Adding objects for mwconfig
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for mwconfig
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for discovery
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for node
2018-11-28 10:02:30 [INFO] conftool::load: Removing stale objects for service
Done.
banyek@puppetmaster1001:~ $ exit
logout